Lecture Notes for MATH6106

March 25, 2010

1

Contents

1 Vectors 41.1 Points in Space . . . . . . . . . . . . . . . . . . . . . . . . . . 41.2 Distance between Points . . . . . . . . . . . . . . . . . . . . . 41.3 Scalars and Vectors . . . . . . . . . . . . . . . . . . . . . . . . 51.4 Vectors and Coordinate Systems . . . . . . . . . . . . . . . . . 51.5 Vector Operations . . . . . . . . . . . . . . . . . . . . . . . . . 71.6 Revision of Vectors . . . . . . . . . . . . . . . . . . . . . . . . 121.7 Quiz . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2 Topics in 3D Geometry 192.1 Planes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192.2 Lines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202.3 Surfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212.4 Functions of several variables . . . . . . . . . . . . . . . . . . 232.5 Change of Coordinates . . . . . . . . . . . . . . . . . . . . . . 25

3 Partial Derivatives 313.1 First Order Partial Derivatives . . . . . . . . . . . . . . . . . . 313.2 Higher Order Partial Derivatives . . . . . . . . . . . . . . . . . 333.3 Chain Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353.4 Maxima and Minima . . . . . . . . . . . . . . . . . . . . . . . 38

4 A little Vector Calculus 434.1 Gradient . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434.2 Directional Derivative . . . . . . . . . . . . . . . . . . . . . . 454.3 Curl and Divergence . . . . . . . . . . . . . . . . . . . . . . . 484.4 Laplacian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

5 Linear Equations 565.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 565.2 Row Echelon Form . . . . . . . . . . . . . . . . . . . . . . . . 575.3 Soving Systems of Linear Equations . . . . . . . . . . . . . . . 59

6 Linear Independence and Gram Schmidt 646.1 Linear Combination of Vectors . . . . . . . . . . . . . . . . . . 646.2 Linear Independence of Vectors . . . . . . . . . . . . . . . . . 666.3 Orthogonal and orthonormal vectors . . . . . . . . . . . . . . 68

6.4 Gram-Schmidt Process . . . . . . . . . . . . . . . . . . . . . . 70

7 Matrices 747.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 747.2 Matrix Operations . . . . . . . . . . . . . . . . . . . . . . . . 747.3 Special matrices . . . . . . . . . . . . . . . . . . . . . . . . . . 767.4 Determinants . . . . . . . . . . . . . . . . . . . . . . . . . . . 777.5 Inverses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

8 Eigenvalues, Eigenvectors and Diagonalization 838.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 838.2 Method to find eigenvalues and eigenvectors . . . . . . . . . . 838.3 Dimension of Eigenspace . . . . . . . . . . . . . . . . . . . . . 978.4 Diagonalization . . . . . . . . . . . . . . . . . . . . . . . . . . 1018.5 Diagonalization of symmetric matrices . . . . . . . . . . . . . 105

9 Operators and Commutators 1129.1 Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1129.2 Linear Operators . . . . . . . . . . . . . . . . . . . . . . . . . 1149.3 Composing Operators . . . . . . . . . . . . . . . . . . . . . . . 1179.4 Commutators . . . . . . . . . . . . . . . . . . . . . . . . . . . 118

10 Fourier Series 12510.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12510.2 Definition of a Fourier Series . . . . . . . . . . . . . . . . . . . 12510.3 Why the coefficients are found as they are. . . . . . . . . . . . 12610.4 Fourier Series . . . . . . . . . . . . . . . . . . . . . . . . . . . 12810.5 Odd and even functions . . . . . . . . . . . . . . . . . . . . . 13210.6 Functions with period 2L . . . . . . . . . . . . . . . . . . . . . 13610.7 Parseval’s Result . . . . . . . . . . . . . . . . . . . . . . . . . 139

1 Vectors

1.1 Points in Space

A point in two dimensional (2D) space is represented on the XY -plane usingtwo coordinates (x, y).

Example. Plot P (1, 2).

3D Coordinate system

The coordinate system in 3D space consists of three axes X,Y and Z. Todetermine the directions of these axes, we follow the right hand rule:We first define a plane in 3D space as the XY plane. Then place your righthand on the X-axis, curl your fingers in the direction of the Y -axis, and letyour thumb stick out. Your thumb then points in the direction of the positiveZ axis.

A point in three dimensional (3D) space is represented using three coor-dinates (x, y, z).

Example. Plot P (1, 2, 3).

1.2 Distance between Points

In 2D-space, the distance between two points P (x1, y1) and Q(x2, y2) is

d(P,Q) =√

(x2 − x1)2 + (y2 − y1)2.

Similarly in 3D-space, the distance between two points P (x1, y1, z1) andQ(x2, y2, z2) is

d(P,Q) =√

(x2 − x1)2 + (y2 − y1)2 + (z2 − z1)2.

Example. Find the distance between P (1, 2, 3) and Q(0, 0, 1).

d(P,Q) =√

(x2 − x1)2 + (y2 − y1)2 =√

(0− 1)2 + (0− 2)2 + (1− 3)2 = 3.

1.3 Scalars and Vectors

Certain quantities in physical applications of mathematics are representedby vectors. To understand what a vector is it is perhaps better to firstunderstand what a scalar is:

Definition. A scalar quantity is defined completely by a single number, e.g.length, area, volume, mass, time etc. It has only magnitude.

Definition. (Physical) A vector quantity is defined by magnitude anddirection, e.g. force, velocity, acceleration.

Example. 1. Force is reresented by a vector; the direction of the vectordescribes the direction in whch the force is applied, and the magnitudeof the vector indicates the strength applied.

2. Velocity of a car is represented by a vector; the direction gives thedirection of motion and the magnitude gives the speed of the car.

Note. Two vectors with the same length and same direction are equal, wedo not care about their position in space.

1.4 Vectors and Coordinate Systems

Vectors can occur in two, three or even higher dimensions.

Definition. (Geometric) A vector in 2D-space can be represented by adirected line segment, with an initial point and a terminal point. We denotea vector by an over head arrow, e.g. ~v. Then the magnitude of ~v is given bythe length of the directed line segment, and is denotd by |~v|, and the directionof ~v is given by the direction of the line segment.

Example. Plot the vector ~PQ with initial point P (2, 3) and terminal point

Q(−1, 4). Find the magnitude of the vector ~PQ.

| ~PQ| =√

(−1− 2)2 + (4− 3)2 =√

10

Example. We can also represent the vector ~PQ as the difference of thevector ~OQ and ~OP (Use diagram.)

~PQ = ~OQ− ~OP

= (−1, 4)− (2, 3)

= (−3, 1)

Then the directed line segment from O(0, 0) to R(−3, 1) also represents the

vector ~PQ. Verify that the magnitude of both line segments is the same.

| ~OR| =√

(−3− 0)2 + (1− 0)2 =√

10

The two vectors also point in the same direction.

Definition. (Algebraic) A vector in 2D-space is represented by two com-ponents (x, y), indicating the change in the X and Y directions from theorigin (0, 0).

Note. When we say a point P (−3, 1) in 2D-space, we mean the point withx-coordiante −3 and y-coordinate 1. When we say the vector ~v = (−3, 1)in 2D-space, we mean the directed line segment from the point (0, 0) to thepoint (−3, 1), which has magnitude or length

√(−3)2 + 12 =

√10 and points

in the direction of the line segment.

Definition. A vector in 3D-space is given by a three tuple (x, y, z), indi-cating the change in the X, Y and Z directions from the origin.

Note. We represent a vector ~v = (x, y, z) in 3D-space as the directedline segment from the point (0, 0, 0) to the point (x, y, z), which has length√x2 + y2 + z2 and points in the direction of the line segment.

Example. Plot the vector ~v = (1, 2, 3) and determine it’s length.

1.5 Vector Operations

Addition and Subtraction

We add and subtract vectors component-wise.

Example. Let ~v1 = (1, 0, 1) and ~v2 = (−1, 2, 3). Then

~v1 + ~v2 = (1− 1, 0 + 2, 1 + 3) = (0, 2, 4)

and~v1 − ~v2 = (1− (−1), 0− 2, 1− 3) = (2,−2,−2).

Scalar Multiplication

If c is a real number and ~v = (x, y, z), then

c~v = (cx, cy, cz).

This is known as scaling - we change the length but not the direction of v.

|c~v| =√c2x2 + c2y2 + c2z2 = c

√x2 + y2 + z2 = c|~v|

Special vectors

The zero vector: ~o = (0, 0, 0).

Definition. A unit vector is a vector with length equal to one. i.e. |~v| = 1.

In 2D space, the unit vectors in theX and Y direction are special and denotedby:

~i = (1, 0)~j = (0, 1)

Similarly, in 3D space, the unit vectors in the X, Y and Z directions aredenoted by:

~i = (1, 0, 0)~j = (0, 1, 0)

~k = (0, 0, 1)

Example. Using the special unit vectors i, j, k, we get a different way ofrepresenting a vector,

~v = (x, y, z) = x~i+ y~j + z~k.

Consider ~v = (13, 6,−8). Then,

~v = (13, 6,−8)

= 13(1, 0, 0) + 6(0, 1, 0)− 8(0, 0, 1)

= 13~i+ 6~j − 8~k

This representation is often useful and we will use both notations in thecourse.

Example. Find a unit vector in the direction of the vector ~v = (6, 8, 10).

Dot Product

Definition. The dot product of 2 vectors ~a = (a1, a2, a3) and~b = (b1, b2, b3)in 3-dimensional space is defined to be the scalar

~a.~b = a1b1 + a2b2 + a3b3.

This is the algebraic definition of the dot product

On the other hand, if θ is the angle between the vectors ~a and ~b, using theLaw of Cosines on the triangle with sides ~a, ~b, ~a−~b we get,

|~a−~b|2 = |~a|2 + |~b|2 − 2|~a|.|~b| cos θ

(a1 − b1)2 + (a2 − b2)

2 + (a3 − b3)2 = a2

1 + a22 + a2

3 + b21 + b22 + b23 − 2|~a|.|~b| cos θ

a1b1 + a2b2 + a3b3 = |~a|.|~b| cos θ

~a.~b = |~a|.|~b| cos θ

Definition. Therefore we could equivalenty define the dot product to be

~a.~b = |~a|.|~b| cos θ.

This is the geometric definition of the dot product.

Remark. The two main applications of taking dot products;

(i) to find the angle between vectors and

(ii) to determine if two vectors are perpendicular.

Example. Find the dot product of ~a = (1,−1,√

3) and ~b = (2,−1, 0). Alsofind the angle between the two vectors.

Solution:

~a.~b = 1.2 + (−1)(−1) +√

3.0 = 3

|~a| =√

1 + 1 + 3 =√

5

|~b| =√

4 + 1 =√

5

cos θ =~a.~b

|~a||~b|= 3/5

θ = cos−1(3

5)

Proposition Let ~a and ~b be non-zero vectors. The vectors, ~a and ~b, areperpendicular to each other if and only if ~a ·~b = 0.

Solution: Let ~a and ~b be non zero vectors, then |~a| 6= 0 and |~b| 6= 0. Let θbe the angle between the two vectors.

~a ·~b = 0

⇔ |~a|.|~b| cos θ = 0

⇔ cos θ = 0; Since |~a| 6= 0, |~b| 6= 0, we can divide the equation by them.

⇔ θ = π/2

Therefore the angle between ~a and ~b is 90 degrees.

Example. Determine if the vectors ~a = (1,−1, 1) and ~b = (1, 2, 1) are per-pendicular to each other.

~a ·~b = (1)(1) + (−1)(2) + (1)(1) = 0

Therefore by the above proporsition, the vectors are perpendicular toeach other.

Cross Product

Definition. Given 2 vectors ~a = (a1, a2, a3) and~b = (b1, b2, b3) in 3-dimensional

space the cross product ~a×~b is a vector defined as follows:

~a×~b = (|~a|.|~b| sin θ)~n

where ~n is a unit vector which is perpendicular to the plane containing ~a and~b, determined by the right-hand-rule. i.e. place your right hand on ~a andcurl it towards ~b, ~n now points in the direction of your thumb. This is thegeometric definition of the cross-product.

We also have an algebraic definition of the cross-product, but before we candefine it, we need to be able to compute determinants!

Definition. The determinant of a 2× 2 matrix is defined as follows:∣∣∣∣ a bc d

∣∣∣∣ = ad− bc.

Example. Compute the determinant.∣∣∣∣ 1 3−5 6

∣∣∣∣ = 1.6− 3(−5) = 9.

Definition. The determinant of a 3× 3 matrix is defined as follows:∣∣∣∣∣∣a11 a12 a13

a21 a22 a23

a31 a32 a33

∣∣∣∣∣∣ = a11

∣∣∣∣ a22 a23

a32 a33

∣∣∣∣− a12

∣∣∣∣ a21 a23

a31 a33

∣∣∣∣ + a13

∣∣∣∣ a21 a22

a31 a33

∣∣∣∣= a11(a22a33 − a32a23)− a12(a21a33 − a31a23) + a13(a21a32 − a31a22)

Example. Compute the determinant.∣∣∣∣∣∣4 2 1−2 −6 3−7 5 0

∣∣∣∣∣∣ = 4

∣∣∣∣ −6 35 0

∣∣∣∣− 2

∣∣∣∣ −2 3−7 0

∣∣∣∣ + 1

∣∣∣∣ −2 −6−7 5

∣∣∣∣= 4(−15)− 2(21) + (1)(−52)

= −154

Definition. Given 2 vectors ~a = (a1, a2, a3) and~b = (b1, b2, b3) in 3-dimensional

space the cross product ~a×~b can also be defined as follows:

~a×~b =

∣∣∣∣∣∣~i ~j ~ka1 a2 a3

b1 b2 b3

∣∣∣∣∣∣where ~i = (1, 0, 0), ~j = (0, 1, 0), ~k = (0, 0, 1) are unit vectors along the X,Y , Z axes respectively. However while calculating the determinant they onlyhave symbolic value.

Remark. The cross product operates on 2 vectors and produces a new vec-tor.

Example. Consider the two vectors ~a = (2, 1, 0) and ~b = (−1, 1, 0) in 3-dimensional space. Find the cross-product of the two vectors and verify thatit is indeed perpendicular to both ~a and ~b. Also find the magnitude of ~a×~b.

Solution:

~a×~b =

∣∣∣∣∣∣~i ~j ~k2 1 0−1 1 0

∣∣∣∣∣∣=

∣∣∣∣ 1 01 0

∣∣∣∣~i− ∣∣∣∣ 2 0−1 0

∣∣∣∣~j +

∣∣∣∣ 2 1−1 1

∣∣∣∣~k= 0~i− 0~j + 3~k

= 0(1, 0, 0)− 0(0, 1, 0) + 3(0, 0, 1)

= (0, 0, 3)

Observe that the two vectors lie in the XY -plane and the cross product lieson the Z-axis and is indeed perpendicular to the plane containing the twovectors. But we can verify this more concretely using the dot product. Let~n = ~a×~b = (0, 0, 3).

~n · ~a = (2, 1, 0).(0, 0, 3) = 0 + 0 + 0 = 0.

~n ·~b = (−1, 1, 0).(0, 0, 3) = 0 + 0 + 0 = 0.

Since both the dot products are zero, we can conclude that ~a×~b is perpen-dicular to both ~a and ~b.Finally the magnitude of ~a×~b is given by,

|~a×~b| =√

0 + 0 + 9 = 3.

Remark. The two important properties of cross products are:

(1) The direction of the cross product ~a×~b is perpendicular to both ~a and~b. In particular it points in the direction of your thumb if the fingers ofyour right hand curl from ~a to ~b.

(2) The magnitude of ~a×~b is given by |~a|.|~b| sin θ, if θ is the angle between

~a and ~b. It is the area of the parallelogram determined by ~a and ~b.

1.6 Revision of Vectors

Example. Find a unit vector in the direction of the following vectors:

(i) ~v1 = (2,−3).

(ii) ~v2 = (1,−2, 5).

(iii) ~v3 = (1, 1, 1, 1).

Solution: To get a unit vector in the direction of a given vector, all weneed to do is scale the size of the vector by the inverse of its length.

(i) ~v1 = (2,−3).The length of ~v1 is |~v1| =

√22 + (−3)2 =

√13.

A unit vector u1 in the direction of v1 may then be obtained as follows:

u1 =1√13

(2,−3) = (2√13,−3√13

).

(ii) ~v2 = (1,−2, 5).The length of ~v2 is |~v2| =

√12 + (−2)2 + 52 =

√30.

A unit vector u2 in the direction of v2 may then be obtained as follows:

u2 =1√30

(1,−2, 5) = (1√30,−2√30,

5√30

).

(iii) ~v3 = (1, 1, 1, 1).The length of ~v3 is |~v3| =

√12 + 12 + 12 + 12 =

√4 = 2.

A unit vector u3 in the direction of v3 may then be obtained as follows:

u3 =1

2(1, 1, 1, 1) = (

1

2,1

2,1

2,1

2).

Example. Consider the two vectors ~a = (2, 1,−4) and ~b = (3,−2, 5) in3-dimensional space. Find the cross-product of the two vectors.

Solution:

~a×~b =

∣∣∣∣∣∣~i ~j ~k2 1 −43 −2 5

∣∣∣∣∣∣=

∣∣∣∣ 1 −4−2 5

∣∣∣∣~i− ∣∣∣∣ 2 −43 5

∣∣∣∣~j +

∣∣∣∣ 2 13 −2

∣∣∣∣~k= −3~i− 22~j − 7~k

= (−3,−22,−7)

1.7 Quiz

1. Find ddx

(tan(x2) cos(ex).

Solution:

d

dx(tan(x2) cos(ex) = tan(x2)

d

dx(cos(ex) + cos(ex)

d

dx(tan(x2))

= tan(x2)(− sin(ex))ex + cos(ex)(sec2(x2))2x

2. Find the stationary points of y = x3 − 3x.

Solution: Let f(x) = x3 − 3x. Since f(x) is continuous, the stationarypoints of f(x) are the solutions of the equation f ′(x) = 0.

f(x) = x3 − 3x

f ′(x) = 3x2 − 3

3x2 − 3 = 0

3(x− 1)(x+ 1) = 0

The stationary points are: {1,−1}.

3. Find∫x lnxdx.

Solution:

u = ln(x); v =x2

2

du =dx

x; dv = xdx∫

x ln(x)dx =

∫udv = uv −

∫vdu

=x2 ln(x)

2−

∫x2

2

dx

x

=x2 ln(x)

2−

∫x

2dx

=x2 ln(x)

2− x2

4+ C.

4. Find∫

(1− x2)−1/2dx.

Solution: ∫1√

a2 − u2du = sin−1(

u

a) + c.

Therefore, ∫(1− x2)−1/2dx = sin−1(x) + c.

5. Find the determinant of

(2 −1−6 2

).

Solution: ∣∣∣∣ 2 −1−6 2

∣∣∣∣ = (2)(2)− (−1)(−6) = −2

6. Let ~u = (2, 3, 5), ~v = (1, 1,−1). Find

(a) The unit vector in the direction of ~v = (1, 1,−1) is,

1√12 + 12 + (−1)2

(1, 1,−1) =1√3(1, 1,−1) = (

1√3,

1√3,−1√

3).

(b)

3~u− 5~v = 3(2, 3, 5)− 5(1, 1,−1) = (6, 9, 15)− (5, 5,−5) = (1, 4, 20).

(c)~u · ~v = (2, 3, 5).(1, 1,−1) = 2 + 3− 5 = 0.

(d)

~u× ~v =

∣∣∣∣∣∣~i ~j ~k2 3 51 1 −1

∣∣∣∣∣∣=

∣∣∣∣ 3 51 −1

∣∣∣∣~i− ∣∣∣∣ 2 51 −1

∣∣∣∣~j +

∣∣∣∣ 2 31 1

∣∣∣∣~k= (3(−1)− (5)(1))~i− (2(−1)− (5)(1))~j + ((2)(1)− (3)(1))~k

= −8~i+ 7~j − ~k= (−8, 7,−1)

7. Determine if ~u = (−2, 2, 1,−1) is perpendicular to ~v = (−2,−3, 1, 1).

Solution:

~u.~v = (−2)(−2) + (2)(−3) + (1)(1) + (−1)(1) = −2 6= 0.

Hence the two vectors are not perpendicular.

Summary

1. The dot product of 2 vectors ~a = (a1, a2, a3) and ~b = (b1, b2, b3) in3-dimensional space is defined to be the scalar

~a.~b = a1b1 + a2b2 + a3b3.

or~a.~b = |~a|.|~b| cos θ.

2. Let ~a and ~b be non-zero vectors. The vectors, ~a and ~b, are perpendic-ular to each other if and only if ~a ·~b = 0.

3. Given 2 vectors ~a = (a1, a2, a3) and ~b = (b1, b2, b3) in 3-dimensional space

the cross product ~a×~b can be defined as follows:

~a×~b =

∣∣∣∣∣∣~i ~j ~ka1 a2 a3

b1 b2 b3

∣∣∣∣∣∣where ~i = (1, 0, 0), ~j = (0, 1, 0), ~k = (0, 0, 1) are unit vectors along the X,Y , Z axes respectively.

4. The direction of the cross product ~a×~b is perpendicular to both~a and ~b. In particular it points in the direction of your thumb if thefingers of your right hand curl from ~a to ~b.

5. The magnitude of ~a×~b is given by |~a|.|~b| sin θ, if θ is the angle between ~a

and ~b. It is the area of the parallelogram determined by ~a and ~b.

References

1. A complete set of notes on Pre-Calculus, Single Variable Calculus, Mul-tivariable Calculus and Linear Algebra. Here is a link to the chapter onVectors.http://tutorial.math.lamar.edu/Classes/CalcII/VectorsIntro.aspx.

2. A collection of examples, animations and notes on Multivariable Calculus.http://people.usd.edu/ jflores/MultiCalc02/WebBook/Chapter13/Graphics/Chapter131/DemoHtml131/13.1coorsys.htm

3. Gallery of animated and graphical demonstrations of calculus and relatedtopics, from the University of Minnesota.http://www.math.umn.edu/ nykamp/m2374/readings/completelist.html.

4. Links to various resources on Calculus.http://www.calculus.org/.

5. Engineering Mathematics, by K. A. Stroud.

2 Topics in 3D Geometry

In two dimensional space, we can graph curves and lines. In three dimensionalspace, there is so much extra space that we can graph planes and surfaces inaddition to lines and curves. Here we will have a very brief introduction toGeometry in three dimensions.

2.1 Planes

Just as it is easy to write the equation of a line in 2D space, it is easy towrite the equation of a plane in 3D space.

The point-normal equation of a plane

A vector perpendicular to a plane is said to be normal to the plane and iscalled a normal vector, or simply a normal.

To write the equation of a plane we need a point P (x0, y0, z0) on the planeand a normal vector ~n = (a, b, c) to the plane.

Let P = (x0, y0, z0) be a point on the plane and ~n be a vector perpendicularto the plane. Then a point Q(x, y, z) lies on the plane,

⇔ the vector ~PQ lies on the plane,

⇔ ~PQ and ~n are perpendicular,

⇔ ~n · ~PQ = 0,

⇔ (a, b, c) · (x− x0, y − y0, z − z0) = 0,

⇔ a(x− x0) + b(y − y0) + c(z − z0) = 0.

Definition. The point-normal equation of a plane that contains the pointP (x0, y0, z0) and has normal vector ~n = (a, b, c) is

a(x− x0) + b(y − y0) + c(z − z0) = 0.

Example. Let P be a plane determined by the points A = (1, 2, 3), B =(2, 3, 4), and C = (−2, 0.3). Find a vector which is normal to the plane. Findan equation of the plane.

Solution: We need a point on the plane and a normal to the plane. Thevector ~AB× ~AC = (2,−3, 1) is a normal to the plane and we take A = (1, 2, 3)as a point on the plane (you can choose B or C instead of A if you want).The equation on the plane in point-normal form is:

2(x− 1)− 3(y − 2) + (z − 3) = 0

or equivalently,2x− 3y + z = −1

Observe that the coefficients of x, y and z are (2,−3, 1) which is the normalto the plane.

2.2 Lines

Vector equation of a line

To write the vector equation of a line, we need a point P (x0, y0, z0) on theline and a vector ~v = (a, b, c) that is parallel to the line.

Definition. The vector equation of a line that contains the point P (x0, y0, z0)and is parallel to the vector ~v = (a, b, c) is:

P + t~v = ~r, where t is scalar.

or,

(x0, y0, z0) + t(a, b, c) = (x, y, z)

(x0 + ta, y0 + tb, z0 + tc) = (x, y, z)

Parametric equation of a line

The parametric equation of a line is derived from the vector equation of aline.

Definition. The parametric equation of a line that contains the point P (x0, y0, z0)and is parallel to the vector ~v = (a, b, c) is:

x = x0 + ta

y = y0 + tb

z = z0 + tc

Example. Let L which passes through the points P (1, 1, 1) and Q(3, 2, 1).Find a vector which is parallel to the line. Find the vector-equation andparametric equation of the line.

Solution: The vector ~PQ = (2, 1, 0) is parallel to the line and we take thepoint P (1, 1, 1) on the line.The vector equation of the line:

(1, 1, 1) + t(2, 1, 0) = (x, y, z)

The parametric equations of the line:

x = 1 + 2t

y = 1 + t

z = 1

Example. Find the equation of the plane which contains the point (0, 1, 2)and is perpendicular to the line (1, 1, 1) + t(2, 1, 0) = (x, y, z).

2.3 Surfaces

The graph in 3D space of an equation in x, y and z is a surface. Often thegraph is too difficut to draw, but here we sketch the graph of a few specialtypes of equations whose graphs are easy to visualize.

Cylindrical surfaces

The graph in 3D space of an equation containing only one or two of the threevariables x, y, z is called a cylindrical surface.

Example. Plot y = x2.

Plot x2 + y2 = 5.

Quadric Surfaces

The graph in 2D space of a second degree equation in x and y is an ellipse,parabola or hyperbola. In 3D space, the graph of a second degree equationin x, y and z is one of six quadric surfaces.

1. Ellipsoid x2

a2 + y2

b2+ z2

c2= 1

2. Elliptic Cone z2 = x2

a2 + y2

b2

3. Elliptic Paraboloid z = x2

a2 + y2

b2

4. Hyperbolic Parabolid z = y2

b2− x2

a2

5. Elliptic Hyperboloid of one sheet x2

a2 + y2

b2− z2

c2= 1

6. Elliptic Hyperboloid of two sheets −x2

a2 − y2

b2+ z2

c2= 1

Cross-sections of some quadric surfaces

2.4 Functions of several variables

So far you have studied about functions of one variable, e.g.

f(x) = x+ x2.

You have learned how to graph these functions, perhaps how to determinetheir domain and range. You have gone much further in your quest to under-stand functions; you have learned how to differentiate them, then to calculatethe maximum and minimum, then to integrate them. At the end of this term(in March) we will learn how to write a function as a sum of simpler func-tions, i.e. a Fourier Series Expansion of a function.

However now we will learn something new. We will learn about functionsof several variables, e.g.

f(x, y) = x2 + y2, g(x, y) = 2xy + 7, h(x, y) = ex + 2y.

You will very quickly see that although the concepts are new, the techniquesare old and familiar.

Graphs and Level Sets

To draw the graph of f(x) = x2, we drew the graph of the equation

y = x2.

Similarly, to draw the graph of the equation f(x, y) = x2 + y2, we draw thegraph of the equation

z = x2 + y2.

We now use the methods developed in the last lecture to draw graphs. Notehowever that it is difficult to graph general surfaces.

Remark. The graph of a function f(x) of one variable is the graph of theequation y = f(x), a curve in 2D space. The graph of a function f(x, y) oftwo variables is the graph of the equation z = f(x, y), a surface in 3D space.The graph of a function f(x, y, z) is a set of points in 4D space and we cannotdraw the graph.

One way to understand functions of two or more variables is by usinglevel sets.

Example. An example of level sets is a topographic map, which maps hillsand valleys in a region by drawing curves indicating height or elevation. Ifh(x, y) is the height function over a region, say the Himalayas, then if wemark all the points (x, y) on the ground at which the height of the mountainis h(x, y) = 3000m, we get the level set for L = 3000. If we draw the levelsets for different heights, e.g 2000m, 3000m, 4000m, 5000m, 6000m, 7000m,8000m, we get a rough topographic map for the Himalayas.

Note that we draw the level sets on the ground, i.e. in the domain.

Definition. We fix a number C. The level set of a function of two variablesf(x, y) is the set of points (x, y) in the domain which satisfy the equation

f(x, y) = C.

For every real number C, we get a level set.In general, for a fixed number C and a function of several variables f , we

define the level set to be the collection of points in the domain which satisfythe equation f = C.

Example. Let f(x) = x2 and say the constant c = 1. The level set is theset of points such that

x2 = 1

x = 1,−1

Hence the level set for c = 1 is {1, -1}.

Example. Find the level sets of the function f(x, y) = x2+y2 for C = 1, 4, 9.

Solution: The level sets are

C = 1 : x2 + y2 = 1; a circle of radius 1

C = 4 : x2 + y2 = 4; a circle of radius 2

C = 9 : x2 + y2 = 9; a circle of radius 3

Example. Let f(x, y, z) = x2 + y2 + z2 and say the constant c = 1. Thelevel set is the set of points such that

x2 + y2 + z2 = 1

Hence the level set for c = 1 is the set of points lying on the unit sphere.

2.5 Change of Coordinates

Before we describe cylindrical and spherical coordinate systems, we will recallthe polar coordinate system in 2D space.

Polar Coordinates

The polar coordinate system is equivalent to the rectangular coordinate sys-tem. It locates points using two coordinates r and θ. The coordinate r is thedistance from a point to the origin, and θ is the angle used in trignometrywhich measures the counterclockwise rotation from the positive X-axis.

Conversion from rectangular to polar coordinates:Let (x, y) be a point in the rectangular coordinate system.

r =√x2 + y2

tan θ =y

x

Conversion from polar to rectangular coordinates:Let (r, θ) be a point in the polar coordinate system.

x = r cos θ

y = r sin θ

Example. Express in polar coordiantes the portion of the unit disc that liesin the first quadrant.

Solution: The region may be expressed in polar coordinates as

0 ≤ r ≤ 1; 0 ≤ θ ≤ π/2

Example. Express in polar coordinates the function

f(x, y) = x2 + y2 + 2yx.

Solution: We substitute x = r cos θ and y = r sin θ in f , to get

f(r, θ) = r2 cos2(θ) + r2 sin2(θ) + 2r2 sin(θ) cos(θ) = r2(1 + sin(2θ)).

Cylindrical Coordinates

This is a three dimensional extension of plane polar coordinates. Cylindricalcoordinates are given by the 3-tuple (r, θ, z), the polar coordinates of theX, Y plane and the rectangular coordinate z. Given a point (x, y, z) in 3-dimensional space, to calculate r and θ we project the point to the XY-plane(x, y, z) 7→ (x, y). Then calculate (r, θ) as in the previous section.

Cylindrical to Rectangular Conversion Formulas:

x = r cos θ

y = r sin θ

z = z

Rectangular to Cylindrical Conversion Formulas:

r =√x2 + y2

θ = tan−1(y

x)

z = z

Example. 1. Express in cylindrical coordinates the function

f(x, y, z) = x2 + y2 + z2 − 2z√x2 + y2

2. Express in rectangular coordinates the equation

r = sin θ

Spherical coordinates

Spherical coordinates consist of the 3-tuple (ρ, θ, φ). These are determinedas follows:

1. ρ = the distance from the origin to the point.

2. θ = the same angle that we saw in polar/cylindrical coordinates.

3. φ = the angle between the positive z-axis and the line from the origin tothe point.

Spherical to Rectangular Conversion Formulas:

x = ρ sin(φ) cos(θ)

y = ρ sin(φ) sin(θ)

z = ρ cosφ

Rectangular to Spherical Conversion Formulas:

ρ =√x2 + y2 + z2

θ = tan−1(y

x)

φ = cos−1(z

ρ)

Example. 1. Express in spherical coordinates the function

f(x, y, z) = x2 + y2 + z2

2. Express in rectangular coordinates the equation

ρ = 5

Summary

1. The point-normal equation of a plane that contains the point P (x0, y0, z0)and has normal vector ~n = (a, b, c) is

a(x− x0) + b(y − y0) + c(z − z0) = 0.

2. The vector equation of a line that contains the point P (x0, y0, z0) andis parallel to the vector ~v = (a, b, c) is:

P + t~v = ~r, where t is scalar.

3. The parametric equation of a line that contains the point P (x0, y0, z0)and is parallel to the vector ~v = (a, b, c) is:

x = x0 + ta

y = y0 + tb

z = z0 + tc

4. We fix a number C. The level set of a function of two variables f(x, y)is the set of points (x, y) in the domain which satisfy the equation

f(x, y) = C.

5. The polar coordinate system has two coordinates r and θ. The coor-dinate r is the distance from a point to the origin, and θ is the angleused in trignometry which measures the counterclockwise rotation fromthe positive X-axis.

Conversion from rectangular to polar coordinates:Let (x, y) be a point in the rectangular coordinate system.

r =√x2 + y2

tan θ =y

x

Conversion from polar to rectangular coordinates:Let (r, θ) be a point in the polar coordinate system.

x = r cos θ

y = r sin θ

6. Cylindrical Coordinate System is a three dimensional extension ofplane polar coordinates. Cylindrical coordinates are given by the 3-tuple(r, θ, z), the polar coordinates of the X, Y plane and the rectangularcoordinate z.

Cylindrical to Rectangular Conversion Formulas:

x = r cos θ

y = r sin θ

z = z

Rectangular to Cylindrical Conversion Formulas:

r =√x2 + y2

θ = tan−1(y

x)

z = z

7. Spherical coordinates consist of the 3-tuple (ρ, θ, φ). These are deter-mined as follows:

1. ρ = the distance from the origin to the point.

2. θ = the same angle that we saw in polar/cylindrical coordinates.

3. φ = the angle between the positive z-axis and the line from the originto the point.

Spherical to Rectangular Conversion Formulas:

x = ρ sin(φ) cos(θ)

y = ρ sin(φ) sin(θ)

z = ρ cosφ

Rectangular to Spherical Conversion Formulas:

ρ =√x2 + y2 + z2

θ = tan−1(y

x)

φ = cos−1(z

ρ)

References

1. A complete set of notes on Pre-Calculus, Single Variable Calculus, Mul-tivariable Calculus and Linear Algebra. Here is a link to the chapter onLines, Planes and Quadric Surfaces. Also read the section on cylindricaland spherical coordiantes.http://tutorial.math.lamar.edu/Classes/CalcIII/3DSpace.aspx.

2. Another gallery of animated and graphical demonstrations of calculus andrelated topics, from the University of Minnesota.

http://www.math.umn.edu/%7Erogness/quadrics/.

3. Links to various resources on Calculus.http://www.calculus.org/.

3 Partial Derivatives

3.1 First Order Partial Derivatives

A function f(x) of one variable has a first order derivative denoted by f ′(x)or

df

dx= lim

h→0

f(x+ h)− f(x)

h.

It calculates the slope of the tangent line of the function f at x.

A function f(x, y) of two variables has two first order partials ∂f∂x

, ∂f∂y

.Just like in the one variable case, the partial derivatives are also related tothe tangent plane of the function at a point (x, y)

Definition. ∂f∂x

is defined as the derivative of f with respect to x with ytreated as a constant.

∂f

∂x= lim

h→0

f(x+ h, y)− f(x, y)

h.

∂f∂y

is defined as the derivative of f with respect to y with x treated as aconstant.

∂f

∂y= lim

k→0

f(x, y + k)− f(x, y)

k.

Similarly for f(x, y, z) we can define three first order partial derivatives: ∂f∂x

,∂f∂y

, ∂f∂z

.

Example. 1.

f(x, y) = x2y3

∂f

∂x=

∂x2y3

∂x= y3∂x

2

∂x= 2xy3

∂f

∂y=

∂x2y3

∂y= 3x2y2

2. f(x, y, z) = ze2x+3y+4z

3. f(x, y) = x2 + y2

Example. Consider the function f(x, y) = x2/y. Calculate the first orderpartial derivatives and evaluate them at the point P (2, 1).

Solution:

f(x, y) =x2

y∂f

∂x=

2x

y∂f

∂x|{x=2,y=1} = 4

∂f

∂y= −x

2

y2

∂f

∂y|{x=2,y=1} = −1

9

Remark. Partial derivatives are used in the same manner as the derivativeof a function of one variable. The partial of f(x, y) with respect to x isthe rate of change (or the slope) of f with respect to x as y stays constant.Similarly the partial of f(x, y) with respect to y is the rate of change (or theslope) of f with respect to y as x stays constant. For instance in the aboveexample, the slope of the function at the point P (2, 1) in the x direction is4, and the slope of the function at P in the y direction is −1/9.

Theorem Let f(x, y) be a function of two variables and let P = (x0, y0, z0)be a point on the graph. The equation of the tangent plane to the graph off(x, y) at the point P is given by,

z − z0 =∂f

∂x|{x0,y0}(x− x0) +

∂f

∂y|{x0,y0}(y − y0).

We will see why this is true after we study about Directional Derivative andthe Gradient. For now we accept it as a result and use it in some examples.

Example. We consider the function f(x, y) = x3 + 2xy − y + 3. Computethe tangent plane at the point P0 = (1, 2, 6) on the graph of the function.

Solution: (Solution:) We first check that the point P0(x0, y0, z0) = (1, 2, 6)is on the graph of the function:

f(1, 2) = 13 + 2.1.2− 2 + 3

= 1 + 4− 2 + 3

= 6

So the point P0 does indeed lie on the graph of f .Then the equation of the tangent plane is given by;

z − z0 =∂f

∂x {x0,y0}|(x− x0) +

∂f

∂y|{x0,y0}(y − y0).

∂f

∂x|{1,2} = 3x2 + 2y|{1,2} = 7

∂f

∂y|{1,2} = 2x− 1|{1,2} = 1

z − 6 = 7(x− 1) + 1(y − 2)

= 7x− 7 + y − 2

z = 7x+ y − 3

Therefore the tangent plane to the graph of f at (1, 2, 6) is

7x+ y − z − 3 = 0.

3.2 Higher Order Partial Derivatives

If f is a function of several variables, then we can find higher order partialsin the following manner.

Definition. If f(x, y) is a function of two variables, then ∂f∂x

and ∂f∂y

arealso functions of two variables and their partials can be taken. Hence we candifferentiate them with respect to x and y again and find,

∂2f∂x2 , the derivative of f taken twice with respect to x,

∂2f∂x∂y

, the derivative of f with respect to y and then with respect to x,

∂2f∂y∂x

, the derivative of f with respect to x and then with respect to y,

∂2f∂y2 , the derivative of f taken twice with respect to y.

We can carry on and find ∂3f∂x∂y2 , which is taking the derivative of f first with

respect to y twice, and then differentiating with respect to x, etc. In thismanner we can find nth-order partial derivatives of a function.

Theorem ∂2f∂x∂y

and ∂2f∂y∂x

are called mixed partial derivatives. They are equal

when ∂2f∂x∂y

and ∂2f∂y∂x

are continuous.

Note. In this course all the fuunctions we will encounter will have equalmixed partial derivatives.

Example. 1. Find all partials up to the second order of the function

f(x, y) = x4y2 − x2y6.

Solution:

∂f

∂x= 4x3y2 − 2xy6

∂f

∂y= 2x4y − 6x2y5

∂2f

∂x2= 12x2y2 − 2y6

∂2f

∂y∂x= 8x3y − 12xy5

∂2f

∂y2= 2x4 − 30x2y4

∂2f

∂x∂y= 8x3 − 12xy5

Notations:

fx =∂f

∂x

fy =∂f

∂y

fxx =∂2f

∂x2

fyy =∂2f

∂y2

fxy =∂2f

∂x∂y

3.3 Chain Rule

You are familiar with the chain rule for functions of one variable: if f isa function of u, denoted by f = f(u), and u is a function of x, denotedu = u(x). Then

df

dx=df

du

du

dx.

Chain Rules for First-Order Partial Derivatives

For a two-dimensional version, suppose z is a function of u and v, denoted

z = z(u, v)

and u and v are functions of x and y,

u = u(x, y) and v = v(x, y)

then∂z

∂x=∂z

∂u

∂u

∂x+∂z

∂v

∂v

∂y

∂z

∂y=∂z

∂u

∂u

∂y+∂z

∂v

∂v

∂y

z∂z∂u

wwoooooooooooooo∂z∂v

''PPPPPPPPPPPPPP

u∂u∂x

������

���� ∂u

∂y

��???

????

? v∂v∂x

������

���� ∂v

∂y

��???

????

x y x y

Example. 1. Find the first partial derivatives using chain rule.

z = z(u, v)

u = xy

v = 2x+ 3y

Solution:

∂z

∂x=

∂z

∂u

∂u

∂x+∂z

∂v

∂v

∂x

= y∂z

∂u+ 2

∂z

∂v∂z

∂y=

∂z

∂u

∂u

∂y+∂z

∂v

∂v

∂y

= x∂z

∂u+ 3

∂z

∂v

Chain Rule for Second Order Partial Derivatives

To find second order partials, we can use the same techniques as first orderpartials, but with more care and patience!

Example. Let

z = z(u, v)

u = x2y

v = 3x+ 2y

1. Find ∂2z∂y2 .

Solution: We will first find ∂2z∂y2 .

∂z

∂y=∂z

∂u

∂u

∂y+∂z

∂v

∂v

∂y= x2 ∂z

∂u+ 2

∂z

∂v.

Now differentiate again with respect to y to obtain

∂2z

∂y2=

∂

∂y(x2 ∂z

∂u) +

∂

∂y(2∂z

∂v)

= x2 ∂

∂y(∂z

∂u) + 2

∂

∂y(∂z

∂v)

Note that z is a function of u and v, and u and v are functions of x andy. Then the partial derivatives ∂z

∂uand ∂z

∂vare also initially functions of

u and v and eventually functions of x and y. In other words we havethe same dependence diagram as z.

Note. ∂∂y

( ∂z∂u

) 6= ∂2z∂y∂u

). Never write ∂2z∂y∂u

as it is mathematically mean-ingless.

∂z∂u∂2z

∂u2

wwpppppppppppppp∂2z

∂v∂u

''NNNNNNNNNNNNNN

u∂u∂x

������

���� ∂u

∂y

��???

????

? v∂v∂x

������

���� ∂v

∂y

��???

????

x y x y

∂z∂v

∂2z∂u∂v

wwpppppppppppppp ∂2z∂v2

''NNNNNNNNNNNNNN

u∂u∂x

������

���� ∂u

∂y

��???

????

? v∂v∂x

������

���� ∂v

∂y

��???

????

x y x y

Therefore

∂2z

∂y2= x2(

∂2z

∂u2

∂u

∂y+

∂2z

∂v∂u

∂v

∂y) + 2(

∂2z

∂u∂v

∂u

∂y+∂2z

∂v2

∂v

∂y)

= x2(x2 ∂2z

∂u2+ 2

∂2z

∂v∂u) + 2(x2 ∂2z

∂u∂v+ 2

∂2z

∂v2)

The mixed partials are equal so the answer simplifies to

∂2z

∂y2= x4 ∂

2z

∂u2+ 4x2 ∂2z

∂u∂v+ 4

∂2z

∂v2.

2. Find ∂2z∂x∂y

Solution: We have

∂z

∂y=∂z

∂u

∂u

∂y+∂z

∂v

∂v

∂y= x2 ∂z

∂u+ 2

∂z

∂v.

Now differentiate again with respect to x to obtain

∂2z

∂x∂y=

∂

∂x(x2 ∂z

∂u) +

∂

∂x(2∂z

∂v)

= x2 ∂

∂x(∂z

∂u) + 2x

∂z

∂u+ 2

∂

∂x(∂z

∂v)

= x2(∂2z

∂u2

∂u

∂x+

∂2z

∂v∂u

∂v

∂x) + 2x

∂z

∂u+ 2(

∂2z

∂u∂v

∂u

∂x+∂2z

∂v2

∂v

∂x)

= x2(2xy∂2z

∂u2+ 3

∂2z

∂v∂u) + 2x

∂z

∂u+ 2(2xy

∂2z

∂u∂v+ 3

∂2z

∂v2)

= 2x3y∂2z

∂u2+ (3x2 + 4xy)

∂2z

∂v∂u) + 2x

∂z

∂u+ 6

∂2z

∂v2

3.4 Maxima and Minima

Recall from 1-dimensional calculus, to find the points of maxima and minimaof a function, we first find the critical points i.e where the tangent line ishorizontal f ′(x) = 0. Then

(i) If f ′′(x) > 0 the gradient is increasing and we have a local minimum.

(ii) If f ′′(x) < 0 the gradient is decreasing and we have a local maximum.

(iii) If f ′′(x) = 0 it is inconclusive.

Critical Points of a function of 2 variables

We are now interested in finding points of local maxima and minima for afunction of two variables.

Definition. A function f(x, y) has a relative minimum (resp. maximum)at the point (a, b) if f(x, y) ≥ f(a, b) (resp. f(x, y) ≤ f(a, b)) for all points(x, y) in some region around (a, b)

Definition. A point (a, b) is a critical point of a function f(x, y) if one ofthe following is true

(i) fx(a, b) = 0 and fy(a, b) = 0

(ii) fx(a, b) and/or fy(a, b) does not exist.

Classification of Critical Points

We will need two quantities to classify the critical points of f(x, y):

1. fxx, the second partial derivative of f with respect to x.

2. H = fxxfyy − f 2xy the Hessian

If the Hessian is zero, then the critical point is degenerate. If the Hessianis non-zero, then the critical point is non-degenerate and we can classify thepoints in the following manner:

case(i) If H > 0 and fxx < 0 then the critical point is a relative maximum.

case(ii) If H > 0 and fxx > 0 then the critical point is a relative minimum.

case(iii) If H < 0 then the critical point is a saddle point.

Example. Find and classify the critical points of

12x3 + y3 + 12x2y − 75y.

Solution: We first find the critical points of the function.

fx = 36x2 + 24xy = 12x(3x+ 2y)

fy = 3y2 + 12x2 − 75 = 3(4x2 + y2 − 25).

The critical points are the points where fx = 0 and fy = 0

fx = 0

12x(3x+ 2y) = 0

Therefore either x = 0 or 3x+ 2y = 0. We handle the two cases separately;

case(i) x = 0.Then substituting this in fy we get fy = 3(y2−25) = 0 implies y = ±5.

case(ii) 3x+ 2y = 0.Then y = −3x/2 and substituting this in fy we get,

fy = 3(4x2 +9x2

4− 25)

=3

4(16x2 + 9x2 − 100)

=3

4(25x2 − 100)

=75

4(x2 − 4)

fy = 0

75

4(x2 − 4) = 0

x2 − 4 = 0

x = ±2

Thus we have found four critical points: (0, 5), (0,−5), (2,−3), (−2, 3).We must now classify these points.

fxx = 72x+ 24y = 24(3x+ y)

fxy = 24x

fyy = 6y

H = fxxfyy − f 2xy

= (24)(3x+ y)(6y)− (24x)2

Points fxx H Type(0, 5) 120 3600 Minimum(0,−5) -120 3600 Maximum(2,−3) 72 -3600 Saddle(−2, 3) -72 -3600 Saddle

Summary

1. The First Order Partial Derivative∂f

∂xis defined as the derivative of

f with respect to x, with y treated as a constant.

Similarly we can define∂f

∂y.

2.∂f

∂xand

∂f

∂yare also functions of two variables and hence their partials

can be taken. We can differentiate them with respect to x and y againand find Higher Order Partial Derivatives,

∂2f

∂x2,∂2f

∂x∂y,∂2f

∂y∂x,∂2f

∂y2,∂3f

∂x3, etc.

3. Chain Rule for Partial Dericatives. Suppose z is a function of u andv, denoted z = z(u, v) and u and v are functions of x and y, u = u(x, y)and v = v(x, y), then

∂z

∂x=∂z

∂u

∂u

∂x+∂z

∂v

∂v

∂y,

∂z

∂y=∂z

∂u

∂u

∂y+∂z

∂v

∂v

∂y

4. A point (a, b) is a critical point of a function f(x, y) if one of the followingis true

(i) fx(a, b) = 0 and fy(a, b) = 0

(ii) fx(a, b) and/or fy(a, b) does not exist.

5. We will need two quantities to classify the critical points of f(x, y):

1. fxx, the second partial derivative of f with respect to x.

2. H = fxxfyy − f 2xy the Hessian

If the Hessian is zero, then the critical point is degenerate. If the Hessianis non-zero, then the critical point is non-degenerate and we can classifythe points in the following manner:

case(i) If H > 0 and fxx < 0 then the critical point is a relative maximum.

case(ii) If H > 0 and fxx > 0 then the critical point is a relative minimum.

case(iii) If H < 0 then the critical point is a saddle point.

References

1. Engineering Mathematics, by K. A. Stroud.

2. Engineering Mathematics, by K. A. Stroud.

3. A complete set of notes on Pre-Calculus, Single Variable Calculus, Mul-tivariable Calculus and Linear Algebra. Here is a link to the chapter onPartial Differentiation.http://tutorial.math.lamar.edu/Classes/CalcIII/PartialDerivatives.aspx.

4. Here is a link to the chapter on Higher Order Partial Differentiation.http://tutorial.math.lamar.edu/Classes/CalcIII/HighOrderPartialDerivs.aspx.

Here is a link to the chapter on Chain Rules.http://tutorial.math.lamar.edu/Classes/CalcIII/ChainRule.aspx.

Here is a link to the chapter on Maxima and Minima.http://tutorial.math.lamar.edu/Classes/CalcIII/RelativeExtrema.aspx

5. A collection of examples, animations and notes on Multivariable Calculus.http://people.usd.edu/ jflores/MultiCalc02/WebBook/Chapter 15/

6. Links to various resources on Calculus.http://www.calculus.org/.

4 A little Vector Calculus

4.1 Gradient

Vector Function/ Vector Fields

The functions of several variables we have so far studied would take a point(x, y, z) and give a real number f(x, y, z). We call these types of functionsscalar-valued functions i.e. for example

f(x, y, z) = x2 + 2xyz.

We are now going to talk about vector-valued functions, where we take apoint (x, y, z) and the value of f(x, y, z) is a vector. i.e. for example

f(x, y, z) = (y, x, z2) = y~i+ x~j + z2~k.

Definition. A vector function is a function that takes one or more variablesand returns a vector.

Example. 1. A vector function of a single variable:

r(t) = (2 + t, 3 + 2t, 1− 3t).

Let us look at a few values.

f(2) = (4, 7,−5), f(−2)= (0,−1, 7),

f(1) = (3, 5,−2), f(−1) = (1, 1, 4),

f(0) = (2, 3, 1)

2. A vector function of three variables:

f(x, y, z) = (y, x, z2).

Let us look at a few values.

f(0, 0, 0) = (0, 0, 0), f(1, 0, 0)= (0, 1, 0),

f(0, 1, 0) = (1, 0, 0), f(0, 0, 1)= (0, 0, 1),

f(1, 2, 1) = (2, 1, 1), f(2, 1, 2) = (1, 2, 4)

Definition. A vector field is an assignment of a vector to every point inspace.

Example. 1. In a magnetic field, we can assign a vector which describesthe force and direction to every point in space.

2. Normal to a surface. For a surface, at every point on the surface, wecan get a tangent plane and hence a normal vector to every point on thesurface.

Example. Let f(x, y, z) = x2 + 2xyz be a scalar-valued function. We thendefine a vector-valued function by taking its partial derivatives.

∇f = (∂f

∂x,∂f

∂y,∂f

∂z)

= (2x+ 2yz, 2xz, 2xy)

This kind of vector function has a special name, the gradient.

Definition. Suppose that f(x, y) is a scalar-valued function of two vari-ables. Then the gradient of f is the vector function defined as,

∇f = (∂f

∂x,∂f

∂y) =

∂f

∂x~i+

∂f

∂y~j.

Similarly if f(x, y, z) is a scalar-valued function of three variables. Then thegradient of f is the vector function defined as,

∇f = (∂f

∂x,∂f

∂y,∂f

∂z) =

∂f

∂x~i+

∂f

∂y~j +

∂f

∂z~k.

Remark. The gradient of a scalar function is a vector field which points inthe direction of the greatest rate of increase of the scalar function, and whosemagnitude is the greatest rate of change.

Example. Consider the scalar-valued function defined by f(x, y) = x2 + y2.Find the gradient of f at the point x = 2, y = 5.

Solution: Then gradient of f , is a vector function given by,

∇f = (∂f

∂x,∂f

∂y)

= (2x, 2y)

∇f(2, 5) = (4, 10)

4.2 Directional Derivative

For a function of 2 variables f(x, y), we have seen that the function can beused to represent the surface

z = f(x, y)

and recall the geometric interpretation of the partials:

(i) fx(a, b)-represents the rate of change of the function f(x, y) as we varyx and hold y = b fixed.

(ii) fy(a, b)-represents the rate of change of the function f(x, y) as we varyy and hold x = a fixed.

We now ask, at a point P can we calculate the slope of f in an arbitrarydirection?

Recall the definition of the vector function ∇f ,

∇f =(∂f∂x,∂f

∂y

).

We observe that,

∇f · i = fx

∇f · j = fy

This enables us to calculate the directional derivative in an arbitrary direc-tion, by taking the dot product of ∇f with a unit vector, ~u, in the desireddirection.

Definition. The directional derivative of the function f in the direction ~udenoted by D~uf , is defined to be,

D~uf =∇f · ~u|~u|

Example. What is the directional derivative of f(x, y) = x2 + xy, in thedirection ~i+ 2~j at the point (1, 1)?

Solution: We first find ∇f .

∇f = (∂f

∂x,∂f

∂y)

= (2x+ y, x)

∇f(1, 1) = (3, 1)

Let u =~i+ 2~j.|~u| =

√12 + 22 =

√1 + 4 =

√5.

D~uf(1, 1) =∇f · ~u|~u|

=(3, 1).(1, 2)√

5

=(3)(1) + (1)(2)√

5

=5√5

=√

5

Properties of the Gradient deduced from the formula of DirectionalDerivatives

D~uf =∇f · ~u|~u|

=|∇f ||~u| cos θ

|~u|= |∇f | cos θ

1. If θ = 0, i.e. i.e. ~u points in the same direction as ∇f , then D~uf ismaximum. Therefore we may conclude that

(i) ∇f points in the steepest direction.

(ii) The magnitude of ∇f gives the slope in the steepest di-rection.

2. At any point P , ∇f(P ) is perpendicular to the level set throughthat point.

Example. 1. Let f(x, y) = x2 + y2 and let P = (1, 2, 5). Then P lies onthe graph of f since f(1, 2) = 5. Find the slope and the direction of thesteepest ascent at P on the graph of f

Solution: • We use the first property of the Gradient vector. Thedirection of the steepest ascent at P on the graph of f is the directionof the gradient vector at the point (1, 2).

∇f = (∂f

∂x,∂f

∂y)

= (2x, 2y)

∇f(1, 2) = (2, 4).

• The slope of the steepest ascent at P on the graph of f is the mag-nitude of the gradient vector at the point (1, 2).

|∇f(1, 2)| =√

22 + 42 =√

20.

2. Find a normal vector to the graph of the equation f(x, y) = x2 + y2 atthe point (1, 2, 5). Hence write an equation for the tangent plane at thepoint (1, 2, 5).

Solution: We use the second property of the gradient vector. For afunction g, ∇g(P ) is perpendicular to the level set. So we want oursurface z = x2 + y2 to be the level set of a function.Therefore we define a new function,

g(x, y, z) = x2 + y2 − z.

Then our surface is the level set

g(x, y, z) = 0

x2 + y2 − z = 0

z = x2 + y2

∇g = (∂g

∂x,∂g

∂y,∂g

∂z)

= (2x, 2y,−1)

∇g(1, 2, 5) = (2, 4,−1)

By the above property, ∇g(P ) is perpendicular to the level set g(x, y, z) =0. Therefore ∇g(P ) is the required normal vector.Finally an equation for the tangent plane at the point (1, 2, 5) on thesurface is given by

2(x− 1) + 4(y − 2)− 1(z − 5) = 0.

4.3 Curl and Divergence

We denoted the gradient of a scalar function f(x, y, z) as

∇f = (∂f

∂x,∂f

∂y,∂f

∂z)

Let us separate or isolate the operator ∇ = ( ∂∂x, ∂

∂y, ∂

∂z). We can then define

various physical quantities such as div, curl by specifying the action of theoperator ∇.

Divergence

Definition. Given a vector field ~v(x, y, z) = (v1(x, y, z), v2(x, y, z), v3(x, y, z)),the divergence of ~v is a scalar function defined as the dot product of the vectoroperator ∇ and ~v,

Div~v = ∇ · ~v= (

∂

∂x,∂

∂y,∂

∂z) · (v1, v2, v3)

=∂v1

∂x+∂v2

∂y+∂v3

∂z

Example. Compute the divergence of (x− y)~i+ (x+ y)~j + z~k.

Solution:

~v = ((x− y), (x+ y), z)

∇ = (∂

∂x,∂

∂y,∂

∂z)

Div~v = ∇ · ~v= (

∂

∂x,∂

∂y,∂

∂z) · ((x− y), (x+ y), z)

=∂(x− y)

∂x+∂(x+ y)

∂y+∂z

∂z= 1 + 1 + 1

= 3

Curl

Definition. The curl of a vector field is a vector function defined as thecross product of the vector operator ∇ and ~v,

Curl~v = ∇× ~v =

∣∣∣∣∣∣i j k∂∂x

∂∂y

∂∂z

v1 v2 v3

∣∣∣∣∣∣= (

∂v3

∂y− ∂v2

∂z)i− (

∂v3

∂x− ∂v1

∂z)j + (

∂v2

∂x− ∂v1

∂y)k

Example. Compute the curl of the vector function (x− y)~i+(x+ y)~j+ z~k.

Solution:

Curl~v = ∇× ~v =

∣∣∣∣∣∣i j k∂∂x

∂∂y

∂∂z

(x− y) (x+ y) z

∣∣∣∣∣∣= (

∂z

∂y− ∂(x+ y)

∂z)i− (

∂z

∂x− ∂(x− y)

∂z)j + (

∂(x+ y)

∂x− ∂(x− y)

∂y)k

= (0− 0)~i− (0− 0)~j + (1− (−1))~k

= 2~k

4.4 Laplacian

We have seen above that given a vector function, we can calculate the diver-gence and curl of that function. A scalar function f has a vector function∇f associated to it. We now look at Curl(∇f) and Div(∇f).

Curl(∇f) = ∇×∇f

= (∂fz

∂y− ∂fy

∂z)i+ (

∂fx

∂z− ∂fz

∂x)j + (

∂fy

∂x− ∂fx

∂y)k

= (fyz − fzy)i+ (fzx − fxz)j + (fxy − fyx)k

= 0

Div(∇f) = ∇ · ∇f

= (∂

∂x,∂

∂y,∂

∂z) · (∂f

∂x,∂f

∂y,∂f

∂z)

=∂2f

∂x2+∂2f

∂y2+∂2f

∂z2

Definition. The Laplacian of a scalar function f(x, y) of two variables isdefined to be Div(∇f) and is denoted by ∇2f ,

∇2f =∂2f

∂x2+∂2f

∂y2.

The Laplacian of a scalar function f(x, y, z) of three variables is defined tobe Div(∇f) and is denoted by ∇2f ,

∇2f =∂2f

∂x2+∂2f

∂y2+∂2f

∂z2.

Example. Compute the Laplacian of f(x, y, z) = x2 + y2 + z2.

Solution:

∇2f =∂2f

∂x2+∂2f

∂y2+∂2f

∂z2

=∂2x

∂x+∂2y

∂y+∂2z

∂z= 2 + 2 + 2

= 6.

We have the following identities for the Laplacian in different coordinatesystems:

Rectangular : ∇2f =∂2f

∂x2+∂2f

∂y2+∂2f

∂z2

Polar : ∇2f =1

r

∂

∂r

(r∂f

∂r

)+

1

r2

∂2f

∂θ2

Cylindrical : ∇2f =1

r

∂

∂r

(r∂f

∂r

)+

1

r2

∂2f

∂θ2+∂2f

∂z2

Spherical : ∇2f =1

ρ2

∂

∂ρ

(ρ2∂f

∂ρ

)+

1

ρ2 sin θ

∂

∂θ

(sin θ

∂f

∂θ

)+

1

ρ2 sin2 θ

∂2f

∂φ2

Example. Consider the same function f(x, y, z) = x2 + y2 + z2. We haveseen that in rectangular coordinates we get

∇2f =∂2f

∂x2+∂2f

∂y2+∂2f

∂z2= 6.

We now calculate this in cylindrical and spherical coordinate systems, usingthe formulas given above.

1. Cylindrical Coordinates.We have x = r cos θ and y = r sin θ so

f(r, θ, z) = r2 cos2 θ + r2 sin2 θ + z2 = r2 + z2.

Using the above formula:

∇2f =1

r

∂

∂r

(r∂f

∂r

)+

1

r2

∂2f

∂θ2+∂2f

∂z2

=1

r

∂

∂r(r2r) + 0 +

∂(2z)

∂z

=1

r(4r) + 2

= 4 + 2

= 6

2. Spherical Coordinates.We have x = ρ sinφ cos θ, y = ρ sinφ sin θ, z = ρ cosφ and ρ =

√x2 + y2 + z2,

sof(r, θ, z) = ρ2.

Using the above formula:

∇2f =1

ρ2

∂

∂ρ

(ρ2∂f

∂ρ

)+

1

ρ2 sin θ

∂

∂θ

(sin θ

∂f

∂θ

)+

1

ρ2 sin2 θ

∂2f

∂φ2

=1

ρ2

∂

∂ρ(ρ22ρ) + 0 + 0

=1

ρ2

∂

∂ρ(2ρ3)

=1

ρ2(6ρ2)

= 6.

These three different calculations all produce the same result because ∇2

is a derivative with a real physical meaning, and does not depend on thecoordinate system being used.

Summary

1. If f(x, y) is a scalar-valued function of two variables. Then the gradientof f is the vector function defined as,

∇f = (∂f

∂x,∂f

∂y) =

∂f

∂x~i+

∂f

∂y~j.

2. Important Properties about the Gradient.

• ∇f points in the steepest direction.

• The magnitude of ∇f gives the slope in the steepest direc-tion.

• At any point P , ∇f(P ) is perpendicular to the level set throughthat point.

3. The directional derivative of the function f in the direction ~u denotedby D~uf , is defined to be,

D~uf =∇f · ~u|~u|

4. Given a vector field ~v(x, y, z) = (v1(x, y, z), v2(x, y, z), v3(x, y, z)), the di-vergence of the vector field, ~v is a scalar function defined as the dotproduct of the vector operator ∇ and ~v,

Div~v = ∇ · ~v= (

∂

∂x,∂

∂y,∂

∂z) · (v1, v2, v3)

=∂v1

∂x+∂v2

∂y+∂v3

∂z

5. The curl of a vector field is a vector function defined as the crossproduct of the vector operator ∇ and ~v,

Curl~v = ∇× ~v =

∣∣∣∣∣∣i j k∂∂x

∂∂y

∂∂z

v1 v2 v3

∣∣∣∣∣∣= (

∂v3

∂y− ∂v2

∂z)i− (

∂v3

∂x− ∂v1

∂z)j + (

∂v2

∂x− ∂v1

∂y)k

6. We have the following identities for the Laplacian in different coordinatesystems:

Rectangular : ∇2f =∂2f

∂x2+∂2f

∂y2+∂2f

∂z2

Polar : ∇2f =1

r

∂

∂r

(r∂f

∂r

)+

1

r2

∂2f

∂θ2

Cylindrical : ∇2f =1

r

∂

∂r

(r∂f

∂r

)+

1

r2

∂2f

∂θ2+∂2f

∂z2

Spherical : ∇2f =1

ρ2

∂

∂ρ

(ρ2∂f

∂ρ

)+

1

ρ2 sin θ

∂

∂θ

(sin θ

∂f

∂θ

)+

1

ρ2 sin2 θ

∂2f

∂φ2

References

1. A briliant animated example, showing that the maximum slope at a pointoccurs in the direction of the gradient vector. The animation shows:

• a surface

• a unit vector rotating about the point (1, 1, 0), (shown as a rotatingblack arrow at the base of the figure)

• a rotating plane parallel to the unit vector, (shown as a grey grid)

• the traces of the planes in the surface, (shown as a black curve onthe surface)

• the tangent lines to the traces at (1, 1, f (1, 1)), (shown as a blueline)

• the gradient vector (shown in green at the base of the figure)

http://archives.math.utk.edu/ICTCM/VOL10/C009/dd.gif

2. A complete set of notes on Pre-Calculus, Single Variable Calculus, Multi-variable Calculus and Linear Algebra.Here is a link to the chapter on Directional Derivatives.http://tutorial.math.lamar.edu/Classes/CalcIII/DirectionalDeriv.aspx.

Here is a link to the chapter on Curl and Divergence.http://tutorial.math.lamar.edu/Classes/CalcIII/CurlDivergence.aspx

5 Linear Equations

5.1 Introduction

A linear equation is one in which all the unknown variables occur with apower of one. i.e. 2x + y = 5 is a linear equation, but 2x2 + y = 5 is not alinear equation, since x has power 2. Systems of linear equations are commonin all branches of science. In school you have learned how to simultaeouslysolve a system of two linear equations.

Example. Solve the system of two linear equations:

3x1 + 2x2 = 7

−x1 + x2 = 6

Then x1 = −1 and x2 = 5 is a solution. Geometrically the solution, (−1, 5)is the point of intersection of the two lines given by the two equations.

Here is an example of larger system of equations from Chemistry.

Example. Under certain controled conditions, mix toulene C7H8 and nitricacid HNO3 to produce trinitrotoluene (TNT) C7H5O6N3 along with wa-ter. In what proportion should those components be mixed, i.e. solve forx, y, z, w?

Solution: The number of atoms of each element before the reaction mustequal the number present afterward!

xC7H8 + yHNO3 → zC7H5O6N3 + wH2O

So we get the system:

7x = 7z

8x+ y = 5z + 2w

y = 3z

3y = 6z + w

To find the answer we need to solve the above system of linear equations. Inthis chapter we will learn an easy method to solve such a system.

5.2 Row Echelon Form

In this section we learn Gauss’ Method to solve a system of linear equations.We will use an example to understand the method.

Example. Find the solution set of the linear system

x− y = 0

2x− 2y + z + 2w = 4

y + w = 0

2z + w = 5

We can abbreviate this linear system with the linear array of numbers,whose entries are the coefficients of the equations:

1 −1 0 0 02 −2 1 2 40 1 0 1 00 0 2 1 5

.

The vertical bar just reminds us that the coefficients of the system are onthe left hand side of the bar and the constants are on the right. We call thisan augmented matrix. We can now proceed with Gauss’ method.

1 −1 0 0 02 −2 1 2 40 1 0 1 00 0 2 1 5

R2=−2R1+R2→

1 −1 0 0 00 0 1 2 40 1 0 1 00 0 2 1 5

R2↔R3→

1 −1 0 0 00 1 0 1 00 0 1 2 40 0 2 1 5

R4=−2R3+R4→

1 −1 0 0 00 1 0 1 00 0 1 2 40 0 0 −3 −3

The resulting equations are:

x− y = 0

y + w = 0

z + 2w = 4

w = 1

Back substitution now gives w = 1, z = 2, y = −1, x = −1.

Remark. We transformed the original set of equations into a new simplerset of equations, using row operations to put the array into Row EchelonForm. You probably have two questions;

1. Why do both the original system and the new simpler system has thesame solution set?Answer: There is a theorem which says that, if a linear system is changedto another by one of these operations

• an equation is swapped with another

• an equation has both sides multiplied by a nonzero constant

• an equation is replaced by the sum of itself and a multiple of another

then the two systems have the same set of solutions.

2. What is Row Echelon Form?Answer: A matrix is in Row Echelon Form if it has the following form:

• any all zero rows are at the bottom of the reduced matrix

• in a non-zero row, the first-from-left nonzero value is 1

• the first 1 in each row is to the right of the first 1 in the row above

Algorithm to put a matrix into Row Echelon Form

(i) Make the first entry of the first row non-zero by doing a swap if neces-sary.

(ii) Make the first entry of the first row 1 by multiplying by the reciprocal.The first row is now in the correct form.

(iii) Make the first entry in all the rows below the first row 0, by using the 1

in the first row above to subtract. The first column should be

100...

(iv) Now we can disregard the first row and first column because they are in

the correct form, and only consider the remaining sub-matrix. Repeatsteps (i)-(iv) for the smaller sub-matrix. Continue till you run out ofrows.

5.3 Soving Systems of Linear Equations

The solution set of a system of linear equations can be;

(i) Empty

(ii) A one point set, i.e. a unique solution

(iii) Infinite

An example where the solution set is empty.

Example. Consider the system

x+ 2y = 4

x+ 2y = 8

The augmented matrix:(1 2 41 2 8

)R2=−R1+R2→

(1 2 40 0 4

)

We get the absurd relation 0 = 4. This implies that the system has nosolutions. Geometrically the two equations are two parallel lines and hencethere is no point of inersection.

An example where the solution set is infinite.

Example. Consider the linear system of equations

x1 + 2x2 = 4

x2 − x3 = 0

x1 + 2x3 = 4.

The associated augmented matrix: 1 2 0 40 1 −1 01 0 2 4

We now reduce the matrix to row echelon form: 1 2 0 4

0 1 −1 01 0 2 4

R3=−R1+R3→

1 2 0 40 1 −1 00 −2 2 0

R3=2R2+R3→

1 2 0 40 1 −1 00 0 0 0

We have a bottom row of zeros. The second row gives x2 = x3 and the first

row gives x1 +2x2 = 4. We can express x1 in terms of x3 to get x1 = 4−2x3.Thus we can write the solution set for the system in the following manner:

S = {

x1

x2

x3

|x2 = x3 and x1 = 4− 2x3}

= {

4 −2x3

x3

x3

}= {

400

+

−211

x3|x3 ∈ R}

Example. Consider the linear system

x+ y + z − w = 1

y − z + w = −1

3x + 6z − 6w = 6

− y + z − w = 1

The associated augmented matrix1 1 1 −1 10 1 −1 1 −13 0 6 −6 60 −1 1 −1 1

Gauss’ Method:

1 1 1 −1 10 1 −1 1 −13 0 6 −6 60 −1 1 −1 1

R3=−3R1+R3→

1 1 1 −1 10 1 −1 1 −10 −3 3 −3 30 −1 1 −1 1

R3=3R2+R3,R4=R2+R4→

1 1 1 −1 10 1 −1 1 −10 0 0 0 00 0 0 0 0

x+ y + z − w = 1

y − z + w = −1

From the second equation we get, y = −1 + z − w and substituting this inequation one we get, x + (−1 + z − w) + z − w = 1. Solving for x we get

x = 2− 2z + 2w. The soution set:

S = {

xyzw

|y = −1 + z − w and x = 2− 2z + 2w}

= {

2 −2z +2w−1 +z −w

zw

}

= {

2−100

+

−2110

z +

2−101

w|z, w ∈ R}

Summary

1. A matrix is in Row Echelon Form if it has the following form:

• any all zero rows are at the bottom of the reduced matrix

• in a non-zero row, the first-from-left nonzero value is 1

• the first non-zero entry in each row is to the right of the first non-zeroentry in the row above

2. Algorithm to put a matrix into Row Echelon Form:

(i) Make the first entry, of the first row, non-zero by doing a swap ifnecessary.

(ii) Make the first entry, of the first row 1, by multiplying by the recip-rocal. The first row is now in the correct form.

(iii) The first column should be

100...

. We do it, by making the first

entry in all the rows below the first row 0, by using the 1 in the firstrow.

(iv) Now we can disregard the first row and first column because theyare in the correct form, and only consider the remaining sub-matrix.Repeat steps (i)-(iv) for the smaller sub-matrix. Continue till yourun out of rows.

3. Algorithm to solve a system of equations:

(i) Find the augmented matrix

(ii) Reduce the augmented matrix to row echelon form

(iii) Back substitute to find the solution set.

6 Linear Independence and Gram Schmidt

6.1 Linear Combination of Vectors

A linear combination of vectors in R2 is a sum of the form

α

(x1

x2

)+ β

(y1

y2

).

e.g.

2

(1−1

)+ (−4)

(01

).

A linear combination of vectors in R3 is a sum of the form

α

x1

x2

x3

+ β

y1

y2

y3

.

e.g.

2

1−11

+ (−4)

013

.

Example. Write

983

as a linear combination of the vectors

123

,

2−10

,

31−1

.

Solution: We need to find scalars like x, y, z such that

x

123

+ y

2−10

+ z

31−1

=

983

x +2y +3z

2x −y +z3x −z

=

983

We get a system of three linear equations:

x+ 2y + 3z = 9

2x− y + z = 8

3x − z = 3

We can use the methods of the last chapter to find the solution set of thissystem of linear equations. The associated augmented matrix: 1 2 3 9

2 −1 1 83 0 −1 3

We now reduce the matrix to row echelon form: 1 2 3 9

2 −1 1 83 0 −1 3

R2=−2R1+R2,R3=−3R1+R3→

1 2 3 90 −5 −5 −100 −6 −10 −24

R2=−1

5R2→

1 2 3 90 1 1 20 −6 −10 −24

R3=6R2+R3→

1 2 3 90 1 1 20 0 −4 −12

R3=−1

4R3→

1 2 3 90 1 1 20 0 1 3

Therefore we get the relations:

z = 3

y = 2− z = 2− 3 = −1

x = 9− 2y − 3z = 9− 2(−1)− 3(3) = 2

Therefore we write,

(2)

123

+ (−1)

2−10

+ (3)

31−1

=

983

6.2 Linear Independence of Vectors

Definition. A set of vectors S is linearly independent if there is no non-trivial linear combination of vectors that sums to zero.

Remark. If a set is not linearly independent, it is linearly dependent..

Example. 1. Consider the linear combination which sums to zero,

α

(10

)+ β

(01

)=

(00

)(α0

)+

(0β

)=

(00

)(αβ

)=

(00

)α = 0

β = 0

is a unique solution. Therefore the vectors

(10

)and

(01

)are linearly

independent.

2. Consider the linear combination which sums to zero

x

(10

)+ y

(01

)+ z

(11

)=

(00

)(x0

)+

(0y

)+ +

(zz

)=

(00

)(x+ zy + z

)=

(00

)x+ z = 0; x = −zy + z = 0; y = −z

So for any value of z, say z = 1, we can choose x = y = −z = −1 and get

(−1)

(10

)+ (−1)

(01

)+ (1)

(11

)=

(00

)a non-trivial linear combination which sums to zero.

So the vectors

(10

),

(01

)and

(11

)are not linearly independent.

3. Determine whether the set of vectors {[1, 1, 0], [1, 0, 1], [2, 1, 1]} is linearlyindependent.

Solution: Take a linear combination of the vectors which sums to zero.

x

112

+ y

101

+ z

011

=

000

x +y +2z

x +zy +z

=

000

We get a system of three linear equations:

x+ y + 2z = 0

x + z = 0

y + z = 0

We can use the methods of the last chapter to find the solution set of thissystem of linear equations. If x = y = z = 0 is a unique solution, thenthe vectors are linearly independent, other wise we can find a dependency.The associated augmented matrix: 1 1 2 0

1 0 1 00 1 1 0

We now reduce the matrix to row echelon form: 1 1 2 0

1 0 1 00 1 1 0

R2=−R1+R2→

1 1 2 00 −1 −1 00 1 1 0

R3=R2+R3→

1 1 2 00 1 1 00 0 0 0

We have a bottom row of zeros and hence cannot have a unique solution.The second row gives y+ z = 0 and the first row gives x+ y+ 2z = 0. So

y = −zx = −y − 2z = −(−z)− 2z = −z

If we choose z = 1, then x = −1 and y = −1we get a non-trivial linearcombination which sums to zero:

(−1)

110

+ (−1)

101

+ (1)

211

=

000

Therefore the set of vectors is not linearly independent.

Theorem Let S = {~v1, ~v2, ..., ~vn} be a set of n, n-dimensional vectors, thenthe set S is linearly independent if and only if the determinant of the matrixhaving the n vectors as columns is non-zero.

Example. 1. We can apply this result to the example above. Let {[1, 1, 0], [1, 0, 1], [2, 1, 1]}be three, 3-dimensional vectors.∣∣∣∣∣∣

1 1 21 0 10 1 1

∣∣∣∣∣∣ = 1(0− 1)− 1((1− 0) + 2(1− 0) = −1− 1 + 2 = 0.

Since the determinant is zero, the set of vectors are not linearly indepen-dent.

2. Consider the set {[1,−1, 1], [1, 0, 1], [1, 1, 2]}.∣∣∣∣∣∣1 1 1−1 0 11 1 2

∣∣∣∣∣∣ = 1(0− 1)− 1((−1)(2)− (1)(1)) + 1((−1)(1)− 0) = 1 6= 0.

Since the determinant is non-zero, the set of vectors are linearly indepen-dent.

6.3 Orthogonal and orthonormal vectors

Definition. We say that 2 vectors are orthogonal if they are perpendicularto each other. i.e. the dot product of the two vectors is zero.

Definition. We say that a set of vectors {~v1, ~v2, ..., ~vn} are mutually or-thogonal if every pair of vectors is orthogonal. i.e.

~vi.~vj = 0, for all i 6= j.

Example. The set of vectors

10−1

,

1√2

1

,

1

−√

21

is mutually

orthogonal.

(1, 0,−1).(1,√

2, 1) = 0

(1, 0,−1).(1,−√

2, 1) = 0

(1,√

2, 1).(1,−√

2, 1) = 0

Definition. A set of vectors S is orthonormal if every vector in S hasmagnitude 1 and the set of vectors are mutually orthogonal.

Example. We just checked that the vectors

~v1 =

10−1

, ~v2 =

1√2

1

, ~v3 =

1

−√

21

are mutually orthogonal. The vectors however are not normalized (this termis sometimes used to say that the vectors are not of magnitude 1). Let

~u1 =~v1

|~v1|=

1√2

10−1

=

1/√

20

−1/√

2

~u2 =

~v2

|~v2|=

1

2

1√2

1

=

1/2√2/2

1/2

~u3 =

~v3

|~v3|=

1

2

1

−√

21

=

1/2

−√

2/21/2

The set of vectors {~u1, ~u2, ~u3} is orthonormal.

Proposition An orthogonal set of non-zero vectors is linearly independent.

6.4 Gram-Schmidt Process

Given a set of linearly independent vectors, it is often useful to convert theminto an orthonormal set of vectors. We first define the projection operator.

Definition. Let ~u and ~v be two vectors. The projection of the vector ~v on~u is defined as folows:

Proj~u~v =(~v.~u)

|~u|2~u.

Example. Consider the two vectors ~v =

(11

)and ~u =

(10

).

These two vectors are linearly independent.However they are not orthogonal to each other. We create an orthogonalvector in the following manner:

~v1 = ~v − (Proj~u~v)

Proj~u~v =(1)(1) + (1)(0)

(√

12 + 02)2

(10

)= (1)

(10

)~v1 =

(11

)− (1)

(10

)=

(01

)~v1 thus constructed is orthogonal to ~u.

The Gram-Schmidt Algorithm:

Let v1, v2, ..., vn be a set of n linearly independent vectors in Rn. Then wecan construct an orthonormal set of vectors as follows:

Step 1. Let ~u1 = ~v1. ~e1 = ~u1

|~u1| .

Step 2. Let ~u2 = ~v2 − Proj~u1~v2. ~e2 = ~u2

|~u2| .

Step 3. Let ~u3 = ~v3 − Proj~u1~v3 − Proj~u2

~v3. ~e3 = ~u3

|~u3| .

Step 4. Let ~u4 = ~v4 − Proj~u1~v4 − Proj~u2

~v4 − Proj~u3~v4. ~e4 = ~u4

|~u4| .

•

•

Example. We will apply the Gram-Schmidt algorithm to orthonormalizethe set of vectors

~v1 =

1−11

, ~v2 =

101

, ~v3 =

112

.

To apply the Gram-Schmidt, we first need to check that the set of vectorsare linearly independent.∣∣∣∣∣∣

1 1 1−1 0 11 1 2

∣∣∣∣∣∣ = 1(0− 1)− 1((−1)(2)− (1)(1)) + 1((−1)(1)− 0) = 1 6= 0.

Therefore the vectors are linearly independent.Gram-Schmidt algorithm:

Step 1. Let

~u1 = ~v1 =

1−11

~e1 =

~u1

|~u1|=

1√3

1−11

.

Step 2. Let

~u2 = ~v2 − Proj~u1~v2

Proj~u1~v2 =

(1, 0, 1).(1,−1, 1)

12 + (−1)2 + 12

1−11

=2

3

1−11

~u2 =

101

− 2

3

1−11

=

1/32/31/3

~e2 =

~u2

|~u2|=

3√6

1/32/31/3

.

Step 3. Let

~u3 = ~v3 − Proj~u1~v3 − Proj~u2

~v3

Proj~u1~v3 =

(1, 1, 2).(1,−1, 1)

12 + (−1)2 + 11

1−11

=2

3

1−11

Proj~u2

~v3 =(1, 1, 2).(1/3, 2/3, 1/3)

(1/3)2 + (2/3)2 + (1/3)2

1/32/31/3

=5

2

1/32/31/3

~u3 =

112

− 2

3

1−11

− 5

2

1/32/31/3

=

−1/20

1/2

~e3 =

~u3

|~u3|=√

2

−1/20

1/2

.

Example. Consider the vectors {[3, 0, 4], [−1, 0, 7], [2, 9, 11]} Check that thevectors are linearly independent and use the Gram-Schmidt process to findorthogonal vectors.

Ans. {[3, 0, 4], [−4, 0, 3], [0, 9, 0]} Check that the vectors are mutually orthog-onal.

Summary

We assume ~v1, ~v2, ~v3 are vectors and a, b, c are real numbers for this summary.

1. To write vector ~v as a linear combination of vectors ~v1, ~v2, ~v3, we have tofind real numbers a, b, c such that

~v = a~v1 + b~v2 + c~v3.

This amounts to solving a system to linear equations.

2. A set of vectors ~v1, ~v2, ~v3 is linearly independent if the only solution of thevector equation

a~v1 + b~v2 + c~v3 = ~0,

is a = b = c = 0.

If one can find a non-zero solution, then the vectors are linearly dependent.i.e. we can express one vector as a linear combination of the remainingvectors.

3. We say that a set of vectors {~v1, ~v2, ..., ~vn} are mutually orthogonal if everypair of vectors is orthogonal. i.e.

~vi.~vj = 0, for all i 6= j.

4. A set of vectors S is orthonormal if every vector in S has magnitude 1and the set of vectors are mutually orthogonal.

5. The Gram-Schmidt Algorithm:Let v1, v2, ..., vn be a set of n linearly independent vectors in Rn. Thenwe can construct an orthonormal set of vectors as follows:

Step 1. Let ~u1 = ~v1. ~e1 = ~u1

|~u1| .

Step 2. Let ~u2 = ~v2 − Proj~u1~v2. ~e2 = ~u2

|~u2| .

Step 3. Let ~u3 = ~v3 − Proj~u1~v3 − Proj~u2

~v3. ~e3 = ~u3

|~u3| .

Step 4. Let ~u4 = ~v4 − Proj~u1~v4 − Proj~u2

~v4 − Proj~u3~v4. ~e4 = ~u4

|~u4| .

••

7 Matrices

7.1 Introduction

Definition. A m × n matrix is a rectangular array of numbers having mrows and n columns.

A =

a11 a12 . . . a1n

a21 a22 . . . a2n

a31 a32 . . . a3n

. . . .

. . . .am1 am2 . . . amn

The element aij represents the entry in the ith row and jth column. Wesometimes denote A by (aij)m×n.

7.2 Matrix Operations

Addition

We can only add two matrices of the same dimension i.e. same number ofrows and columns. We then add element-wise.

Example. (2 11 3

)+

(4 35 6

)=

(6 46 9

).

Scalar Multiplication

If c is a real number and A = (aij)m×n is a matrix then cA = (caij)m×n.

Example.

5

(2 11 3

)=

(10 55 15

).

Matrix Multiplication

Given a matrix A = (aij)m×n and a matrix B = (bij)r×s, we can only multiplythem if n = r. In such a case the multiplication is defined to be the matrix

C = (cij)m×s as follows:

cij =n∑

k=1

aikbkj.

We may view the cijth element as the dot product of the ith row of thematrix A and jth column of the matrix B.

Example. 1.(2 11 3

) (4 35 6

)=

((2)(4) + (1)(5) (2)(3) + (1)(6)(1)(4) + (3)(5) (1)(3) + (3)(6)

)=

(13 1219 21

).

2.

1 2 32 1 00 3 1

2 0 01 3 20 2 0

=

4 12 45 3 23 11 6

7.3 Special matrices

(a) A matrix is called a square matrix if the number of rows is equal to thenumber of columns.

(b) The transpose of a square matrix A = (aij) is the matrix AT = (aji).The rows of A become the columns of AT .

Example. The transpose of A =

1 2 32 1 00 3 1

is

AT =

1 2 02 1 33 0 1

.

(c) A square matrix is said to be symmetric if A = AT , i.e. aij = aji,∀i, j.

(d) A square matrix is said to be a diagonal matrix, if all the non-diagonalelements in the matrix are zero.

e.g.

3 0 00 −4 00 0 1

.

(e) A square matrix is said to be the identity matrix, if it is a diagonalmatrix and all the non-zero elements are 1.

e.g.

1 0 00 1 00 0 1

.

The identity marix has the property that

AI = A = IA

for all square matrices A.

7.4 Determinants

Definition. The determinant of a 2× 2 matrix is defined as follows:∣∣∣∣ a bc d

∣∣∣∣ = ad− bc.

Definition. The determinant of a 3× 3 matrix is defined as follows:∣∣∣∣∣∣a11 a12 a13

a21 a22 a23

a31 a32 a33

∣∣∣∣∣∣ = a11

∣∣∣∣ a22 a23

a32 a33

∣∣∣∣− a12

∣∣∣∣ a21 a23

a31 a33

∣∣∣∣ + a13

∣∣∣∣ a21 a22

a31 a33

∣∣∣∣= a11(a22a33 − a32a23)− a12(a21a33 − a31a23) + a13(a21a32 − a31a22)

Definition. The definition of determinants can be generalized for a squarematrix of any size. Let A = (aij) be a square n × n matrix. Fix a row i.Then the determinant is defined to be,

det (A) =n∑

j=1

(−1)i+jaijdet (Aij),

where Aij is the minor matrix obtained by deleting row i and column j. It iscalled the ij Cofactor of A. T

Example. Find the determinant of

A =

1 1 0 −10 −1 1 40 0 2 21 0 1 2

.

Choose row i = 1.

A11 =

−1 1 40 2 20 1 2

det (A11) = −2

A12 =

0 1 40 2 21 1 2

det (A11) = −6

A13 =

0 −1 40 0 21 0 2

det (A11) = −2

A14 =

0 −1 10 0 21 0 1

det (A11) =− 2

det (A) =n∑

j=1

(−1)1+ja1jdet (A1j)

= (−1)2(1)det (A11) + (−1)3(1)det (A12) + (−1)4(0)det (A13) + (−1)5(−1)det (A14)

= (1)(1)(−2) + (−1)(1)(−6) + (1)(0)(−2) + (−1)(−1)(−2)

= 2

7.5 Inverses

Definition. A matrix I which has 1’s on the diagonal and 0’s everywhereelse is called the identity matrix. This matrix has the property that AI = A.

Example. 1. A 2× 2 identity matrix has the form I =

(1 00 1

).

2. A 3× 3 identity matrix has the form I =

1 0 00 1 00 0 1

.

Definition. Let A = (aij) be a square matrix. A matrix B = (bij) is calledthe inverse of A if

AB = BA = I.

Remark. A matrix A has an inverse if and only if det(A) 6= 0.

Inverse of a 2× 2 matrix

Definition. The inverse of a 2× 2 matrix, A =

(a bc d

)is given by,

A−1 =1

ad− bc

(d −b−c a

).

Example. Find the inverse of the matrix A =

(1 11 2

).

Solution: We first check that the inverse of A exists!

det(A) = (1)(2)− (1)(1) = 1 6= 0.

Hence the inverse of A must exist and is given by

A−1 =1

1

(2 −1−1 1

).

We check that this is correct by multiplying A−1A to see if we get theidentity matrix.(

2 −1−1 1

) (1 11 2

)=

((2)(1) + (−1)(1) (1)(1) + (−1)(2)(−1)(1) + (1)(2) (−1)(1) + (1)(2)

)=

(1 00 1

).

We can find the inverse in another way: Write as (A|I2)(1 1 1 01 2 0 1

).

Then use matrix row operations to get it into the form (I2|B).(1 0 a b0 1 c d

).

You will find that B = A−1.(1 1 1 01 2 0 1

)R2=R2−R1→

(1 1 1 00 1 −1 1

)R1=R1−R2→

(1 0 2 −10 1 −1 1

)Observe that the matrix obtained(

a bc d

)=

(2 −1−1 1

)is the inverse of A.

Inverse of a 3× 3 matrix

The method is the same as in the 2× 2 case:

• Check determinant A is non-zero.

• Rewrite as (A|I3).

• Use row operations to put in the form (I3|A−1).

Example. Let A =

1 −1 10 −2 1−2 −3 0

. Find A−1.

1. Check that det(A) 6= 0.

det(A) = 1(0− (−3)(1))− (−1)(0− (−2)(1)) + 1(0− (−2)(−2))

= 3 + 2− 4

= 1 6= 0

Therefore the inverse exists.

2. Rewrite as (A|I3). 1 −1 1 1 0 00 −2 1 0 1 0−2 −3 0 0 0 1

.

3. Use row operations to put in the form (I3|A−1). 1 −1 1 1 0 00 −2 1 0 1 0−2 −3 0 0 0 1

R3=R3+2R1→

1 −1 1 1 0 00 −2 1 0 1 00 −5 2 2 0 1

R2=−1

2R2→

1 −1 1 1 0 00 1 −1/2 0 −1/2 00 −5 2 2 0 1

R3=R3+5R2→

1 −1 1 1 0 00 1 −1/2 0 −1/2 00 0 −1/2 2 −5/2 1

R3=−2R3→

1 −1 1 1 0 00 1 −1/2 0 −1/2 00 0 1 −4 5 −2

R2=R2+ 1

2R3→

1 −1 1 1 0 00 1 0 −2 2 −10 0 1 −4 5 −2

R1=R1−R3→

1 −1 0 5 −5 20 1 0 −2 2 −10 0 1 −4 5 −2

R1=R1+R2→

1 0 0 3 −3 10 1 0 −2 2 −10 0 1 −4 5 −2

4.

A−1 =

3 −3 1−2 2 −1−4 5 −2

.

Summary

1. The determinant of a 2× 2 matrix is defined as follows:∣∣∣∣ a bc d

∣∣∣∣ = ad− bc.

2. The determinant of a 3× 3 matrix is defined as follows:∣∣∣∣∣∣a11 a12 a13

a21 a22 a23

a31 a32 a33

∣∣∣∣∣∣ = a11

∣∣∣∣ a22 a23

a32 a33

∣∣∣∣− a12

∣∣∣∣ a21 a23

a31 a33

∣∣∣∣ + a13

∣∣∣∣ a21 a22

a31 a33

∣∣∣∣= a11(a22a33 − a32a23)− a12(a21a33 − a31a23) + a13(a21a32 − a31a22)

3. The method to find the inverse of a matrix:

• Check determinant A is non-zero.

• Rewrite as (A|I3).• Use row operations to put in the form (I3|A−1).

8 Eigenvalues, Eigenvectors and Diagonaliza-

tion

8.1 Introduction

Definition. Suppose that A is an n × n square matrix. Suppose also that~x is a non-zero vector in R

n and that λ is a scalar so that,

A~x = λ~x.

We then call ~x an eigenvector of A and λ an eigenvalue of A.

Example. Suppose A =

(4 21 3

).

Then

(21

)is an eigenvector associated to the eigenvalue 5 because

A~x =

(4 21 3

) (21

)= 5

(21

)= 5~x.

Also

(−11

)is an eigenvector associated to the eigenvalue 2 because

A~x =

(4 21 3

) (−11

)= 2

(−11

)= 2~x.

In this chapter we will learn how to find these eigenvalues and eigenvec-tors.

8.2 Method to find eigenvalues and eigenvectors

We start with A~x = λ~x and rewrite it as follows,

A~x = λI~x

λI~x− A~x = 0

(λI − A)~x = 0

Theorem λ is an eigenvalue of A if and only if λI − A is not invertible ifand only if det (λI − A) = 0.

Steps to find eigenvalues and eigenvectors:

1. Form the characteristic equation

det(λI − A) = 0.

2. To find all the eigenvalues of A, solve the characteristic equation.

3. For each eigenvalue λ, the corresponding set of eigenvectors are the non-zero solutions the linear system of equations

(λI − A)~x = 0

Example. Find the eigenvalues and eigenvectors of A.

1. A =

(4 21 3

).

Solution: (i) The characteristic equation of A.We must first find the matrix λI − A.

λI − A = λ

(1 00 1

)−

(4 21 3

)=

(λ 00 λ

)−

(4 21 3

)=

(λ− 4 2

1 λ− 3

)The characteristic equation:

det(λI − A) = 0∣∣∣∣ λ− 4 21 λ− 3

∣∣∣∣ = 0

(λ− 4)(λ− 3)− 2 = 0

λ2 − 7λ+ 12− 2 = 0

λ2 − 7λ+ 10 = 0

(ii) To find all the eigenvalues of A, solve the characteristic equa-tion.

λ2 − 7λ+ 10 = 0

(λ− 5)(λ− 2) = 0

So we have two eigenvalues λ1 = 2, λ2 = 5.

(iii) For each eigenvalue λ, the corresponding set of eigenvectorsare the non-zero solutions of the linear system of equationsgiven by,

(λI − A)~x = 0.

case(i) λ1 = 2.

(2I − A)~x = 0(2− 4 −2−1 2− 3

) (x1

x2

)=

(00

)(−2 −2−1 −1

) (x1

x2

)=

(00

)(−2x1 − 2x2

−x1 − x2

)=

(00

)We get two equations:

2x1 + 2x2 = 0

x1 + x2 = 0

Both equations give the relation x1 = −x2. Therefore the set ofeigenvectors corresponding to λ = 2 is given by:

{(x1

x2

)|x1 = −x2}

{(−x2

x2

)}

{ x2

(−11

)|x2 is a non-zero real number.}

An eigenvector corresponding to λ1 = 2 is

(−11

).

case(i) λ2 = 5.

(5I − A)~x = 0(5− 4 −2−1 5− 3

) (x1

x2

)=

(00

)(

1 −2−1 2

) (x1

x2

)=

(00

)(

x1 − 2x2

−x1 + 2x2

)=

(00

)

We get two equations:

x1 − 2x2 = 0

−x1 + 2x2 = 0

Both equations give the relation x1 = 2x2. Therefore a generaleigenvector corresponding to λ2 = 2 is,

{(x1

x2

)|x1 = 2x2}

{(

2x2

x2

)}

{ x2

(21

)|x2 is a non-zero real number.}

An eigenvector corresponding to λ2 = 5 is

(21

).

2. An example of three distinct eigenvalues.

A =

4 0 1−1 −6 −25 0 0

.

Solution: (i) Form the Characteristic Equation.The characteristic equation is:

det (λI − A) = 0∣∣∣∣∣∣λ− 4 0 −1

1 λ+ 6 2−5 0 λ

∣∣∣∣∣∣ = 0

(λ− 4)((λ+ 6)(λ)− 0)− 1(0− (−5)(λ+ 6)) = 0

λ3 + 2λ2 − 29λ− 30 = 0

(ii) Find the eigenvalues.We need to solve the characteristic equation. i.e. we need to factor-ize the characteristic polynomial. We can factorize it by either using

long division or by directly trying to spot a common factor.

Method 1: Long Division.We want to factorize this cubic polynomial. In general it is quite dif-ficult to guess what the factors may be. We try λ = ±1,±2,±3, etc.and hope to quickly find one factor. Let us try λ = −1. We dividethe polynomial λ3 + 2λ2 − 29λ− 30 by λ+ 1, to get,

λ2 +λ −30λ+ 1 |λ3 +2λ2 −29λ −30;

−λ3 +λ2

λ2 −29λ −30− λ2 +λ

−30λ −30− −30λ −30

0

The quotient is λ2 + λ− 30.The remainder is 0.Therefore λ3 + 2λ2 − 29λ− 30 = (λ+ 1)(λ2 + λ− 30) + 0.

λ3 + 2λ2 − 29λ− 30 = 0

(λ+ 1)(λ2 + λ− 30) = 0

(λ+ 1)(λ+ 6)(λ− 5) = 0

Therefore the eigenvalues are: {−1,−6, 5}.

Method 2: Direct factorization by spotting common factor.∣∣∣∣∣∣λ− 4 0 0

1 λ+ 6 2−5 0 λ

∣∣∣∣∣∣ = 0

(λ− 4)((λ+ 6)(λ)− 0)− 1(0− (−5)(λ+ 6)) = 0

(λ− 4)((λ+ 6)(λ))− 5(λ+ 6) = 0

(λ+ 6)(λ(λ− 4)− 5) = 0

(λ+ 6)(λ2 − 4λ− 5) = 0

(λ+ 6)(λ− 5)(λ+ 1) = 0

Therefore the eigenvalues of A are: λ1 = −1, λ2 = −6 and λ3 = 5.

(iii) Find Eigenvectors corresponding to each Eigenvalue:

We now need to find eigenvectors corresponding to each eigenvalue.

case(i) λ1 = −1.The eigenvectors are the solution space of the following system: −5 0 −1

1 5 2−5 0 −1

x1

x2

x3

=

000

−5x1 − x3 = 0; x1 =

−1

5x3

x1 + 5x2 + 2x3 = 0; x2 =−9

25x3

The set of eigenvectors corresponding to λ1 = −1 is,

{

x1

x2

x3

|x1 =−1

5x3, x2 =

−9

25x3}

{

−15x3

−925x3

x3

|x3 is a non-zero real number}

{ x3

−15−925

1

|x3 is a nn-zero real number}

An eigenvector corresponding to λ1 = −1 is

−15−925

1

.

case(ii) λ2 = −6.The eigenvectors are the solution space of the following system: −10 0 −1

1 0 2−5 0 −6

x1

x2

x3

=

000

−10x1 − x3 = 0; x3 = −10x1

x1 + 2x3 = 0; x1 = −2x3 = −2(−10)x1

x1 = 0 = x3

The set of eigenvectors corresponding to λ2 = −6 is,

{

x1

x2

x3

|x1 = x3 = 0}

{

0x2

0

|x2 is a non-zero real number}

{ x2

010

|x2 is a non-zero real number}

An eigenvector corresponding to λ2 = −6 is

010

.

case(iii) λ3 = 5.The eigenvectors are the solution space of the following system: 1 0 −1

1 11 2−5 0 5

x1

x2

x3

=

000

x1 = x3

x2 =−3

11x3

The set of eigenvectors corresponding to λ3 = 5 is,

{

x1

x2

x3

|x1 = x3, x2 =−3

11x3}

{

x3−311x3

x3

|x3 is a non-zero real number}

{ x3

1−311

1

|x3 is a non-zero real number}

An eigenvector corresponding to λ3 = 5 is

1−311

1

.

3. An example of repeated eigenvalue having two eigenvectors.

A =

0 1 11 0 11 1 0

.

Solution: (i) Form the Characteristic Equation.The characteristic equation is: ∣∣∣∣∣∣

λ −1 −1−1 λ −1−1 −1 λ

∣∣∣∣∣∣ = 0

λ(λ2 − (−1)(−1))− (−1)((−1)λ− (−1)(−1))− 1((−1)λ− (−1)(−1)) = 0

λ3 − 3λ2 − 2 = 0

(ii) Find the eigenvalues.We need to solve the characteristic equation. i.e. we need to factor-ize the characteristic polynomial. We can factorize it by either usinglong division or by directly trying to spot a common factor.

Method 1: Long Division.We want to factorize this cubic polynomial. In general it is quite dif-ficult to guess what the factors may be. We try λ = ±1,±2,±3, etc.and hope to quickly find one factor. Let us try λ = −1. We dividethe polynomial λ3 + 2λ2 − 29λ− 30 by λ+ 1, to get,

λ2 −λ −2λ+ 1 |λ3 −3λ −2;

−λ3 +λ2

−λ2 −3λ −2− −λ2 −λ

−2λ −2− −2λ −2

0

The quotient is λ2 − λ− 2.The remainder is 0.

Therefore λ3 − 3λ− 2 = (λ+ 1)(λ2 − λ− 2) + 0.

λ3 − 3λ− 2 = 0

(λ+ 1)(λ2 − λ− 2) = 0

(λ+ 1)(λ+ 1)(λ− 2) = 0

Therefore the eigenvalues are: {−1, 2}.

Method 2: Direct factorization by spotting common factor.

∣∣∣∣∣∣λ −1 −1−1 λ −1−1 −1 λ

∣∣∣∣∣∣ = 0

λ(λ2 − (−1)(−1))− (−1)((−1)λ− (−1)(−1))− 1((−1)λ− (−1)(−1)) = 0

λ(λ2 − 1) + 1(−λ− 1)− 1(1 + λ) = 0

λ(λ− 1)(λ+ 1)− 1(λ+ 1)− 1(1 + λ) = 0

(λ+ 1)(λ(λ− 1)− 1− 1) = 0

(λ+ 1)(λ2 − λ− 2) = 0

(λ+ 1)2(λ− 2) = 0

Therefore the eigenvalues of A are: {−1, 2}.(iii) Find Eigenvectors corresponding to each Eigenvalue:

We now need to find eigenvectors corresponding to each eigenvalue.

case(i) λ1 = −1.The eigenvectors are the solution space of the following system: −1 −1 −1

−1 −1 −1−1 −1 −1

x1

x2

x3

=

000

−x1 − x2 − x3 = 0; x1 = −x2 − x3

The set of eigenvectors corresponding to λ1 = −1 is,

{

x1

x2

x3

|x1 = −x2 − x3}

{

−x2 − x3

x2

x3

| at least one of x2 and x3 is a non-zero real number}

{ x2

−110

+ x3

−101

| at least one of x2 and x3 is a non-zero real number}

We therefore can get two linearly independent eigenvectors cor-responding to λ1 = −1:

~v1 =

−110

, ~v2 =

−101

.

case(ii) λ2 = 2.The eigenvectors are the solution space of the following system: 2 −1 −1

−1 2 −1−1 −1 2

x1

x2

x3

=

000

2x1 − x2 − x3 = 0

−x1 + 2x2 − x3 = 0

−x1 − x2 + 2x3 = 0

Since this system of equations looks fairly complicated, it may

be a good idea to use row reduction to simplify the system. 2 −1 −1 0−1 2 −1 0−1 −1 2 0

R3↔R1→

−1 −1 2 0−1 2 −1 02 −1 −1 0

R1=(−1)R1→

1 1 −2 0−1 2 −1 02 −1 −1 0

R3=R2−2R1−−−−−−−→R2=R1+R2

1 1 −2 00 3 −3 00 −3 3 0

R2=1/3R2→

1 1 −2 00 1 −1 00 −3 3 0

R3=R3+3R2→

1 1 −2 00 1 −1 00 0 0 0

The resulting set of equations is:

x2 − x3 = 0 =⇒ x2 = x3

x1 + x2 − 2x3 = 0 =⇒ x1 = x3

The set of eigenvectors corresponding to λ2 = 2 is,

{

x1

x2

x3

|x1 = x2 = x3}

{

x3

x3

x3

|x3 is a non-zero real number}

{ x3

111

|x3 is a non-zero real number}

An eigenvector corresponding to λ2 = 2 is

111

.

4. An example of a repeated eigenvalue having only one linearly independenteigenvector.

A =

6 3 −80 −2 01 0 −3

.

Solution: The characteristic polynomial for A:∣∣∣∣∣∣λ− 6 −3 8

0 λ+ 2 0−1 0 λ+ 3

∣∣∣∣∣∣ = 0

(λ− 6)((λ+ 2)(λ+ 3)− 0)− 3(0− 0) + 8(0− (−1)(λ+ 2)) = 0

λ3 − λ2 − 16λ− 20 = 0

(λ+ 2)(λ2 − 3λ− 10) = 0

(λ+ 2)2(λ− 5) = 0

Therefore the eigenvalues of A are: λ1 = −2, λ2 = 5.

case(i) λ1 = −2.

The eigenvectors are the solution space of the following system: −8 −3 80 0 0−1 0 1

x1

x2

x3

=

000

−x1 + x3 = 0 =⇒ x1 = x3

−8x1 − 3x2 + 8x3 = 0 =⇒ x2 = 0

The set of eigenvectors corresponding to λ1 = −2 is,

{

x1

x2

x3

|x1 = x3, x2 = 0}

{

x3

0x3

|x3 is a non-zero real number}

{ x3

101

|x3 is a non-zero real number}

An eigenvector corresponding to λ1 = −2 is

101

.

case(ii) λ2 = 5.The eigenvectors are the solution space of the following system: −1 −3 8

0 7 0−1 0 8

x1

x2

x3

=

000

−x1 + 8x3 = 0 =⇒ x1 = 8x3

7x2 = 0 =⇒ x2 = 0

The set of eigenvectors corresponding to λ2 = 5 is,

{

x1

x2

x3

|x1 = 8x3, x2 = 0}

{

8x3

0x3

|x3 is a non-zero real number}

{ x3

801

|x3 is a non-zero real number}

An eigenvector corresponding to λ2 = 5 is

801

.

5. An example of an eigenvalue repeated three times having only two linearlyindependent eigenvectors.

A =

4 0 −10 3 01 0 2

.

Solution: The characteristic equation for A:

det (λI − A) = 0∣∣∣∣∣∣λ− 4 0 1

0 λ− 3 0−1 0 λ− 2

∣∣∣∣∣∣ = 0

λ3 − 9λ2 + 27λ− 27 = 0

(λ− 3)3 = 0

Therefore A has only one eigenvalue: λ = 3.

We now find the eigenvectors corresponding to λ = 3: −1 0 10 0 0−1 0 1

x1

x2

x3

=

000

−x1 + x3 = 0 =⇒ x1 = x3

The set of eigenvectors corresponding to λ = 3 is,

{

x1

x2

x3

|x1 = x3}

{

x3

x2

x3

| at least one of x2 and x3 is a non-zero real number}

{ x2

010

+ x3

101

| at least one on x2 and x3 is a non-zero real number}

Therefore we can find two linearly independent eigenvectors correspondingto the eigenvalue λ = 3: 0

10

and

101

.

8.3 Dimension of Eigenspace

This section is optional for the exam. Here we will see why a repeatedeigenvalue can have one or more linearly independent eigenvectors associatedto it. First some definitions.

Definition. The rank of a matrix is defined to be the number of linearlyindependent rows or columns.

Remark. We use the following results to determine the rank of a matrix.

• Let A be a square matrix and let Aechelon be the row echelon form ofthe matrix. Then both A and Aechelon have the same number of linearlyindependent rows.

• Therefore, Rank(A) = Rank(Aechelon).

• But Rank(Aechelon)= number of linearly independent rows of Aechelon

= number of non-zero rows of Aechelon

• We may conclude that

Rank(A) = Rank(Aechelon) = number of non-zero rows in Aechelon.

Example. 1. Rank

−1 −1 −1−1 −1 −1−1 −1 −1

= Rank

1 1 10 0 00 0 0

= 1.

2. Rank

−8 −3 80 0 0−1 0 1

= Rank

1 0 −10 −3 00 0 0

= 2.

3. Rank

−1 0 10 0 0−1 0 1

= Rank

−1 0 10 0 00 0 0

= 1.

Definition. Let A be a square matrix of size n×n having an eigenvalue λ.The set of eigenvectors corresponding to the eigenvalue λ, alongwith the zerovector, is called the eigenspace corresponding to eigenvalue λ.The dimension of the eigenspace is defined to be the maximum number oflinearly independent vectors in the eigenspace.

Theorem Let A be a matrix of size n × n and let λ be an eigenvalue of Arepeating k times. Then the dimension of the eigenspace of λ or equivalentlythe maximum number of linearly independent eigenvectors corresponding toλ is:

n− Rank(λI − A).

Example. Let us use this result to see why some repeated eigenvalues hadone or more linearly independent eigenvalues in the examples in the previoussection.

1. Recall A =

0 1 11 0 11 1 0

• has eigenvalue λ = −1 repeating 2 times.

• We found two linearly independent eigenvectors corresponding to thiseigenvalue.

~v1 =

−110

, ~v2 =

−101

.

We will use the above theorem to confirm that we should indeed expectto find two linearly independent eigenvectors. By the above theorem, themaximum number of linearly independent eigenvectors corresponding toλ = −1 is:

= 3− Rank(λI − A)

= 3− Rank

−1 −1 −1−1 −1 −1−1 −1 −1

= 3− 1

= 2

2. Recall A =

6 3 −80 −2 0−1 0 1

• has eigenvalue λ = −2 repeating 2 times.

• We found only one linearly independent eigenvector correspondingto this eigenvalue.

~v1 =

101

.

We will use the above theorem to confirm that we should indeed expectto find only one linearly independent eigenvector. By the above theorem,the maximum number of linearly independent eigenvectors correspondingto λ = −2 is:

= 3− Rank(λI − A)

= 3− Rank

−8 −3 80 0 0−1 0 1

= 3− 2

= 1

3. Recall A =

4 0 −10 3 01 0 2

• has eigenvalue λ = 3 repeating 3 times.

• We found only two linearly independent eigenvectors correspondingto this eigenvalue.

~v1 =

010

, ~v2 =

101

.

We will use the above theorem to confirm that we should indeed expect tofind only two linearly independent eigenvectors. By the above theorem,the maximum number of linearly independent eigenvectors corresponding

to λ = 3 is:

= 3− Rank(λI − A)

= 3− Rank

−1 0 10 0 0−1 0 1

= 3− 1

= 2

8.4 Diagonalization

Definition. A square matrix of size n is called a diagonal matrix if it is ofthe form

d1 0 . . . 00 d2 0 . . 0. . .. . .0 . . dn−1 00 . . . 0 dn

Definition. Two square matrices A and B of size n are said to be similar,if there exists an invertible matrix P such that

B = P−1AP

Definition. A matrix A is diagonalizable if A is similar to a diagonal ma-trix. i.e. there exists an invertible matrix P such that P−1AP = D where Dis a diagonal matrix. In this case, P is called a diagonalizing matrix.

Example. Recall the example we did in the last chapter.

A =

(4 21 3

).

Solution: In previous lectures we have found,the eigenvalues of A are: λ1 = 2, λ2 = 5.

An eigenvector corresponding to λ1 = 2 is

(−11

).

An eigenvector corresponding to λ2 = 5 is

(21

).

Define a new matrix P such that its columns are the eigenvectors of A.

P =

(−1 21 1

).

Note that since eigenvectors corresponding to distinct eigenvalues are linearlyindependent, the columns of P are linearly independent, hence P is an in-vertible matrix. (We can also confirm this by verifying that the determinantof P is non-zero).

P−1 =−1

3

(1 −2−1 −1

)

P−1AP =−1

3

(1 −2−1 −1

) (4 21 3

) (−1 21 1

)=

−1

3

(2 −4−5 −5

) (−1 21 1

)=

−1

3

(−6 00 −15

)=

(2 00 5

),

which is a diagonal matrix. Therefore A is diagonalizable and a diagonalizingmatrix for A is

P =

(−1 21 1

),

and the corresponding diagonal matrix D is

D =

(2 00 5

).

Theorem Let A be a square matrix of size n× n. Then A is diagonalizableif and only if there exists n linearly independent eigenvectors. In this case adiagonalizing matrix P is formed by taking as its columns the eigenvectors ofA,

P−1AP = D.

where D is a diagonal matrix whose diagonal entries are eigenvalues corre-sponding to the eigenvectors of A. The ith diagonal entry is the eigenvaluecorresponding to the eigenvector in the ith column of P .

Proposition Let A be a square matrix of size n× n. If λ1 and λ2 are twodistinct eigenvalues of A, and ~v1 and ~v2 are eigenvectors corresponding to λ1

and λ2 respectively, then ~v1 and ~v2 are linearly independent.

Corollary Let A be a square matrix of size n. If A has n distinct eigen-values then A is diagonalizable.

Example. For the following matrices:

(i) Find the eigenvalues and eigenvectors of A.

(ii) Is A diagonalizable?

(iii) If it is diagonalizable, find a diagonalizing matrix P such that P−1AP =D, where D is a diagonal matrix.

1.

A =

4 0 1−1 −6 −25 0 0

.

Solution: (i) In previous lectures we have found,The eigenvalues of A are: λ1 = −1, λ2 = −6 and λ3 = 5.

An eigenvector corresponding to λ1 = −1 is

−15−925

1

.

An eigenvector corresponding to λ2 = −6 is

010

.

An eigenvector corresponding to λ3 = 5 is

1−311

1

.

(ii) The matrix A is diagonalizable, since A is a square matrix of size3 × 3 and we have found three linearly independent eigenvectorsfor A.

(iii) A diagonalizing matrix P has the eigenvectors as its columns. Let

P =

−15

0 1−925

1 −311

1 0 1

.

The corresponding diagonal matrix D will have the eigenvalues asdiagonal entries corresponding to the columns of P . i.e. the ithdiagonal entry will be the eigenvalue corresponding to the eigen-vector in the ith column.

D =

−1 0 00 −6 00 0 5

.

We will now verify that P−1AP = D.

AP =

4 0 1−1 −6 −25 0 0

−15

0 1−925

1 −311

1 0 1

=

1/5 0 59/25 −6 −15/11−1 0 5

PD =

−15

0 1−925

1 −311

1 0 1

−1 0 00 −6 00 0 5

=

1/5 0 59/25 −6 −15/11−1 0 5

AP = PD (1)

Since the columns of P are linearly independent, P has non-zero de-terminant and is therefore invertible. We multiply (1) by P−1 on bothsides.

P−1AP = P−1PD,

= ID

= D

2. An example where the matrix is not diagonalizable.

A =

6 3 −80 −2 0−1 0 1

.

Solution: In previous lectures we have found,A had two eigenvalues:λ1 = −1 occurring with multiplicity 2, andλ2 = 5 occurring with multiplicity 1.We could find only one linearly independent eigenvector correspondingto λ1 = −1,

and one linearly independent eigenvector corresponding to λ2 = 5.Therefore we could find only two linearly independent eigenvectors. Bythe above theorem, A is not diagonalizable.

8.5 Diagonalization of symmetric matrices

Definition. Let A be a square matrix of size n. A is a symmetric matrixif AT = A

Definition. A matrix P is said to be orthogonal if its columns are mutuallyorthogonal.

Definition. A matrix P is said to be orthonormal if its columns are unitvectors and P is orthogonal.

Proposition An orthonormal matrix P has the property that

P−1 = P T .

Theorem If A is a real symmetric matrix then there exists an orthonormalmatrix P such that

(i) P TAP = D, where D a diagonal matrix.

(ii) The diagonal entries of D are the eigenvalues of A.

(iii) If λi 6= λj then the eigenvectors are orthogonal.

(iv) The column vectors of P are linearly independent eigenvectors of A,that are mutually orthogonal.

Example. Recall

A =

0 1 11 0 11 1 0

.

(i) Find the eigenvalues and eigenvectors of A.

(ii) Is A diagonalizable?

(iii) Find an orthonormal matrix P such that P TAP = D, where D is adiagonal matrix.

Solution: We have found the eigenvalues and eigenvectors of this matrixin a previous lecture.

(i), (ii) Observe that A is a real symmetric matrix. By the above theorem, weknow that A is diagonalizable. i.e. we will be able to find a sufficientnumber of linearly independent eigenvectors.The eigenvalues of A were; −1, 2. We found two linearly independent

eigenvectors corresponding to λ1 = −1: ~v1 =

−110

, ~v2 =

−101

.

And one eigenvector corresponding to λ2 = 2:

111

.

(iii) We now want to find an orthonormal diagonalizing matrix P .Since A is a real symmetric matrix, eigenvectors corresponding to dis-tinct eigenvalues are orthogonal. i.e. 1

11

is orthogonal to

−110

and

−101

.

However the eigenvectors corresponding to eigenvalue λ1 = −1, ~v1 = −110

and ~v2 =

−101

are not orthogonal to each other, since

we chose them from the eigenspace by making arbitrary choices*. Wewill have to use Gram Schmidt to get two corresponding orthogonal

vectors.

~u1 = ~v1

Proj~u1~v2 =

((−1, 1, 0).(−1, 0, 1))

2

−110

=

1

2

−110

~u2 = ~v2 − Proj~u1

~v2

=

−101

−

−1/21/20

=

−1/2−1/2

1

Note that since ~u1 = ~v1 and ~u2 is a linear combination of ~v1 and ~v2,the two new vectors are also eigenvectors of λ = −1. We now have aset of orthogonal eigenvectors:

{

111

,

−110

,

−1/2−1/2

1

}.We normalize the vectors to get a set of orthonormal vectors:

{

1/√

3

1/√

3

1/√

3

,

−1/√

2

1/√

20

,

−1/√

6

−1/√

6

2/√

6

}.We are now finally ready to write the orthonormal diagonalizing matrix:

P =

1√

3 −1/√

2 −1/√

6

1√

3 1/√

2 −1/√

6

1√

3 0 2/√

6

and the corresponding diagonal matrix D

D =

2 0 00 −1 00 0 −1

.

We will now verify that P TAP = D.

AP =

0 1 11 0 11 1 0

1√

3 −1/√

2 −1/√

6

1√

3 1/√

2 −1/√

6

1√

3 0 2/√

6

=

2√

3 1/√

2 1/√

6

2√

3 −1/√

2 1/√

6

2√

3 0 −2/√

6

PD =

1√

3 −1/√

2 −1/√

6

1√

3 1/√

2 −1/√

6

1√

3 0 2/√

6

2 0 00 −1 00 0 −1

=

2√

3 1/√

2 1/√

6

2√

3 −1/√

2 1/√

6

2√

3 0 −2/√

6

AP = PD (1)

Since the columns of P are linearly independent, P has non-zero determinantand is therefore invertible. We multiply (1) by P−1 on both sides.

P−1AP = P−1PD,

= ID

= D (2)

Also since P is orthonormal, we have

P−1 = P T

i.e. PP T = I = P TP . 1√

3 −1/√

2 −1/√

6

1√

3 1/√

2 −1/√

6

1√

3 0 2/√

6

1√

3 1/√

3 1/√

3

−1√

2 1/√

2 0

−1√

6 −1/√

6 2/√

6

=

1 0 00 1 00 0 1

.

Therefore from (2) and since P−1 = P T we finally get the relation

P TAP = D.

Note. *:Look back at how we selected the eigenvecors ~v1 and ~v2; we chosex2 = 1, x3 = 0 to get ~v1 and x2 = 0, x3 = 1 to get ~v2. If we had chosenx2 = 1, x3 = 0 to get ~v1 and x2 = −1/2, x3 = 1 to get ~v2, then ~v1 and ~v2

would be orthogonal. However it is much easier to make arbitrary choicesfor x1 and x2 and then use the Gram Schmidt Process to orthogonalize thevectors as we have done in this example.

Summary

1. Suppose that A is an n × n square matrix. Suppose also that ~x is anon-zero vector in R

n and that λ is a scalar so that,

A~x = λ~x.

We then call ~x an eigenvector of A and λ an eigenvalue of A.

2. Steps to find eigenvalues and eigenvectors:

(i) The characteristic equation of A is:

det(λI − A) = 0.

(ii) To find all the eigenvalues of A, solve the characteristic equation forλ.

(iii) For each eigenvalue λ, to find the corresponding set of eigenvectors,find a non-zero solution to the linear system of equations:

(λI − A)~x = 0

3. Let A be a square matrix of size 3 × 3. Then A is diagonalizable ifand only if there exists 3 linearly independent eigenvectors. In this case adiagonalizing matrix P is formed by taking as its columns the eigenvectorsof A. Then we have

P−1AP = D.

where D is a diagonal matrix whose ith diagonal entry is the eigenvaluecorresponding to the eigenvector in the ith column of P .

4. If A is a real symmetric matrix of size 3× 3 then A is diagonalizable andthere exists an othogonal matrix P such that

(i) P TAP = D, D a diagonal matrix.

(ii) The diagonal entries of D are the eigenvalues of A

(iii) The column vectors of P are eigenvectors of A

(iv) If λi 6= λj then the eigenvectors are orthogonal.

Note. Point (iv) above says that if we have 3 distinct eigenvalues, thenthe 3 corresponding eigenvectors will be mutually orthogonal and we candefine P to be the matrix whose columns are these eigenvectors.

However if we have a repeated root, then the 2 eigenvectors correspondingto the repeated eigenvalue will not necessarily be orthogonal. Here we willhave to use Gram Schmidt to orthogonalize the vectors. Note that theorthogonalized vectors will also be eigenvectors of the repeated eigenvalue.

References

1. http://tutorial.math.lamar.edu/Classes/LinAlg/EVal EVect Intro.aspx

2. http://www.sosmath.com/matrix/diagonal/diagonal.html

3. K. A. Stroud, Engineering Mathematics

9 Operators and Commutators

9.1 Operators

Operators are commonly used to perform a specific mathematical operationon another function. The operation can be to take the derivative or integratewith respect to a particular term, or to multiply, divide, add or subtract anumber or term with regards to the initial function. Operators are commonlyused in physics, mathematics and chemistry.

Example. 1. A regular function can be thought of as an operator.

(i)f : x 7→ ax,

where a is a real number.Operator: the function fOperates on: real numbersAction: multiply by a.

(ii) Squaringf : x 7→ x2

Operator: the function fOperates on: real numbersAction: squaring

2. Operators which acts on functions:

(i)a : f(x) 7→ af(x),

where a is a real number.Operator: aOperates on: scalar functionsAction: multiply by a.

(ii)x : f(x) 7→ xf(x),

Operator: xOperates on: scalar functionsAction: multiply by x.

(iii) Differentiation Operator

D : f(x) 7→ d

dxf(x)

Operator: DOperates on: scalar functionsAction: Differentiate with respect to x.We can similarly define partial differentiation operators, grad opera-tor, integral operator, etc.

(iv) The Momentum Operator

P =~i

∂

∂x

where:

– ~ is Planck’s constant,

– i is the imaginary unit.

P =~i

∂

∂x: ψ(x, t) 7→ ~

i

∂ψ(x, t)

∂x

Operator: POperates on: wave function, ψ(x, t).

(iv) The Energy Operator

E =~i

∂

∂twhere:

– ~ is Planck’s constant,

– i is the imaginary unit.

E =~i

∂

∂t: ψ(x, t) 7→ ~

i

∂ψ(x, t)

∂t

Operator: EOperates on: wave function, ψ(x, t).

(iv) The Hamiltonian Operator

H = − ~2

2m

∂2

∂x2

where:

– ~ is Planck’s constant,

– m is the mass of the particle (regard as constant).

– i is the imaginary unit.

H = − ~2

2m

∂2

∂x2: ψ(x, t) 7→ − ~2

2m

∂2

∂ψ(x, t)x2

Operator: HOperates on: wave function, ψ(x, t).

3. Operators which act on vectors.

(i) Let A =

(1 10 1

)be a square matrix and ~v =

(x1

x2

)be an arbi-

trary vector. Define an operator in the following manner:

A : ~v 7→ A~v(x1

x2

)7→

(1 10 1

) (x1

x2

)=

(x1 + x2

x2

).

Operator: AOperates on: two dimensional vectors

Action: maps a vector

(x1

x2

)to

(x1 + x2

x2

).

9.2 Linear Operators

An operator O is a linear operator if it satisfies the following two conditions:

(i) O(f + g) = O(f) +O(g).

(ii) O(λf) = λO(f), where λ is a scalar.

Example. Determine if the following operators are linear:

1. O = a : f(x) 7→ af(x).

Solution:

O(f + g) = a(f + g)

= af + ag

= O(f) +O(g)

O(λf) = a(λf)

= λaf

= λO(f)

Therefore O is linear.

2. O = squaring : x 7→ x2.

Solution:

O(x+ y) = (x+ y)2

= x2 + y2 + 2xy

= O(x) +O(y) + 2xy

O(1 + 2) = O(3) = 9

O(1) +O(2) = 1 + 4 = 5

Therefore O is not linear.

3. D : f(x) 7→ ddxf(x).

Solution:

D(f + g) =d

dx(f + g)

=d

dxf +

d

dxg

= D(f) +D(g)

D(λf) =d

dx(λf)

= λd

dxf

= λD(f)

Therefore D is linear.

4. A =

(1 10 1

): ~v 7→ A~v.

Solution:

A(~v + ~w) =

(1 10 1

)(

(x1

x2

)+

(y1

y2

))

=

(1 10 1

)(

(x1 + y1

x2 + y2

))

=

((x1 + y1) + (x2 + y2)

x2 + y2

)=

(x1 + x2

x2

)+

(y1 + y2

y2

)=

(1 10 1

) (x1

x2

)+

(1 10 1

) (y1

y2

)= A(~v) + A(~w)

A(λ~v) =

(1 10 1

)(λ

(x1

x2

))

=

(1 10 1

) (λx1

λx2

)=

(λx1 + λx2

λx2

)= λ

(x1 + x2

x2

)= λ

(1 10 1

) (x1

x2

)= λA~v

Therefore A is linear.

9.3 Composing Operators

When we have two operators and we want to apply them to a function insequence, we use the notation O1 ◦ O2.

O1 ◦ O2 : f 7→ O1(O2(f)).

First apply O2 and then O1. The order is important.

Example. 1. Consider the following two operators.

O1 : f(x) 7→ xf(x)

O2 : g(x) 7→ (g(x))2

Find O1 ◦ O2 and O2 ◦ O1.

Solution: We first find O1 ◦ O2.

O1 ◦ O2(h(x)) = O1(O2(h(x))

= O1((h(x)2)

= x(h(x)2)

O1 ◦ O2 : h(x) 7→ x(h(x)2

We now find O2 ◦ O1.

O2 ◦ O1(h(x)) = O2(O1(h(x))

= O2(xh(x))

= x2(h(x)2)

O2 ◦ O1 : h(x) 7→ x2(h(x)2)

Observe that O1 ◦O2 6= O2 ◦O1. Hence the order in which you apply theoperators is important.

9.4 Commutators

Let OA and OB be two operators. The commutator of OA and OB is theoperator defined as

[OA,OB] = OA ◦ OB −OB ◦ OA.

Example. 1. Consider the following two operators.

OA : ~v 7→ A~v, A =

(1 00 0

)OB : ~v 7→ B~v, B =

(1 11 0

)Find [OA,OB].

Solution: We first find OA ◦ OB.

OA ◦ OB(~v) = OA(OB(~v))

= OA(B~v)

= AB~v

OA ◦ OB : ~v 7→ AB~v, AB =

(1 10 0

)(x1

x2

)7→

(1 10 0

) (x1

x2

)=

(x1 + x2

0

)We now find O2 ◦ O1.

OB ◦ OA(~v) = OB(OA(~v))

= OB(A~v)

= BA~v

OB ◦ OA : ~v 7→ BA~v, BA =

(1 01 0

)(x1

x2

)7→

(1 01 0

) (x1

x2

)=

(x1

x1

)Therefore the commutator of OA and OB is

[OA,OB] = OA ◦ OB −OB ◦ OA

[OA,OB](~v) = (OA ◦ OB −OB ◦ OA)(~v)

= (AB −BA)(~v)

2. Consider the following two operators.

D : f(x) 7→ (D)f(x) =d

dxf(x)

xD : f(x) 7→ (xD)f(x) = xd

dxf(x)

Find [D, xD].

Solution:

(D ◦ xD)(f(x)) = D(xD(f(x)))

= D(xd

dxf(x))

=d

dx(x

d

dxf(x))

= xd2f

dx2+df

dx= (xD2 +D)(f(x))

(xD ◦D)(f(x)) = xD(D(f(x)))

= xD(d

dxf(x))

= xd

dx(d

dxf(x))

= xd2f

dx2

= (xD2)(f(x))

Therefore the commutator of D and xD is

[D, xD] = D ◦ xD − xD ◦D= xD2 +D − xD2

= D

3. Consider the two operators

O1 = D + 1 : f(x) 7→ (D + 1)f(x) =d

dxf(x) + f(x)

O1 = D + 2 : f(x) 7→ (D + 2)f(x) =d

dxf(x) + 2f(x)

Find the commutator [O1,O2].

Solution:

O1 ◦ O2(f(x)) = O1(O2(f(x)))

= O1(d

dxf(x) + 2f(x))

=d

dx(d

dxf(x) + 2f(x)) + (

d

dxf(x) + 2f(x))

=d2f

dx2+ 2

df

dx+df

dx+ 2f

= (D2 + 3D + 2)(f(x))

O2 ◦ O1(f(x)) = O2(O1(f(x)))

= O2(d

dxf(x) + 2f(x))

=d

dx(d

dxf(x) + f(x)) + 2(

d

dxf(x) + f(x))

=d2f

dx2+df

dx+ 2

df

dx+ 2f

= (D2 + 3D + 2)(f(x))

Therefore the commutator of O1 and O2 is

[O1,O2] = O1 ◦ O2 −O2 ◦ O1

= (D2 + 3D + 2)− (D2 + 3D + 2) = 0

4. Find the commutator [x ∂∂y, y ∂

∂x].

Solution:

[x∂

∂y, y

∂

∂x] = x

∂

∂y◦ y ∂

∂x− y

∂

∂x◦ x ∂

∂y

(x∂

∂y◦ y ∂

∂x)(f) = x

∂

∂y(y∂

∂x(f)

= x(y∂2f

∂y∂x+∂f

∂x)

= (xy∂2

∂y∂x+ x

∂

∂x)(f)

(y∂

∂x◦ x ∂

∂y)(f) = y

∂

∂x(x

∂

∂y(f)

= y(x∂2f

∂x∂y+∂f

∂y)

= (xy∂2

∂x∂y+ y

∂

∂x)(f)

Therefore the commutator is:

[x∂

∂y, y

∂

∂x] = x

∂

∂y◦ y ∂

∂x− y

∂

∂x◦ x ∂

∂y

= (xy∂2

∂y∂x+ x

∂

∂x)− (xy

∂2

∂x∂y+ y

∂

∂x)

= x∂

∂x− y

∂

∂y

5. Find the commutator of the following two operators.

E : ψ(x, t) 7→ i~∂

∂tψ(x, t)

t : ψ(x, t) 7→ tψ(x, t)

Solution:

[E, t]ψ(x, t) = (i~∂

∂t◦ t− t ◦ i~ ∂

∂t)(ψ(x, t))

= (i~∂

∂t◦ t)(ψ(x, t))− (t ◦ i~ ∂

∂t)(ψ(x, t))

= i~∂

∂t(tψ(x, t))− ti~

∂(ψ(x, t))

∂t

= i~t∂(ψ(x, t))

∂t+ i~(ψ(x, t))

∂t

∂t− ti~

∂(ψ(x, t))

∂t= i~(ψ(x, t))

6. Find the commutator of the following two operators.

H : ψ(x, t) 7→ − ~2

2m

∂2

∂x2ψ(x, t)

x : ψ(x, t) 7→ xψ(x, t)

Solution:

[H, x] = H ◦ x− x ◦ H

(H ◦ x)(ψ(x, t)) = − ~2

2m

∂2

∂x2(x(ψ))

= − ~2

2m

∂2

∂x2(x(ψ))

= − ~2

2m

∂

∂x(∂

∂x(xψ))

= − ~2

2m

∂

∂x(x∂ψ

∂x+ ψ)

= − ~2

2m(∂

∂x(x∂ψ

∂x) +

∂ψ

∂x)

= − ~2

2m(x∂2ψ

∂x2+ 2

∂ψ

∂x)

H ◦ x = −~2x

2m

∂2

∂x2− ~2

m

∂

∂x

(x ◦ H)(ψ(x, t)) = (x ◦ − ~2

2m

∂2

∂x2)(ψ)

= −~2x

2m

∂2ψ

∂x2

x ◦ H = −~2x

2m

∂2

∂x2

[H, x] = − ~2

2mx∂2

∂x2− ~2

m

∂

∂x+

~2x

2m

∂2

∂x2

= −~2

m

∂

∂x

7. Prove the following identity for three operators A,B, C:

[A ◦B,C] = A ◦ [B,C] + [A,C] ◦B.

Solution:

A ◦ [B,C] + [A,C] ◦B = A ◦ (B ◦ C − C ◦B) + (A ◦ C − C ◦ A) ◦B= A ◦B ◦ C − A ◦ C ◦B + A ◦ C ◦B − C ◦ A ◦B= (A ◦B) ◦ C − C ◦ (A ◦B)

= [A ◦B,C]

8. Find the commutator [D2, ex].

Solution: We can find the commutator directly

[D2, ex] = D2 ◦ ex − ex ◦D2

or by using the above identity we get,

[D2, ex] = D ◦ [D, ex] + [D, ex] ◦D.

Try the direct method at home. In class we will use the identity to findthe commutator.

We first find the commutator [D, ex].

(D ◦ ex)(f) =d

dx(exf)

= ex df

dx+ f

dex

dx

= ex df

dx+ fex

= (exD + ex)(f)

(ex ◦D)(f) = (exD)(f)

[D, ex] = (exD + ex)− (exD)

= ex

[D2, ex] = D ◦ [D, ex] + [D, ex] ◦D= D ◦ ex + ex ◦D= (exD + ex) + exD

= ex(2D + 1)

Summary

1. Operators are commonly used to perform a specific mathematical opera-tion on another function.

2. An operator O is a linear operator if it satisfies the following two condi-tions:

(i) O(f + g) = O(f) +O(g).

(ii) O(λf) = λO(f), where λ is a scalar.

3. Let OA and OB be two operators. The commutator of OA and OB is theoperator defined as

[OA,OB] = OA ◦ OB −OB ◦ OA.

10 Fourier Series

10.1 Introduction

When the French mathematician Joseph Fourier (1768-1830) was trying tostudy the flow of heat in a metal plate, he had the idea of expressing theheat source as an infinite series of sine and cosine functions. Although theoriginal motivation was to solve the heat equation, it later became obviousthat the same techniques could be applied to a wide array of mathematicaland physical problems.

In this course, we will learn how to find Fourier series to represent periodicfunctions as an infinite series of sine and cosine terms.

A function f(x) is said to be periodic with period T , if

f(x+ T ) = f(x), for all x.

The period of the function f(t) is the interval between two successive repe-titions.

10.2 Definition of a Fourier Series

Let f be a bounded function defined on the [−π, π] with at most a finitenumber of maxima and minima and at most a finite number of discontinuitiesin the interval. Then the Fourier series of f is the series

f(x) =1

2a0 + a1 cosx+ a2 cos 2x+ a3 cos 3x+ ...

+ b1 sin x+ b2 sin 2x+ b3 sin 3x+ ...

where the coefficients an and bn are given by the formulae

a0 =1

π

∫ π

−π

f(x)dx

an =1

π

∫ π

−π

f(x)cos(nx)dx

bn =1

π

∫ π

−π

f(x)sin(nx)dx

10.3 Why the coefficients are found as they are.

We want to derive formulas for a0, an and bn. To do this we take advantageof some properties of sinusoidal signals. We use the following orthogonalityconditions:

Orthogonality conditions

(i) The average value of cos(nx) and sin(nx) over a period is zero.∫ π

−π

cos(nx)dx = 0∫ π

−π

sin(nx)dx = 0

(ii) The average value of sin(mx)cos(nx) over a period is zero.∫ π

−π

sin(mx)cos(nx)dx = 0

(iii) The average value of sin(mx)sin(nx) over a period,∫ π

−π

sin(mx)sin(nx)dx =

{π if m = n 6= 00 otherwise

(iv) The average value of cos(mx)cos(nx) over a period,∫ π

−π

cos(mx)cos(nx)dx =

2π if m = n = 0π if m = n 6= 00 if m 6= n

Remark. The following trigonometric identities are useful to prove theabove orthogonality conditions:

cosA cosB =1

2[cos(A−B) + cos(A+B)]

sinA sinB =1

2[cos(A−B)− cos(A+B)]

sinA cosB =1

2[sin(A−B) + sin(A+B)]

Finding an

We assume that we can represent the function by

f(x) =1

2a0 + a1 cosx+ a2 cos 2x+ a3 cos 3x+ ...

+ b1 sin x+ b2 sin 2x+ b3 sin 3x+ ... (1)

Multiply (1) by cos(nx), n ≥ 1 and integrate from −π to π and assume it ispermissible to integrate the series term by term.∫ π

−π

f(x) cos(nx)dx =1

2a0

∫ π

−π

cos(nx) + a1

∫ π

−π

cosx cos(nx)dx+ a2

∫ π

−π

cos 2x cos(nx)dx+ ...

+ b1

∫ π

−π

sin x cos(nx)dx+ b2

∫ π

−π

sin 2x cos(nx)dx+ ...

= anπ, because of the above orthogonality conditions

an =1

π

∫ π

−π

f(x) cos(nx)dx

Similarly if we multiply (1) by sin(nx), n ≥ 1 and integrate from −π toπ, we can find the formula for the coefficient bn. To find a0, simply integrate(1) from −π to π.

10.4 Fourier Series

Definition. Let f(x) be a 2π-periodic function which is integrable on [−π, π].Set

a0 =1

π

∫ π

−π

f(x)dx

an =1

π

∫ π

−π

f(x)cos(nx)dx

bn =1

π

∫ π

−π

f(x)sin(nx)dx

then the trigonometric series

1

2a0 +

∞∑n=1

(ancos(nx) + bnsin(nx))

is called the Fourier series associated to the function f(x).

Remark. Notice that we are not saying f(x) is equal to its Fourier Series.Later we will discuss conditions under which that is actually true.

Example. Find the Fourier coefficients and the Fourier series of the square-wave function f defined by

f(x) =

{0 if −π ≤ x < 01 if 0 ≤ x < π

and f(x+ 2π) = f(x)

Solution: So f is periodic with period 2π and its graph is:

Using the formulas for the Fourier coefficients we have

a0 =1

π

∫ π

−π

f(x)dx

=1

π(

∫ 0

−π

0dx+1

π

∫ π

0

1dx)

=1

π(π)

= 1

an =1

π

∫ π

−π

f(x) cos(nx)dx

=1

π(

∫ 0

−π

0dx+1

π

∫ π

0

cos(nx)dx)

=1

π(0 +

[sin(nx)

n

]π

0)

=1

nπ(sinnπ − sin 0)

= 0

bn =1

π

∫ π

−π

f(x) sin(nx)dx

=1

π(

∫ 0

−π

0dx+1

π

∫ π

0

sin(nx)dx)

=1

π(−

[cos(nx)

n

]π

0)

= − 1

nπ(cosnπ − cos 0)

= − 1

nπ((−1)n − 1), since cosnπ = (−1)n.

=

{0, if n is even

2

nπ, if n is odd

The Fourier Series of f is therefore

f(x) =1

2a0 + a1 cosx+ a2 cos 2x+ a3 cos 3x+ ...

+ b1 sin x+ b2 sin 2x+ b3 sin 3x+ ...

=1

2+ 0 + 0 + 0 + ...

+2

πsin x+ 0 sin 2x+

2

3πsin 3x+ 0 sin 4x+

2

5πsin 5x+ ...

=1

2+

2

πsin x+

2

3πsin 3x+

2

5πsin 5x+ ...

Remark. In the above example we have found the Fourier Series of thesquare-wave function, but we don’t know yet whether this function is equalto its Fourier series. If we plot

1

2+

2

πsin x

1

2+

2

πsin x+

2

3πsin 3x

1

2+

2

πsin x+

2

3πsin 3x+

2

5πsin 5x+

1

2+

2

πsin x+

2

3πsin 3x+

2

5πsin 5x+

2

7πsin 7x+

1

2+

2

πsin x+

2

3πsin 3x+

2

5πsin 5x+

2

7πsin 7x+

2

9πsin 9x+

2

11πsin 11x+

2

13πsin 13x,

we see that as we take more terms, we get a better approximation to thesquare-wave function. The following theorem tells us that for almost allpoints (except at the discontinuities), the Fourier series equals the function.

Theorem (Fourier Convergence Theorem) If f is a periodic func-tion with period 2π and f and f ′ are piecewise continuous on [−π, π], thenthe Fourier series is convergent. The sum of the Fourier series is equal tof(x) at all numbers x where f is continuous. At the numbers x where f isdiscontinuous, the sum of the Fourier series is the average value. i.e.

1

2[f(x+) + f(x−)].

Remark. If we apply this result to the example above, the Fourier Seriesis equal to the function at all points except −π, 0, π. At the discontinuity 0,observe that

f(0+) = 1 and f(0−) = 0

The average value is 1/2.Therefore the series equals to 1/2 at the discontinuities.

10.5 Odd and even functions

Definition. A function is said to be even if

f(−x) = f(x) for all real numbers x.

A function is said to be odd if

f(−x) = −f(x) for all real numbers x.

Example. cosx, x2, |x| are examples of even functions. sinx, x, x3 areexamples of odd functions. The product of two even functions is even, theproduct of two odd functions is also even. The product of an even and oddfunction is odd.

Remark. If f is an odd function then∫ π

−π

f(x)dx = 0,

while if f is an even function then∫ π

−π

f(x)dx = 2

∫ π

0

f(x)dx.

(i) If f(x) is odd, then

• f(x) cos(nx) is odd hence an = 0 and

• f(x) sin(nx) is even hence bn = 2π

∫ π

0f(x) sin(nx)dx

(ii) If f(x) is even, then

• f(x) cos(nx) is even hence an = 2π

∫ π

0f(x) cos(nx)dx and

• f(x) sin(nx) is even hence bn = 0

Example. 1. Let f be a periodic function of period 2π such that

f(x) = x for − π ≤ x < π

Find the Fourier series associated to f .

Solution: So f is periodic with period 2π and its graph is:

We first check if f is even or odd.

f(−x) = −x = −f(x), so f(x) is odd.

Therefore,

an = 0

bn =2

π

∫ π

0

f(x)sin(nx)dx

Using the formulas for the Fourier coefficients we have

bn =2

π

∫ π

0

x sin(nx)dx

=2

π([− x

cosnx

n

]π

0−

∫ π

0

(−cosnx

n)dx)

=2

π(1

n[−π cosnπ] + [

sinnx

n2]π0 )

= − 2

ncosnπ

=

{−2/n if n is even

2/n if n is odd

The Fourier Series of f is therefore

f(x) = b1 sin x+ b2 sin 2x+ b3 sin 3x+ ...

= 2 sinx− 2

2sin 2x+

2

3sin 3x− 2

4sin 4x+

2

5sin 5x+ ...

2. Let f be a periodic function of period 2π such that

f(x) = π2 − x2 for − π ≤ x < π

Solution: So f is periodic with period 2π and its graph is:

We first check if f is even or odd.

f(−x) = π2 − (−x)2 = π2 − x2 = f(x), so f(x) is even.

Since f is even,

bn = 0

an =2

π

∫ π

0

f(x) cos(nx)dx

Using the formulas for the Fourier coefficients we have

an =2

π

∫ π

0

f(x) cos(nx)dx

=2

π

∫ π

0

(π2 − x2) cos(nx)dx

=2

π([(π2 − x2)

sinnx

n

]π

0−

∫ π

0

−2xsinnx

ndx)

=2

π([(π2 − π2)

sinnπ

n− (π2 − 0)

sin 0

n] +

∫ π

0

2xsinnx

ndx)

=2

π

2

n

∫ π

0

x sin(nx)dx

=2

π

2

n([− x

cosnx

n

]π

0−

∫ π

0

(−cosnx

n)dx)

=2

π

2

n

1

n([−π cosnπ] + [

sinnx

n2]π0 )

= − 4

n2cosnπ

=

{−4/n2 if n is even

4/n2 if n is odd

It remains to calculate a0.

a0 =2

π

∫ π

0

(π2 − x2)dx

=2

π[π2x− x3

3]π0

=4π2

3

The Fourier Series of f is therefore

f(x) =1

2a0 + a1 cosx+ a2 cos 2x+ a3 cos 3x+ ...

b1 sin x+ b2 sin 2x+ b3 sin 3x+ ...

=2π2

3+ 4(cosx− 1

4cos 2x+

1

9cos 3x− 1

16cos 4x+

1

25cos 5x+ ....)

10.6 Functions with period 2L

If a function has period other than 2π, we can find its Fourier series by makinga change of variable. Suppose f(x) has period 2L, that is f(x+ 2L) = f(x)for all x. If we let

t =πx

Land g(t) = f(x) = f(

Lt

π)

then, g has period 2π and x = ±L corresponds to t = ±π. We know theFourier series of g(t):

1

2a0 +

∞∑n=1

(an cosnt+ bn sinnt).

where

a0 =1

π

∫ π

−π

f(t)dt

an =1

π

∫ π

−π

f(t) cos(nt)dt

bn =1

π

∫ π

−π

f(t) sin(nt)dt

If we now use the Substitution Rule with x = Ltπ

, then t = πxL

, dt = ( πL)dx,

we have the following:

Definition. If f(x) is a piecewise continuous function on [−L,L], its FourierSeries is

1

2a0 +

∞∑n=1

an cos(nπx

L) + bn sin(

nπx

L),

where

a0 =1

L

∫ L

−L

f(x)dx

an =1

L

∫ L

−L

f(x) cos(nπx

L)dx

bn =1

L

∫ L

−L

f(x) sin(nπx

L)dx

Example. Find the Fourier series of the triangular wave function definedby f(x) = |x| − a for −a ≤ x ≤ a and f(x+ 2) = f(x) for all x.

Solution: So f is periodic with period 2a and its graph is:

We first check if f is even or odd.

f(−x) = | − x| − a = |x| − a = f(x), so f(x) is even.

Therefore,

bn = 0

an =2

a

∫ a

0

f(x) cos(nπx

a)dx

Using the formulas for the Fourier coefficients we have

an =2

a

∫ a

0

(|x| − a) cos(nπx

a)dx

=2

a

∫ a

0

(x− a) cos(nπx

a)dx

=2

a([(x− a)

sin(nπxa

)nπa

]a

0−

∫ a

0

asin(nπx

a)

nπa

dx)

=2

a(a2

n2π2

[cos(

nπx

a)]a

0)

=2a

n2π2(cosnπ − 1)

=

{0 if n is even

− 4a

n2π2if n is odd

It remains to determine a0.

a0 =2

a

∫ a

0

|x| − adx

=2

a

∫ a

0

x− adx

=2

a[x2

2− ax]a0

= −a

The Fourier Series of f is therefore

f(x) = a0 + a1 cosx+ a2 cos 2x+ a3 cos 3x+ ...

= −a2− 4a

π2(cos(

πx

a) +

1

9cos(

3πx

a) +

1

25cos(

5πx

a) + ...)

10.7 Parseval’s Result

The folowing result can be used to find the sum of certain series from calcu-lated Fourier series.

Theorem Let f be a bounded function defined in the interval [−L,L] withat most a finite number of maxima and minima and at most a finite numberof discontinuities in the interval [−L,L], and associated to the Fourier series

1

2a0 +

∞∑n=1

(an cosnx+ bn sinnx).

Then1

L

∫ L

−L

(f(x))2dx =1

2a2

0 +∞∑

n=1

(a2n + b2n).

Example. Let f be a periodic function defined by f(x) = x for all −π ≤x ≤ π. Use Parseval’s result to show that

π2

6=

∞∑n=1

1

n2.

Solution: We have shown that

f(x) = 2 sin x− 2

2sin 2x+

2

3sin 3x− 2

4sin 4x+

2

5sin 5x+ ...

Using Parseval’s Identity

1

π

∫ π

−π

(f(x))2dx =1

2a2

0 +∞∑

n=1

(a2n + b2n)

1

π

[x3

3

]π

−π= 4(1 +

1

4+

1

9+ ...)

2π2

3= 4(1 +

1

4+

1

9+ ...)

π2

6= 1 +

1

4+

1

9+ ...

π2

6=

∞∑n=1

1

n2

Summary

1. The Fourier Series of a periodic function f(x) defined on the interval[−π, π] is:

1

2a0 + a1 cosx+ a2 cos 2x+ a3 cos 3x+ ...

+ b1 sin x+ b2 sin 2x+ b3 sin 3x+ ...

where the coefficients an and bn are given by the formulae

a0 =1

π

∫ π

−π

f(x)dx

an =1

π

∫ π

−π

f(x)cos(nx)dx

bn =1

π

∫ π

−π

f(x)sin(nx)dx

2. Even and Odd Functions for functions with period 2π.

If f(x) is odd, (i.e. f(−x) = −f(x)) , then

• an = 0 and

• bn = 2π

∫ π

0f(x) sin(nx)dx

If f(x) is even, (i.e. f(−x) = f(x)), then

• an = 2π

∫ π

0f(x) cos(nx)dx and

• bn = 0

3. The Fourier Series of a periodic function f(x) defined on the interval[−L,L] is:

1

2a0 +

∞∑n=1

an cos(nπx

L) + bn sin(

nπx

L),

where

a0 =1

L

∫ L

−L

f(x)dx

an =1

L

∫ L

−L

f(x) cos(nπx

L)dx

bn =1

L

∫ L

−L

f(x) sin(nπx

L)dx

4. Even and Odd Functions for functions with period 2L.

If f(x) is odd, (i.e. f(−x) = −f(x)) , then

• an = 0 and

• bn = 2π

∫ L

0f(x) sin(nπx

L)dx

If f(x) is even, (i.e. f(−x) = f(x)), then

• an = 2π

∫ L

0f(x) cos(nπx

L)dx and

• bn = 0

5. Parseval’s Result

1

L

∫ L

−L

(f(x))2dx =1

2a2

0 +∞∑

n=1

(a2n + b2n).

References

1. http://www.intmath.com/Fourier-series/Fourier-intro.php

2. http://www.falstad.com/fourier/e-index.html

3. Ian S. Murphy, Advanced Calculus

4. Mary L Boas, Mathematical Methods in the Physical Sciences.