Week 7 Numerical Methodsen1811/15s1/lectures/week07.pdf · Methods: • Bisection • Interpolation...

ENGG1811 © UNSW, CRICOS Provider No: 00098G1 W7 slide 1

Week 7

Numerical Methods

ENGG1811 Computing for Engineers

ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 2

Prologue

Many engineering programs will include a

serious coverage (say a third of a course or

more) on numerical methods.

This lecture covers only a small fraction of the

topic, but delves into some interesting aspects

in detail.

The emphasis this week is on algorithms rather

than coding. However implementation is still

important, and the lab work next week will use

or extend a couple of examples presented here.


Numerical Methods

Numerical Methods are algorithms that provide exact or

approximate solutions to problems that are difficult or

impossible to solve analytically.

They include

• Root finding

• Optimisation (max/min or Solver-type)

• Area of a polygon

• Area under a curve (approximating definite integrals)

• Curve fitting

• Solving systems of linear equations

• Solving differential equations

We will look at some examples of the first four of these.

Curve fitting and solving linear equations are what

Matlab is good at, and ODEs require maths > 1st year.


Root Finding

We used Calc’s Goal Seek tool in the SS part of the

course. How does it work?* (we’ll tell you later)

Goal: find a value of x such that f (x) = 0

• it can also be f (x) = k for some constant k by defining

g (x) = f (x) – k and finding one of its roots

Methods:

• Bisection

• Interpolation (“regula falsi”)

• Brent’s method (composite, not covered)

• Fixed-point iteration (not covered)

* Who cares, you ask? Well, knowing how any tool, system or

technique works, be it software, car suspension, stock markets or a

truss bridge, allows you to understand its strengths and limitations so

you can make more effective use of it professionally.

• Newton-Raphson

• Secant method


Bisection and Interpolation

The simpler methods start with two points x1 and x2 on

either side of the root such that f (x1) < 0 and f (x2) > 0

They pick a new value between x1 and x2 that’s likely to

be much closer to the root

Depending on the sign of the result, replace one or other

bound, continue until the bounds are close enough

Pseudocode: choose x1 and x2 such that f(x1) < 0 and f(x2) > 0 While Abs(x1-x2) > EPSILON pick x3 between x1 and x2 If f(x3) < 0 Then x1 = x3 Else x2 = x3 End If Wend

Bisection: mean of x1 and x2

Interpolation: where the line

intersecting the function at x1

and x2 crosses the x axis

Bisection


x

f (x)

x1

x2

x3

x4

lo hi

x1 x2

x3 x2

x3 x4

Slow but steady

convergence

Interpolation


x

f (x)

x1

x2

lo hi

x1 x2

x3 x2

x4 x2

x3

x4

One end tends to

get stuck while the

other is very close

Interpolation: where

the line intersecting the

function at x1 and x2

crosses the x axis

Modifed Regula Falsi (Interpolation 2)


x

f (x)

x1

x2

x3

x4

For the end that

doesn’t move,

divide the function

value by 2

½ f (x2)

What matters is not

how close x4 is to the

root, but how small

the endpoint

difference is and how

quickly it converges

Interpolation 2 algorithm

• Same structure as bisection

• If a and b are the current estimates, the

interpolated point is given by

• To avoid repeated function

evaluations, keep variables fa and fb as well as a and b:

fa = f(a) : fb = f(b)

While Abs(fa – fb) > EPSILON

interp = (a*fb – b*fa) / (fb – fa)

fx = f(interp)

If sign of fx = sign of fa Then

a = interp : fa = fx : fb = fb/2

Else

b = interp : fb = fx : fa = fa/2


)()(

)()(

afbf

afbbfa

convergence

criteria options

next slide

…

Convergence criteria

We’ve used two ways of deciding what’s “close enough”,

four are common. a and b are the two most recent

estimates:


)(.4

)()(.3

.2

.1

af

bfaf

a

ba

ba Absolute: significant figures vary with magnitude

Relative: won’t work close to zero

OK but sensitive to slope at the root

best overall choice, not sensitive to slope or singularities

• The demo workbook has a GoalSeekers sheet

• These two and the remaining two algorithms coded with a specified iteration stopping point (as a demo)

• Function is defined on the sheet

– we’re solving which has one positive root,

the golden ratio (phi)

• Spin button sets iteration count

• Initial values specified (no evaluation if missing)

• Try the first two examples, these are called first-order solutions to the problem

– estimate range roughly halves (interp is a bit better) on each iteration

– need about log2 10 = 3.3 iterations per decimal place

Evaluation Sheet


012 xx

618034.12

51

• A smarter technique uses a single estimate x1 and

draws a tangent from that point on the curve.

• In most cases, where the tangent crosses the x axis is

much closer to the root

• Tangent is the function derivative f ′ (x)

• Iterate this process until successive estimates are

close enough

Example: calculate the square root of some value b > 0

(this is the way basic calculators do square roots)


Newton-Raphson (NR)

)(

)(1

i

iii

xf

xfxx

i

i

i

iii

x

bx

x

bxxxxxfbxxf

2

1

2;2)(;)(

2

12

Newton-Raphson


x

f (x)

x1

x2

x3

Converges rapidly if

the function is

smooth and the

derivative doesn’t

change sign

22)( 3 xxxf


NR swings and roundabouts

Advantages: converges rapidly under good conditions

Disadvantages:

– need the derivative as well as the function

– near a local min/max the derivative becomes very small and

the algorithm can diverge (below right, near x = ± 0.8165)

– approaching a cusp (left) the slope is ± inf, also diverges

– for high-order polynomials the algorithm can get caught in a

box if a poor starting point is chosen

x1=0, f (x1)=2

x2=1, f (x2)=1

x3=0, ...

Source: Wikipedia


NR Implementation

' Find x such that f(x) = 0, starting with estimate cur Dim fcur As Double, fd As Double ' f(cur) and f'(cur) Dim fprev As Double ' previous fcur for convergence Dim converged As Boolean Dim numIt As Long converged = False While Not converged

fprev = fcur ' save for convergence test fcur = f(cur) : fd = fderiv(cur)

numIt = numIt + 1 If numIt >= ITERATION_LIMIT Or Abs(fd) <= EPSILON Then ' report that NR can’t find a solution Exit Sub End If

cur = cur – fcur / fd converged = Abs(fcur – fprev) <= EPSILON

Wend ' cur is the NR best estimate

• Apparent variation on interpolation but much better

• Starts with two points, need not be either side of

root, draw line (called a secant) through them to

intersect X axis [so far same as interp]

• Gives next x, but secant is always drawn between

the last two estimates

• Iteration formula is

• Looks a bit like NR, in fact as the x values get close,

the secant approaches the tangent, so the secant

method incorporates numeric differentiation too!

• Has similar limitations to NR but no need for explicit

derivative function

Secant method


)()(

))((

1

11

ii

iiiii

xfxf

xxxfxx

Secant


x

f (x)

x1

x3

x4

Like NR, a second-

order algorithm that

converges rapidly

under favourable

conditions

x2


But wait, there’s more

Don’t worry, that’s enough detail for this topic

Goalseekers in specialised software tend to be adaptive,

that is, they will try either

• second-order methods until they show signs of

oscillation or divergence, then switch to first-order, or

• multiple first-order algorithms at each step, choosing

whichever appears to be approaching the root faster

Brent’s method tries secant and quadratic (rather than

linear) interpolation, with a fallback to bisection.

Calc’s Goalseek uses linear interpolation according to the

source code listing at http://opengrok.libreoffice.org/xref/core/sc/source/core/data/documen4.cxx#51

http://opengrok.libreoffice.org/xref/core/sc/source/core/data/documen4.cxx#51


Optimisation

Complete courses could be devoted to this topic, we’ll

just look at one small algorithm that can find a local

max or min value for a range of non-linear functions*

One approach is to find a root of the derivative function

(if available) using one of the root-finding schemes.

* Calc’s Goal Seek can’t do max/min and Calc’s Solver can’t do

non-linear optimisation, so this fills a genuine need.

f ′(x) < 0

f ′(x) > 0

f ′(x) = 0

Of course to use

NR you’ll need

the second

derivative too


Finding the minimum

Consider a function that’s unimodal (slope changes sign

exactly once) in a region, and has a minimum there.

We pick two points that bracket the minimum, and a

third between them, and evaluate the function there:

y4a

y1

y2

y3

a b

c

y4b

Now choose a point

x4 in the larger

region x2|x3

If y4 < y2 as shown,

the min must be in

the region x2|x3

x1 x2 x3

But if y4 > y2 (blue),

the min must be in

the region x1|x4 x4

Minimum

• So in case 1 the region is reduced from a+b to b,

and in case 2 from a+b to a+c. For maximum

efficiency these should be equal. a+c = b ...(1)

• Now we have to decide how x2 was placed. For the

algorithm to work the subintervals on each side

must maintain the original ratio r = b/a. Thus


x3 c x1 x2 x4

a b

01,01

1,01

1,1

:Invert

so,(1)frombut 2case

2

rrr

rra

b

ra

ab

rab

aabcr

c

a

Golden Section Search

r2 – r – 1 = 0 ?

Hang on, we just solved that, r must be equal to , the

Golden Ratio (in fact, interval subdivision like this is one

way to define phi). For this reason the algorithm is

called the Golden Section Search.

It was discovered (remember good algorithms already

exist, they just have to be discovered) by Jack Kiefer in

1953.

The algorithm itself iterates the interval subdivision

principle, choosing one new point each time. An

analogous one to bisection would have to choose 2

points to decide which subinterval to follow.


Golden Section Search algorithm

Note that whichever subdivision we take, the

equivalent of x2 is already placed, we just have to

worry about x4.

However we need to know whether the larger portion is

to the left or right, or equivalently, whether x2 is right

or left of the midpoint, respectively

Larger right: x4 = x1 + b = x1 + (x3 – x2)

Larger left: x4 = x3 – b = x3 – (x2 – x1) = x1 + (x3 – x2)

OMG they’re the same!! That’s a sign of a truly elegant

algorithm (IMHO).

Cases 1 and 2 must be handled separately to establish

the new subintervals, as different end-points move



GSS Solution

' Find local minimum of f(x), lying between x=lo and x=hi ' the subinterval boundary is mid (x2 in the analysis) ' the new x value is probe (x4 in the analysis) ' fLo, fHi, fMid and fProbe are corresponding function values ' PHI is the Golden Ratio mid = lo + (hi – lo) * (1 + 1/PHI) ' the only use of PHI ! While Abs(fHi – fLo) <= EPSILON

probe = lo + (hi – mid) ' x4 fProbe = f(probe) ' only need one func eval per iteration If probe > mid Then ' right is larger

If fProbe < fMid Then ' Case 1

lo = mid : fLo = fMid mid = probe : fMid = fProbe Else ' Case 2 hi = probe : fHi = fProbe ' mid is OK End If

Else ' left is larger: symmetric If statement, with lo hi End If

Wend ' mid is the GSS best estimate

Practicalities

• This algorithm is first-order, about 1/log10 = 4.8

iterations per significant digit, or around 60 to get a highly precise answer

• To turn this into a practical tool we need two things

1. A means of supplying the function and initial estimates, and

2. A funky name for it.

• #2 is easy, it’s MiniMe (apologies to Mike Myers)

• The Optimisers sheet in the demo workbook has a similar test setup to the Goalseekers sheet.


Area of a Polygon

• Given the coordinates in 2D space of the N vertices of a polygon, a neat formula due to Gauss calculates the area


x0 (1,2)

x1 (2,3)

x2 (4,4)

x3 (6,3)

xN = x0

x4 (4,3)

x5 (3,1.5)

1

0

112

1N

i

iiii yxyxA

i xi yi term

0 1 2 3 – 4 = –1

1 2 3 8 – 12 = –4

2 4 4 12 – 24 = –12

3 6 3 18 – 12 = 6

4 4 3 6 – 9 = –3

5 3 1.5 6 – 1.5 = 4.5

6 1 2 sum = –9.5 The cross product form suggests the name Shoelace Formula

How does it work?

Note: this is the principle on which the algorithm works, though each term doesn’t correspond directly to a trapezium


1. Project vertical lines from the vertices to the X axis. These form trapeziums.

2. Add the areas of the trapeziums in clockwise order, noting that as you process the lower vertices, the areas are negative.

3. Eventually the areas under the polygon are subtracted, leaving just the polygon area

Benefits and limitations

• Can be fairly easily adapted to surveying data (distance/bearing for each leg of a closed traverse)

• Can be extended to calculate the centroid (centre of mass if the polygon was a plate of uniform density): search for ‘centroid’ in Paul Bourke’s

http://paulbourke.net/geometry/polygonmesh/

• Need coordinates, so it doesn’t directly give the answer to a question like: what’s the area of a regular pentagon with each side 100 metres?

• Works for polygons with holes, provided holes are traversed in the opposite order to the perimeter


• Does not work for self-intersecting polygons like these:



Numerical Integration

• Definite integrals represent the area under the

curve described by f (x)

• Sometimes the

function integral doesn’t have a closed form

• Sometimes the function is only sampled, or is

empirically determined

• In both cases, the area can be approximated using

suitable algorithms


b

a

dxxfA )(

f (x)

a b

A square units x


Trapezoidal Rule

The simplest of the numerical integration algorithms is

the Trapezoidal Rule, which uses thin vertical slices

with straight lines at the top approximating the function

• Slices are adjacent trapeziums, area easily calculated

• If the panel width is small enough, the calculated area

is asymptotically correct, but...

– area is underestimated if f is convex as viewed from below

– area is overestimated if f is concave

a

f (x)

b

Integral of f (x)

from a to b

is approximated by

the sum of the areas

inside the red trapeziums

Visualisation

A nice animation is available at

http://math.fullerton.edu/mathews/a2001/Animations/Quadrature/Trapezoidal/Trapezoidalaa.html

(numerical integration is also known as quadrature)

The integral of the sample function +1

doesn’t have a closed form*, and the definite integral

from x=0 to x=2 can only be approximated using these

kinds of methods.

Its value is about 2.016

* Matlab’s symbolic integration toolbox can’t integrate it, anyway


3/28sin)( xexf x



Derivation of the TR

The area of a trapezium with base

and heights h1 and h2 is just

The heights are function values, and

the panels are all of equal width , hence

So the outside points are used once and all the

interior points are used twice.


h1 h2 )(

221 hh

)()(2...)2(2)(2(2

)()(2

...)2()(2

)()(2

)(

bfbfafafaf

bfbfafafafafdxxf

b

a

½(h1+ h2)


Trapezoidal Rule, theory

Assuming there are n equally spaced panels, the formula

in terms of successive x values reduces to

*/)( ianabiaxwhere i

)()(2)(2)(2)(2

)( 1210 nn

b

a

xfxfxfxfxfdxxf

Twice the middle values

Panel width

n

ab

End-points counted

once only


Trapezoidal Rule, pseudocode

We can easily convert this to pseudocode:

set sum to f (a) + f (b) as these are the odd ones out

set delta to the panel width (b-a)/n

For p = 1 To n-1

add 2*f(a + delta*p) to sum

Next p

area = sum * delta/2


Trapezoidal Rule, implementation

' Estimates the area under the curve of f(x) from x=a ' to x=b using the Trapezoidal Rule with n panels. ' Uses FuncEvalTrap(x) to obtain function values from ' the active sheet.

Function TrapAreaFunc(a As Double, b As Double, _

numPanels As Long) As Double

' Estimates the area under the curve of f(x) ' expressed as a series of sampled values on the ' active sheet starting with the given row and column ' Uses the Trapezoidal Rule.

Function TrapAreaSampled( _ rowStart As Long, colStart As Long, _ numSamples As Long) As Double

Each of these has a wrapper subprogram that’s linked to a

button on the sheet (so the functions are only called when the

data is stable)


TR1 Wrapper

' First wrapper (second is similar but more complex validity check) Sub CalculateTRbyFormula() Dim a As Double, b As Double Dim n As Long ' fetch problem parameters a = ActiveSheet.Range("TR1a") b = ActiveSheet.Range("TR1b") n = ActiveSheet.Range("TR1n") ' clear result cell during calculations ActiveSheet.Range("TR1Area") = "" ' check limit validity If a >= b Then ActiveSheet.Range("TR1valid") = False Exit Sub End If ActiveSheet.Range("TR1valid") = True ActiveSheet.Range("TR1Area") = TrapAreaFunc(a, b, n) End Sub

Adding buttons for action

Add buttons to the sheet to run the integrators

– View – Toolbars – Form controls

– Design Mode on (all icons visible)

– Pick the button icon, draw on sheet

– Right-click, select Control...

– General tab, relabel appropriately

– Events tab, choose Execute Action, press


Adding a button, continued

• Assign macro by navigating to the Standard module in the workbook, select CalculateTRbyFormula (in this case)

• When it’s all OK, turn Design Mode off again to activate the button


Putting it all together

Use the sample function, but also try quarter, half and full cycles of

For the sampled version, first use evenly spaced calculated values, then irregularly spaced ones.

Finally use the tool to solve the ellipse perimeter and river flow problems overleaf


Integration problems

1. The orbit of the demoted former planet Pluto is elliptical, with axes

a = 5.9065e+9 km and

b = 5.7208e+9 km.

The perimeter of an ellipse is:

determine Pluto’s orbital distance.

2. Estimate the cross-sectional area of a river whose depth

has been measured at 12m intervals:


a

b

a

bakdkaP

222/

0

22 wheresin14

1 11 16 20 22 20 18 12 19 21 20 19 9 10 4 (m)

Other integrators

• Simpson’s Rule – uses quadratic segments rather than straight lines

• Gauss quadrature

• Romberg integration

• etc etc (you don’t need to know this)


Summary

• There are often multiple ways to approximate numeric solutions to tough analytical problems

• Root finding can be achieved with

– simple, reliable but slow first-order algs

– clever but sometimes unreliable second-order algs

• Minimisation can be found through

– direct slicing means (bisection, golden section),

– by finding roots of the derivative

– by clever means that follow the gradient better (we didn’t get to that)

• Polygon area can be calculated using Gauss’ (or Shoelace) formula

• Numeric integration divides the area into geometric segments and uses their areas to approximate the solution


Week 7 Numerical Methodsen1811/15s1/lectures/week07.pdf · Methods: • Bisection • Interpolation...

Documents

Transcript of Week 7 Numerical Methodsen1811/15s1/lectures/week07.pdf · Methods: • Bisection • Interpolation...