Numerical Methods
-
Upload
esug -
Category
Technology
-
view
1.177 -
download
4
description
Transcript of Numerical Methods
![Page 1: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/1.jpg)
April 10, 2023
Numerical MethodsNumerical Methods
ESUG, Gent 1999
Didier BessetDidier BessetIndependent ConsultantIndependent Consultant
![Page 2: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/2.jpg)
©DB Consulting, 1999, all rights reserved
Road MapRoad Map
Dealing with numerical dataFramework for iterative processesNewton’s zero findingEigenvalues and eigenvectorsCluster analysisConclusion
![Page 3: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/3.jpg)
©DB Consulting, 1999, all rights reserved
Road MapRoad Map
Dealing with numerical dataFramework for iterative processesNewton’s zero findingEigenvalues and eigenvectorsCluster analysisConclusion
![Page 4: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/4.jpg)
©DB Consulting, 1999, all rights reserved
Dealing With Numerical DataDealing With Numerical Data
Besides integers, Smalltalk supports two numerical types– Fractions– Floating point numbers
![Page 5: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/5.jpg)
©DB Consulting, 1999, all rights reserved
Using FractionsUsing Fractions
Results of products and sums are exact.Cannot be used with transcendental functions.Computation is slow.Computation may exceed computer’s memory.Convenient way to represent currency values
when combined with rounding.
![Page 6: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/6.jpg)
©DB Consulting, 1999, all rights reserved
Using Floating Point NumbersUsing Floating Point Numbers
Computation is fast.Results have limited precision
(usually 54 bits, ~16 decimal digits).Products preserve relative precision.Sums may keep only wrong digits.
![Page 7: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/7.jpg)
©DB Consulting, 1999, all rights reserved
Using Floating Point NumbersUsing Floating Point Numbers
Sums may keep only wrong digits.Example:– Evaluate
2.718 281 828 459 05 - 2.718 281 828 459 04 – … = 9.76996261670138 x 10-15 – Should have been 10-14 !
![Page 8: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/8.jpg)
©DB Consulting, 1999, all rights reserved
Using Floating Point NumbersUsing Floating Point Numbers
Beware of meaningless precision!– In 1424, Jamshid Masud al-Kashi published
= 3.141 592 653 589 793 25…– …but noted that the error in computing the
perimeter of a circle with a radius 600’000 times that of earth would be less than the thickness of a horse’s hair.
![Page 9: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/9.jpg)
©DB Consulting, 1999, all rights reserved
Using Floating Point NumbersUsing Floating Point Numbers
Donald E. Knuth :– Floating point arithmetic is by
nature inexact, and it is not difficult to misuse it so that the computed answers consist almost entirely of “noise”. One of the principal problems of numerical analysis is to determine how accurate the results of certain numerical methods will be.
![Page 10: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/10.jpg)
©DB Consulting, 1999, all rights reserved
Using Floating Point NumbersUsing Floating Point Numbers
Never use equality between two floating point numbers.
Use a special method to compare them.
![Page 11: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/11.jpg)
©DB Consulting, 1999, all rights reserved
Using Floating Point NumbersUsing Floating Point Numbers
Proposed extensions to class Number
relativelyEqualsTo: aNumber upTo: aSmallNumber| norm |norm := self abs max: aNumber abs.^norm <= aSmallNumber
or: [ (self - aNumber) abs < ( aSmallNumber * norm)]
equalsTo: aNumber
^self relativelyEqualsTo: aNumber upTo: DhbFloatingPointMachine default defaultNumericalPrecision
relativelyEqualsTo: aNumber upTo: aSmallNumber| norm |norm := self abs max: aNumber abs.^norm <= aSmallNumber
or: [ (self - aNumber) abs < ( aSmallNumber * norm)]
equalsTo: aNumber
^self relativelyEqualsTo: aNumber upTo: DhbFloatingPointMachine default defaultNumericalPrecision
![Page 12: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/12.jpg)
©DB Consulting, 1999, all rights reserved
Road MapRoad Map
Dealing with numerical dataFramework for iterative processesNewton’s zero findingEigenvalues and eigenvectorsCluster analysisConclusion
![Page 13: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/13.jpg)
©DB Consulting, 1999, all rights reserved
Iterative ProcessesIterative Processes
Find a result using successive approximations
Easy to implement as a frameworkWide field of application
![Page 14: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/14.jpg)
©DB Consulting, 1999, all rights reserved
Successive ApproximationsSuccessive Approximations
Begin
Compute or choose initial object
Computenext object
Object sufficient?
Yes
End
No
![Page 15: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/15.jpg)
©DB Consulting, 1999, all rights reserved
Iterative ProcessesIterative Processes
NewtonZeroFinder
TrapezeIntegrator
SimpsonIntegrator
RombergIntegrator
EigenValues
MaximumLikelihoodFit
LeastSquareFit
GeneticAlgorithm
SimplexAlgorithm
HillClimbing
IterativeProcess
FunctionalIterator
ClusterAnalyzer
InfiniteSeries
ContinuedFraction
![Page 16: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/16.jpg)
©DB Consulting, 1999, all rights reserved
ImplementationImplementationBegin
iterations := 0
Yes
Perform clean-up
Computenext object
iterations := iterations + 1
No
iterations < maximumIterations
Compute or choose initial object
EndNo
Yes
precision < desiredPrecision
![Page 17: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/17.jpg)
©DB Consulting, 1999, all rights reserved
evaluate
finalizeIteration
hasConverged
evaluateIteration
initializeIteration
MethodsMethodsCompute or choose
initial object
Yes
No
Computenext object
iterations := 0
iterations := iterations + 1
iterations < maximumIterations
Perform clean-up
Begin
EndNo
Yes
precision < desiredPrecision
![Page 18: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/18.jpg)
©DB Consulting, 1999, all rights reserved
Simple example of useSimple example of use
| iterativeProcess result |iterativeProcess := <a subclass of
IterativeProcess> new.result := iterativeProcess evaluate.iterativeProcess hasConverged
ifFalse:[ <special case processing>].
| iterativeProcess result |iterativeProcess := <a subclass of
IterativeProcess> new.result := iterativeProcess evaluate.iterativeProcess hasConverged
ifFalse:[ <special case processing>].
![Page 19: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/19.jpg)
©DB Consulting, 1999, all rights reserved
Class DiagramClass Diagram
IterativeProcess
evaluateinitializeIterationhasConvergedevaluateIterationfinalizeIterationprecisionOf:relativeTo:desiredPrecision (g,s)maximumIterations (g,s)precision (g)iterations (g)result (g)
![Page 20: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/20.jpg)
©DB Consulting, 1999, all rights reserved
precisionOf: aNumber1 relativeTo: aNumber2^aNumber2 > DhbFloatingPointMachine default defaultNumericalPrecision
ifTrue: [ aNumber1 / aNumber2]
ifFalse:[ aNumber1]
precisionOf: aNumber1 relativeTo: aNumber2^aNumber2 > DhbFloatingPointMachine default defaultNumericalPrecision
ifTrue: [ aNumber1 / aNumber2]
ifFalse:[ aNumber1]
Computation of precision should be made relative to the result
General method:
Case of Numerical ResultCase of Numerical Result
ResultAbsolute precision
![Page 21: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/21.jpg)
©DB Consulting, 1999, all rights reserved
Example of useExample of use
| iterativeProcess result |iterativeProcess := <a subclass of
FunctionalIterator> function:( DhbPolynomial new: #(1 2 3).
result := iterativeProcess evaluate.iterativeProcess hasConverged
ifFalse:[ <special case processing>].
| iterativeProcess result |iterativeProcess := <a subclass of
FunctionalIterator> function:( DhbPolynomial new: #(1 2 3).
result := iterativeProcess evaluate.iterativeProcess hasConverged
ifFalse:[ <special case processing>].
![Page 22: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/22.jpg)
©DB Consulting, 1999, all rights reserved
Class DiagramClass Diagram
initializeIterationcomputeInitialValues relativePrecisionsetFunction:functionBlock (s)
FunctionalIterator
IterativeProcess
value:
AbstractFunction
![Page 23: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/23.jpg)
©DB Consulting, 1999, all rights reserved
Road MapRoad Map
Dealing with numerical dataFramework for iterative processesNewton’s zero findingEigenvalues and eigenvectorsCluster analysisConclusion
![Page 24: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/24.jpg)
©DB Consulting, 1999, all rights reserved
Newton’s Zero FindingNewton’s Zero Finding
It finds a value x such that f(x) = 0.More generally, it can be used to find a
value x such that f(x) = c.
![Page 25: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/25.jpg)
©DB Consulting, 1999, all rights reserved
Newton’s Zero FindingNewton’s Zero Finding
Use successive approximation formulaxn+1 = xn – f(xn)/f’(xn)
Must supply an initial value– overload computeInitialValues
Should protect against f’(xn) = 0
![Page 26: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/26.jpg)
©DB Consulting, 1999, all rights reserved
Newton’s Zero FindingNewton’s Zero Finding
x1
x2
x3
f(x)
y = f(x1) + f’(x1)·(x- x1)
![Page 27: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/27.jpg)
©DB Consulting, 1999, all rights reserved
Example of useExample of use
| zeroFinder result |zeroFinder:= DhbNewtonZeroFinder
function: [ :x | x errorFunction - 0.9]
derivative: [ :x | x normal].zeroFinder initialValue: 1.result := zeroFinder evaluate.zeroFinder hasConverged
ifFalse:[ <special case processing>].
| zeroFinder result |zeroFinder:= DhbNewtonZeroFinder
function: [ :x | x errorFunction - 0.9]
derivative: [ :x | x normal].zeroFinder initialValue: 1.result := zeroFinder evaluate.zeroFinder hasConverged
ifFalse:[ <special case processing>].
![Page 28: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/28.jpg)
©DB Consulting, 1999, all rights reserved
Class DiagramClass Diagram
evaluateIterationcomputeInitialValues (o)defaultDerivativeBlock initialValue:setFunction: (o)
NewtonZeroFinder
IterativeProcess
FunctionalIterator
value:
AbstractFunctionderivativeBlock(s)
![Page 29: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/29.jpg)
©DB Consulting, 1999, all rights reserved
Newton’s Zero FindingNewton’s Zero Finding
x1
x2
Effect of a nearly 0 derivative during iterations
x3
![Page 30: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/30.jpg)
©DB Consulting, 1999, all rights reserved
Road MapRoad Map
Dealing with numerical dataFramework for iterative processesNewton’s zero findingEigenvalues and eigenvectorsCluster analysisConclusion
![Page 31: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/31.jpg)
©DB Consulting, 1999, all rights reserved
Eigenvalues & EigenvectorsEigenvalues & Eigenvectors
Finds the solution to the problemM·v = ·v
where M is a square matrix and v a non-zero vector.
is the eigenvalue. v is the eigenvector.
![Page 32: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/32.jpg)
©DB Consulting, 1999, all rights reserved
Eigenvalues & EigenvectorsEigenvalues & Eigenvectors
Example of uses:– Structural analysis (vibrations).– Correlation analysis
(statistical analysis and data mining).
![Page 33: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/33.jpg)
©DB Consulting, 1999, all rights reserved
Eigenvalues & EigenvectorsEigenvalues & Eigenvectors
We use the Jacobi method applicable to symmetric matrices only.
A 2x2 rotation matrix is applied on the matrix to annihilate the largest off-diagonal element.
This process is repeated until all off-diagonal elements are negligible.
![Page 34: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/34.jpg)
©DB Consulting, 1999, all rights reserved
Eigenvalues & EigenvectorsEigenvalues & Eigenvectors
When M is a symmetric matrix, there exist an orthogonal matrix O such that: OT·M·O is diagonal.
The diagonal elements are the eigenvalues.The columns of the matrix O are the
eigenvectors of the matrix M.
![Page 35: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/35.jpg)
©DB Consulting, 1999, all rights reserved
Eigenvalues & EigenvectorsEigenvalues & Eigenvectors
Let i and j be the indices of the largest off-diagonal element.
The rotation matrix R1 is chosen such that (R1
T·M·R1)ij is 0.The matrix at the next step is:
M2 = R1T·M·R1.
![Page 36: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/36.jpg)
©DB Consulting, 1999, all rights reserved
Eigenvalues & EigenvectorsEigenvalues & Eigenvectors
The process is repeated on the matrix M2.One can prove that, at each step, one has:
max |(Mn )ij | <= max |(Mn-1)ij | for all ij.Thus, the algorithm must converge.The matrix O is then the product of all
matrices Rn.
![Page 37: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/37.jpg)
©DB Consulting, 1999, all rights reserved
Eigenvalues & EigenvectorsEigenvalues & Eigenvectors
The precision is the absolute value of the largest off-diagonal element.
To finalize, eigenvalues are sorted in decreasing order of absolute values.
There are two results from this algorithm:– A vector containing the eigenvalues,– The matrix O containing the corresponding
eigenvectors.
![Page 38: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/38.jpg)
©DB Consulting, 1999, all rights reserved
Example of useExample of use
| matrix jacobi eigenvalues transform |matrix := DhbMatrix rows: #( ( 3 -2 0)
(-2 7 1) (0 1 5)).jacobi := DhbJacobiTransformation new:
matrix.jacobi evaluate.iterativeProcess hasConverged
ifTrue:[ eigenvalues := jacobi result. transform := jacobi transform. ]ifFalse:[ <special case processing>].
| matrix jacobi eigenvalues transform |matrix := DhbMatrix rows: #( ( 3 -2 0)
(-2 7 1) (0 1 5)).jacobi := DhbJacobiTransformation new:
matrix.jacobi evaluate.iterativeProcess hasConverged
ifTrue:[ eigenvalues := jacobi result. transform := jacobi transform. ]ifFalse:[ <special case processing>].
![Page 39: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/39.jpg)
©DB Consulting, 1999, all rights reserved
Class DiagramClass Diagram
evaluateIterationlargestOffDiagonalIndicestransformAt:and:finalizeIterationsortEigenValuesexchangeAt:printOn:
JacobiTransformation
IterativeProcess
lowerRows (i)transform (g)
![Page 40: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/40.jpg)
©DB Consulting, 1999, all rights reserved
Road MapRoad Map
Dealing with numerical dataFramework for iterative processesNewton’s zero findingEigenvalues and eigenvectorsCluster analysisConclusion
![Page 41: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/41.jpg)
©DB Consulting, 1999, all rights reserved
Cluster AnalysisCluster Analysis
Is one of the techniques of data mining.Allows to extract unsuspected similarity
patterns from a large collection of objects.Used by marketing strategists (banks).Example: BT used cluster analysis to locate
a long distance phone scam.
![Page 42: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/42.jpg)
©DB Consulting, 1999, all rights reserved
Cluster AnalysisCluster Analysis
Each object is represented by a vector of numerical values.
A metric is defined to compute a similarity factor using the vector representing the object.
![Page 43: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/43.jpg)
©DB Consulting, 1999, all rights reserved
Cluster AnalysisCluster Analysis
At first the centers of each clusters are defined by picking random objects.
Objects are clustered to the nearest center.A new center is computed for each cluster.Continue iteration until all objects remain in
the cluster of the preceding iteration.Empty clusters are discarded.
![Page 44: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/44.jpg)
©DB Consulting, 1999, all rights reserved
Cluster AnalysisCluster Analysis
The similarity metric is essential to the performance of cluster analysis.
Depending on the problem, a different metric may be used.
Thus, the metric is implemented with a STRATEGY pattern.
![Page 45: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/45.jpg)
©DB Consulting, 1999, all rights reserved
Cluster AnalysisCluster Analysis
The precision is the number of objects that changed clusters at each iteration.
The result is a grouping of the initial objects, that is a set of clusters.
Some clusters may be better than others.
![Page 46: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/46.jpg)
©DB Consulting, 1999, all rights reserved
Example of useExample of use
| server finder |server := <a subclass of DhbAbstractClusterDataServer >
new.finder := DhbClusterFinder new: 15
server: servertype: DhbEuclidianMetric.
finder evaluate.finder tally
| server finder |server := <a subclass of DhbAbstractClusterDataServer >
new.finder := DhbClusterFinder new: 15
server: servertype: DhbEuclidianMetric.
finder evaluate.finder tally
![Page 47: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/47.jpg)
©DB Consulting, 1999, all rights reserved
Class DiagramClass Diagram
evaluateIterationaccumulate:indexOfNearestCluster:initializeIterationfinalizeIteration
ClusterFinder
IterativeProcess
dataServer (i)clusters (i,g)
accumulate:distance:isEmpty:reset
Cluster
openDataStreamcloseDataStreamdimensionseedClusterIn:accumulateDataIn:
ClusterDataServer
accumulator(i)metric(i)count(g)
![Page 48: Numerical Methods](https://reader033.fdocuments.in/reader033/viewer/2022061221/54bd8bef4a79592c508b461f/html5/thumbnails/48.jpg)
©DB Consulting, 1999, all rights reserved
ConclusionConclusion
When used with care numerical data can be processed with Smalltalk.
OO programming can take advantage of the mathematical structure.
Programming numerical algorithms can take advantage of OO programming.