Object-Oriented NumericalInterpolation Component in Eiffel

Bachelor Thesis

Benjamin Wuthrich

ETH Zurich

August 12, 2005

Student: Benjamin Wuthrich (bewuethr@student.ethz.ch)

Student-No: 01–918–200

Supervising Assistant: Bernd Schoeller

Supervising Professor: Prof. Dr. Bertrand Meyer

Abstract

This bachelor thesis describes the interpolation cluster of an Eiffel library fornumerical mathematics. It consists of four parts. The first part gives a reviewof the use of object-oriented technology in computational science and presentsNOOCLEUS, the bigger project of which this work is a part. The second partdescribes the design and implementation of the interpolation cluster and itsauxiliary facilities. The third part is an assessment of this cluster’s efficiencyand compares the performance on the basis of four examples with MATLABas an exponent of classic functional programming. The fourth part gives asummary of the thesis and suggests possibilities for future work.

Contents

1 Introduction 4

1.1 Computational science and O-O . . . . . . . . . . . . . . . . . . . 41.2 NOOCLEUS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2 The interpolation cluster 7

2.1 Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.1.1 Polynomial interpolation . . . . . . . . . . . . . . . . . . . 82.1.2 Spline interpolation . . . . . . . . . . . . . . . . . . . . . 122.1.3 Interpolation in two and more dimensions . . . . . . . . . 14

2.2 Design of the cluster . . . . . . . . . . . . . . . . . . . . . . . . . 152.3 Auxiliary classes . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.3.1 NOOCL FUNCTION . . . . . . . . . . . . . . . . . . . . 172.3.2 FUNCTION GIVEN BY TABLE . . . . . . . . . . . . . 172.3.3 FUNCTION GIVEN BY PROGRAM TEXT . . . . . . . 182.3.4 NOOCL VECTOR . . . . . . . . . . . . . . . . . . . . . . 182.3.5 NOOCL MATRIX . . . . . . . . . . . . . . . . . . . . . . 182.3.6 NOOCL INVERTER . . . . . . . . . . . . . . . . . . . . 19

2.4 Interpolation classes . . . . . . . . . . . . . . . . . . . . . . . . . 192.4.1 INTERPOLATED FUNCTION . . . . . . . . . . . . . . 192.4.2 LAGRANGE INTERPOLATED FUNCTION . . . . . . 202.4.3 NEWTON INTERPOLATED FUNCTION . . . . . . . . 202.4.4 HERMITE INTERPOLATED FUNCTION . . . . . . . . 212.4.5 INVERSE INTERPOLATED FUNCTION . . . . . . . . 222.4.6 CUBIC SPLINE INTERPOLATED FUNCTION . . . . 22

3 Assessment of efficiency 23

3.1 Newton interpolation . . . . . . . . . . . . . . . . . . . . . . . . . 243.2 Hermite interpolation . . . . . . . . . . . . . . . . . . . . . . . . 243.3 Linear system of equations . . . . . . . . . . . . . . . . . . . . . . 253.4 Cubic spline interpolation . . . . . . . . . . . . . . . . . . . . . . 26

4 Final remarks 28

4.1 Summary and conclusions . . . . . . . . . . . . . . . . . . . . . . 284.2 Outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

3

Chapter 1

Introduction

This chapter gives a review of the usage (non-usage respectively) of the O-Oparadigm in computational science, presents existing approaches to incorpo-rate O-O in computational science (section 1.1) and gives an introduction toNOOCLEUS (section 1.2), the project of which this work is part.

1.1 Computational science and O-O

Computational science was one of the main reasons for the invention of com-puters: Tasks such as forecasting the weather or predicting the behaviour ofa turbulent flow are assumed1 to be not analytically solvable, so numericalmethods are needed to solve the underlying mathematical problems (e. g. theNavier-Stokes equations).

Computational science (or scientific computing) is the branch of science thatdeals with the mathematical modelling and the implementation of these modelsusing computers for a broad range of engineering and natural science problems.This requires a thorough understanding not only of the problem itself but alsoknowledge in numerics and computer science, so computational scientists arenot “real” computer scientists but rather have an interdisciplinary education2

or—even more distant from being pure computer scientists—used to be me-chanical engineers (or chemists, meteorologists or ...) and acquired numericsand computer science skills only later, in addition of their prime domain.

This may be one of the reasons why regular software in the scientific comput-ing domain is not really state-of-the-art (from a software engineering point ofview)—because its creators are specialists in fields other than computer sci-ence.

Another reason might be the existence of very old, yet powerful libraries such

1Proving this for the Navier-Stokes equations is one of seven “Millennium Problems”, see

http://www.claymath.org/millennium/2As an example see the CSE curriculum at ETH Zurich, http://www.cse.ethz.ch/

4

1.2. NOOCLEUS 5

as BLAS3 (Basic Linear Algebra Subprograms) and, depending on BLAS, LA-PACK4 (Linear Algebra PACKage). BLAS performs basic operations (vec-tor, matrix-vector and matrix-matrix), LAPACK is a library to approach taskssuch as solving systems of linear equations, eigenvalue problems et cetera. Thereference implementation of BLAS and LAPACK is written in Fortran 77, aprocedural language.

Among the biggest benefits of object-oriented programming are factors such asreusability, extendibility and readability (see [1, chapter 2] for an extensive list);all of them are areas where languages like Fortran are weak. Considering alsothe facts that BLAS and LAPACK are old as well as highly optimized and thatmuch of the knowledge put into the algorithms is not documented, we cannotexpect these libraries to represent what we would expect from a modern piece ofsoftware. Quite the contrary, what we find is (formulated a little exaggerated)cryptic code in functions with bulky interfaces.

The operations performed by LAPACK form the core of many applications innumerical mathematics: Solving partial differential equations leads to linearsystems of equations, fitting a polynomial yields a least-squares problem, andso on. This means that the overall efficiency is strongly dependent on theperformance of these fundamental components. LAPACK is freely availableand commercial numerical mathematics libraries employ it extensively; as anexample might be mentioned the NAG C library5 which relies on LAPACK forlinear equations, least-squares and eigenvalue problems and offers an interfaceto BLAS.

One possibility to enjoy the advantages of the O-O paradigm but not to losethe power of the elaborate old libraries is to write wrappers for them in O-Olanguages6. [2] describes in detail the EiffelMath library, a wrapper for the NAGC library at a very high level of abstraction taking advantage of all the Eiffelbenefits such as Design by Contract and (multiple) inheritance. Nonetheless, thedesign of EiffelMath is also not completely free of constraints of the underlyingnon O-O library.

A detailed investigation on the subject of this section can be found in [3, chapter1].

1.2 NOOCLEUS

NOOCLEUS is short for “Numerical O-O Component Library—Eiffel Usedfor Science”. It is an open-source project7 with the final goal of yielding afully-fledged O-O numerics library written in Eiffel. [3] investigates in detailthe suitability of O-O (and Eiffel in particular) for such a task and presentsthe design and implementation of two example clusters, integration and specialfunctions. On the basis of the integration cluster, the author carries out an

3See http://www.netlib.org/blas/4See http://www.netlib.org/lapack/5“The largest commercially available collection of numerical algorithms for C”, by the

Numerical Algorithms Group, see http://www.nag.co.uk/numeric/CL/CLdescription.asp6A large selection of such wrappers can be found at http://www.oonumerics.org/oon/7See http://se.inf.ethz.ch/noocleus/

6 CHAPTER 1. INTRODUCTION

assessment on the performance and compares it non O-O techniques, takingfully optimized C code as a reference.

The bottom line of this assessment is the fact that object technology bears aburden of not more than 30% performance penalty compared with traditionaltechniques, even less in many cases (see [3, page 70] for details).

As this performance penalty can be seen as an investment in the future, ex-pecting the benefits of O-O technology to balance this loss in speed, nothinggets in the way of advancing the development of NOOCLEUS as an exponentof the very small fraction of numerics libraries implemented fully in an O-Olanguage.

Chapter 2

The interpolation cluster

This chapter deals with the core of what this thesis represents, the NOOCLEUSinterpolation cluster. Section 2.1 gives an overview of interpolation as such. Insection 2.2, the considerations behind the final design of the cluster are sketched,the relationships to what is already available in the library are highlighted andthe class hierarchy is presented and commented. Section 2.3 gives details on theclasses that were introduced in the course of designing the interpolation clusterbut are not directly involved in interpolating. Section 2.4 finally is the detaileddescription of the six classes directly involved with interpolation.

2.1 Interpolation

This section gives an overview of interpolation from a mathematical point ofview. Structure and examples follow [4, chapter 3].

The goal of interpolation is to approximate a real-valued function f(x) of thereal variable x or a given data table (xi, yi) by means of a simple function inclosed form. This simple function is constructed in a way that at particularpoints it coincides with the given function f(x) or the data from the table.This in contrast to approximation where the function is determined such thata norm of the difference between the function sought-after and the given table(function, respectively) is minimized.

The question when to use interpolation and when to use approximation for givendata (xi, yi), i = 0, 1, . . . , n can be answered like this in general:

1. If n is large, interpolation does not make much sense, especially when theunderlying data is subject to measurement errors. In this case, a curve asmooth as possible should be put through the data (see figure 2.1 on theleft).

2. If n is small and it makes sense (or is important) that the approximatingfunction at certain positions xi takes the function values yi (or f(xi) re-spectively), the choice of an interpolation method as in figure 2.1 on theright is appropriate.

7

8 CHAPTER 2. THE INTERPOLATION CLUSTER

0 1 2 3 4 5 6 7−1.5

−1

−0.5

0

0.5

1

1.5

0 1 2 3 4 5 6−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

Figure 2.1: Examples of approximation (left) and interpolation (right)

2.1.1 Polynomial interpolation

The Problem

Polynomial interpolation solves the problem of finding for n + 1 given nodesx0, x1, . . . , xn and the corresponding interpolation values y0, y1, . . . , yn a poly-nomial of degree n,

Pn(x) := a0 + a1x + · · · + anxn, (2.1)

which satisfies the interpolation condition

Pn(xi) = yi, i = 0, 1, . . . , n. (2.2)

It can be shown (for a proof see [4, pages 90 and 93]) that exactly one interpo-lation polynomial with property (2.2) exists whose degree is at most n.

If the given interpolation values yi are values of a given function f , handystatements about the error of interpolating the data can be made:

Theorem 1 All nodes are in the interval [a, b]. Let f ∈ Cn+1[a, b] and yi :=f(xi), i = 0, 1, . . . , n. Let Pn the interpolating polynomial of the table (xi, yi)and

ω(x) := (x − x0)(x − x1) · · · (x − xn).

Then for every x ∈ [a, b] there exists a ξ ∈ (a, b) with

f(x) − Pn(x) =ω(x)f (n+1)(ξ)

(n + 1)!.

Thus holds

|f(x) − Pn(x)| ≤|ω(x)|

(n + 1)!max

ξ∈[a,b]|f (n+1)(ξ)|.

A proof can be found in [4, page 91].

2.1. INTERPOLATION 9

Lagrange interpolation

P can be expressed using the n + 1 Lagrange polynomials

Li(x) :=∏j=0j 6=i

(x − xj)

(xi − xj)

=(x − x0) · · · (x − xi−1)(x − xi+1) · · · (x − xn)

(xi − x0) · · · (xi − xi−1)(xi − xi+1) · · · (xi − xn),

namely

Pn(x) :=

n∑i=0

yiLi(x).

Lagrange interpolation is an accurate instrument to prove the uniqueness ofthe interpolation polynomial, but evaluating the polynomial is expensive andnumerically not recommended.

Newton interpolation

The underlying idea of this method is a different representation of the interpo-lation polynomial:

Pn(x) := c0 + c1(x − x0) + c2(x − x0)(x − x1) + · · ·

+ cn(x − x0)(x − x1) · · · (x − xn−1).(2.3)

This is equivalent to (2.1) and the coefficients ci can be computed by insertingthe nodes and interpolation values consecutively. This leads to the so-calleddivided differences that can be computed recursively:

[yk] := yk, k = 0, 1, . . . n,

[yk, yk+1] :=yk+1 − yk

xk+1 − xk

=[yk+1] − [yk]

xk+1 − xk

, k = 0, 1, . . . , n − 1,

[yk, yk+1, . . . , yk+j ] :=[yk+1, . . . , yk+j ] − [yk, . . . , yk+j−1]

xk+j − xk

, (2.4)

j = 2, 3, . . . , n, k = 0, 1, . . . , n − j.

These divided differences get linked to certain interpolation polynomials. Ifwe designate with P ∗

k,k+1,...,k+j the interpolation polynomial of degree j for thenodes xk, xk+1, . . . , xk+j , we have

P ∗k,k+1,...,k+j(xk+i) = yk+i, i = 0, 1, . . . , j,

and with this terminology

P ∗k (xk) = yk = [yk].

10 CHAPTER 2. THE INTERPOLATION CLUSTER

Theorem 2 For i ≤ j ≤ n, the following recursion formula is valid:

P ∗k,k+1,...,k+j(x) =

(x − xk)P ∗k+1,...,k+j(x) − (x − xk+j)P

∗k,...,k+j−1(x)

xk+j − xk

.

For a proof see [4, page 94].

Theorem 2 and (2.4) imply that the divided differences [y0, y1, . . . , yj ] are thesame as the coefficients cj in (2.3). Now the computation of the coefficients canbe summarized in the so-called Newton scheme, displayed below in an examplefor n = 3:

x0 y0 = c0

[y0, y1] = c1

x1 y1 [y0, y1, y2] = c2

[y1, y2] [y0, y1, y2, y3] = c3

x2 y2 [y1, y2, y3][y2, y3]

x3 y3

(2.3) is very convenient to evaluate the polynomial, because common coefficientscan be factored out:

Pn(x) = c0 + c1(x − x0) + · · · + cn(x − x0)(x − x1) · · · (x − xn+1)

= c0 + (x − x0){c1 + (x − x1)[c2 + (x − x2)(· · · cn−1 + (x − xn−1)cn)]}.

This type of polynomial evaluation is called Horner scheme.

The Newton interpolation algorithm combines the computation of the coeffi-cients and the evaluation of the polynomial:

1. Computation of the coefficientsFor k = 0, 1, . . . , n:

ck := yk

For k = 1, 2, . . . , n:for i = n, n − 1, . . . , k:

ci := ci−ci−1

xi−xi−k

2. Horner scheme: Evaluation of the polynomialp := cn

For k = n − 1, n − 2, . . . , 0:p := ck + (x − xk)p

For the value p holds p = Pn(x).

Hermite interpolation

In addition to the interpolation values at the nodes, we now also want to in-terpolate the values of derivatives at the nodes. We can still use the divideddifferences scheme by looking at multiple nodes as limits. The general expressionfor an m-fold node reads

[yk, yk, . . . , yk] :=1

(m − 1)!f (m−1)(xk).

2.1. INTERPOLATION 11

All we have to do is put these values into the Newton scheme whenever a multi-ple node appears. The most common case, double nodes (i. e. first derivatives),yields

[yk, yk] = limh→0

f(xk + h) − f(xk)

(xk + h) − xk

=f(xk + h) − f(xk)

h= f ′(xk).

The complexity of Hermite interpolation with first derivatives for n + 1 nodescorresponds approximately to Newton interpolation for 2n + 2 nodes and theerror/complexity-ratio is roughly equal for both methods.

0 0.2 0.4 0.6 0.8 1−3

−2.5

−2

−1.5

−1

−0.5

0

0.5

1

1.5

f(x)

PN (x)

PH(x)

Figure 2.2: Hermite and Newton interpolation of degree 3

Figure 2.2 compares Hermite and Newton interpolation of the function f(x) =ex sin(5x) with n + 1 = 2 nodes for Hermite and 2n + 2 = 4 nodes for Newtoninterpolation. Although the Hermite method also interpolates the derivativef ′(x) = ex(sin(5x) + 5 cos(5x)) at 0 and 1, the Newton polynomial is muchefficient here. The choice between Newton and Hermite should not be a questionof precision but of availability of data.

Inverse interpolation

Inverse interpolation can be used for a data table (xi, yi = f(xi)), i = 0, 1, . . . , nof a given function f to find a position x where approximately f(x) = y holds fora given y. For reasons of uniqueness, this makes only sense if f(x) is monotonein the interpolation interval. Provided this condition is fulfilled, the Newtonalgorithm is adequate. We designate the interpolation polynomial of the inverse

12 CHAPTER 2. THE INTERPOLATION CLUSTER

−1 −0.5 0 0.5 1 1.5 2 2.5 3−1

−0.5

0

0.5

1

1.5

2

P (x)Q(y)

Figure 2.3: Inverse interpolation and interpolation

problem with x = Q(y); to compute its coefficients we simply interchange thexi and yi in the Newton scheme.

This method can serve to find roots y = 0; figure 2.3 shows inverse and standardinterpolation for the same nodes and interpolation values (−1,−1), (1, 1), (3, 2),but only Q(y) yields results for given y-values, e. g. y = 0 returns x = Q2(y) =−1/3.

2.1.2 Spline interpolation

Polynomial interpolation works well for smaller problems, but as soon as thenumber of nodes and the of the polynomials increases, undesirable oscillatoryeffects show up (see figure 2.4 on the left).

The quality of the interpolation can be enhanced by simply stringing togethermultiple polynomials, but if we are interested not only in function values but theinterpolation function itself, we notice kinks where the function is not smooth,not differentiable. The solution is the spline function, spline in brief. Figure 2.4shows on the right interpolation of the same data as on the left but with twopolynomials and a spline.

In their most general form, splines satisfy three conditions:

• They are piecewise polynomials of order at most k.

• They are p continuously differentiable.

2.1. INTERPOLATION 13

1 2 3 4 5 6 7 8

0

2

4

6

8

10

1 2 3 4 5 6 7 8

2

3

4

5

6

7

Polynomials��

Spline

Figure 2.4: Unwanted oscillatory effects of higher order polynomials (left) andanswers to the problem using multiple polynomials and a spline (right)

• Among the functions which satisfy the other conditions and interpolate agiven table, they are the “smoothest”.

Most usual are cubic splines with k = 3 and p = 2.

Cubic Splines

The problem is defined as follows: Given an interval [a, b] and n + 1 nodesdiffering from each other, a = x0 < x1 < · · · < xn = b, and the associatedinterpolation values y0, y1, . . . , yn, we have to find a function s satisfying thefollowing requirements:

(i) s interpolates the given table: s(xi) = yi, i = 0, 1, . . . , n.

(ii) s is in [a, b] at least once continuously differentiable.

(iii) Inside of each partial interval (xi, xi+1), i = 0, 1, . . . , n − 1, s is at leastfour times continuously differentiable. At the nodes xi, s is differentiablefrom above and below; the limits need not coincide.

(iv) s minimizes

J [s] :=1

2

∫ b

a

s′′(x)2 dx.

This corresponds to minimizing the energy the spline uses to bend andconstitutes the term “smoothest”.

Theorem 3 There exists exactly one such function s.

A detailed proof in the course of which an algorithmic construction of s isderived can be found in [4, pages 106–110]. The idea is to derive a linear systemof equations for the slopes ci := s′(xi). The transformation

t :=x − xi

hi

, hi := xi+1 − xi

maps the partial interval [xi, xi+1] to the interval [0, 1] and thus si to

qi(t) = si(x) = si(xi + hit).

14 CHAPTER 2. THE INTERPOLATION CLUSTER

For each of these cubic polynomials we have four conditions

qi(0) = yi, qi(1) = yi+1,

q′i(0) = hici, q′i(1) = hici+1.

This can be solved with Hermite interpolation (see page 10) and the solutionreads

qi(t) = (1 − t)2yi + t2yi+1+

t(1 − t){(1 − t)(2yi + hici) + t(2yi+1 − hici+1)}.(2.5)

The conditions that lead to the system of linear equations for the ci are on theone hand continuity of s′′i (x) at the inner nodes (n − 1 equations) and on theother hand one of the following:

Natural spline: s′′(x0) = 0, s′′(xn) = 0,Complete spline: s′(x0) = y′

0, s′(xn) = y′n,

Periodic spline: s′(x0) = s′(xn), s′′(x0) = s′′(xn),

each yielding two equations. The resulting coefficient matrix is tridiagonal,symmetric and diagonally dominant and the system can be solved very effi-ciently.

B-splines

The form of the spline function that is the easiest to deal with algorithmicallyis its construction of simple basis functions, the so-called B-splines. In theirmost trivial form, B-splines of first order, they are just hat functions. Higherorder B-splines are computed recursively from B-splines of lower order and donot need extra conditions as the cubic splines do. B-splines of first order andcubic B-splines are described in detail in [4, pages 110–116]; as they are notimplemented in the interpolation cluster we do not examine them any furtherhere.

2.1.3 Interpolation in two and more dimensions

Two dimensional spline methods provide solutions to the problem of interpolat-ing a table of data (xk, yk, fk), k = 1, 2, . . . ,m with a function

S(x, y) :=

n∑ν=1

l∑µ=1

cµνsν(x)tµ

with ln = m and S(xk, yk) = fk, k = 1, 2, . . . ,m. The functions sν and tµ arethe one dimensional ansatz functions.

The most common methods are bilinear and bicubic tensor splines. Again, asthese are not implemented we just refer to [4, pages 117–123].

The last method to be mentioned here is curve interpolation. The problem

is to find for n + 1 given points x(i) = (x

(i)1 , x

(i)2 , . . . , x

(i)m ), i = 0, 1, . . . , n m

parameter functions x1(t), x2(t), . . . , xm(t), t ∈ [0, 1] describing a curve thatinterpolates the given points. The solution is to carry out m one dimensionalspline interpolations; for details see [4, page 124].

2.2. DESIGN OF THE CLUSTER 15

2.2 Design of the cluster

The first decision that influenced the design was the functionality I wanted tooffer in the end. My idea was to include all the methods mentioned in section2.1.1 plus at least cubic splines; B-splines if I would see that I still have timeleft. All the polynomial methods plus cubic splines turned out to be enoughwork so the cluster contains six classes (see figure 2.5).

Figure 2.5: The interpolation cluster

The intention of a first design was to have a deferred top-level class INTERPO-

LATOR not inheriting from any other class. The classes LAGRANGE INTER-

POLATOR, NEWTON INTERPOLATOR, HERMITE INTERPOLATOR andCUBIC SPLINE INTERPOLATOR inherited from INTERPOLATOR and ef-fected the feature item (x position: DOUBLE ): DOUBLE , a function that re-turns for a given value the result of evaluating the interpolation function, bethat a polynomial or a spline.

But what other is a class whose main purpose is returning a computed valuefor a given value than a function? The notion of “function” existed alreadyin NOOCLEUS, there is a class NOOCL FUNCTION . It is deferred and thebody is empty, it simply serves as ancestor of the two types of functions inNOOCLEUS:

• FUNCTION GIVEN BY PROGRAM TEXT is a function in the Eiffelsense of the word; it forwards all incoming calls to a FUNCTION [ANY,

TUPLE [DOUBLE ], DOUBLE ] object. This is what mathematicallycould represent a continuous function.

• FUNCTION GIVEN BY TABLE reflects the fact that functions are notnecessarily given in a closed form but can also appear as tables of argumentand function value pairs.

Details on the design of these three classes can be found in [3, pages 47–48]; seealso sections 2.3.1, 2.3.2 and 2.3.3.

16 CHAPTER 2. THE INTERPOLATION CLUSTER

As consequence of this function-ish behavior of INTERPOLATOR, it became anheir of NOOCL FUNCTION , representing a third type of function. This changeis also reflected in the names of the classes, they are now called functions as inINTERPOLATED FUNCTION .

The classes are all initialized with a FUNCTION GIVEN BY TABLE repre-senting the nodes and interpolation values. All the computations that can bemade based on only this information is carried out in the make feature—e. g.calculation of the polynomial coefficients in Newton interpolation—and the restwhen item is called. There are classes where nothing is computed in make, eitherbecause more information is needed (as in Hermite interpolation where deriva-tives must be given) or because the algorithm needs from the very beginningthe x-position given only in item.

Another question was how to organize Newton-style interpolated functions:Newton, Hermite and inverse interpolation are all based on the Newton schemeto compute polynomial coefficients. If the class that deals with Hermite interpo-lation would offer the possibility to prescribe at every node as many derivativesas desired, one cold end up with the situation of a Hermite interpolated functionwhere no derivatives at all are prescribed; this is nothing else than a Newtoninterpolated function.

One solution considered was having a class that performs Newton interpolationif no derivatives are given and Hermite interpolation if derivatives are given;but this would have harmed the clear structure of the cluster (separate from thefact that HERMITE INTERPOLATED FUNCTION is not able to handle anarbitrary number of derivatives anyways, only first derivatives at every node).In the realized solution it is unambiguous for every client what he is dealingwith.

INVERSE INTERPOLATED FUNCTION finally is an heir of NEWTON IN-

TERPOLATED FUNCTION although it redefines every feature of the latter.An alternative would have been to have an abstraction for the Newton scheme,i. e. a separate class NEWTON SCHEME which would have the Newton, Her-mite and inverse interpolated functions as clients. I decided not to do thisbecause this would be a class exporting only one routine; in [1, page 728] thisis called “glorified subroutine” and not recommended to constitute a properclass.

In the course of implementing the interpolation cluster, the need for some auxil-iary classes arose: NOOCL VECTOR and NOOCL MATRIX in the noocl clus-ter now represent what their names suggest. Their functionality is restrictedto what is needed for interpolation though, they are far from covering a biggerpart of the needs of linear algebra. NOOCL INVERTER is a direct solver forlinear systems of equations but handles only tridiagonal systems without rowinterchange.

2.3. AUXILIARY CLASSES 17

2.3 Auxiliary classes

These classes do not serve the purpose of interpolating anything, but they serveas ancestors or suppliers to the classes dealing with interpolation. With theexception of NOOCL INVERTER they all belong to the cluster noocl , a generalcluster housing classes that offer features that are possibly useful to many classesin the clusters dealing with specific fields of numerics. NOOCL INVERTER isthe only class of the cluster systems of linear equations, a large area by itself,but offers only very limited functionality (see section 2.3.6).

2.3.1 NOOCL FUNCTION

Figure 2.6: NOOCL FUNCTION and its heirs and ancestor

As already mentioned, NOOCL FUNCTION does not do much but being anancestor. In an early phase I asked myself if it would be nice to have make

of INTERPOLATED FUNCTION take not a function given by table, but anyfunction as argument since one often wants to interpolate a function (in themathematical sense of the word). As in the end one has always to providea discrete set of nodes and interpolation values, I decided to leave FUNC-

TION GIVEN BY TABLE as the type of the argument. See also sections 2.3.2and 2.4.1 for further details.

2.3.2 FUNCTION GIVEN BY TABLE

This class used to forward incoming calls to an object of type HASH TABLE

[DOUBLE, DOUBLE ]. This is desirable if we are just interested in values of thefunction at certain points and we know these points. But many algorithms donot state exact positions but only say for example “multiply the values at the

18 CHAPTER 2. THE INTERPOLATION CLUSTER

first and the second position”, and with a hash table this would have resultedin laborious (and slow) CURSOR activity.

So I changed the class to be a wrapper for ARRAY2 [DOUBLE ] at first, andin the end—because many functions just need all the x- or all the y-values—awrapper for two NOOCL VECTOR objects.

A nice to have feature would have been a possibility to initialize a FUNC-

TION GIVEN BY TABLE by providing as the argument a FUNCTION GIV-

EN BY PROGRAM TEXT and one of the following:

• a vector of nodes, or

• a rule such as “equidistant” or “Gauss-Lobatto”

from which the appropriate x- and/or y-vectors would have been computed. Idid not implement this though; this could be subject of future work.

2.3.3 FUNCTION GIVEN BY PROGRAM TEXT

Other than in the cluster integration where this class plays a major role, theinterpolation cluster uses it in only one place: One attribute of INTERPO-

LATED FUNCTION , the continuous version, is of this type (see section 2.4.1for more details).

2.3.4 NOOCL VECTOR

This class was introduced because I needed some kind of “better array”. It in-herits from ARRAY and adds two features, sum of elements (returning the sumof all elements) and product of elements (returning the product of all elements),because these are often needed by the interpolation algorithms.

Because vectors in mathematics (and numerics in particular) always containsome kind of number, genericity is restricted: NOOCL VECTOR [G –> NU-

MERIC ].

2.3.5 NOOCL MATRIX

The notion of “matrix” is omnipresent in numerics: Dealing with systems of lin-ear equations means dealing with (coefficient) matrices, and since cubic spline in-terpolations leads to a such a system, NOOCL MATRIX was introduced.

This class inherits from ARRAY2 and adds functionality. Because cubic splineinterpolation yields only one type of matrix, tridiagonal diagonally dominant(and even symmetric), this class is not really all-purpose. It offers a make-routine taking three vectors as arguments and creating such a tridiagonal ma-trix. Boolean attributes to indicate the fact that the matrix is tridiagonal anddiagonally dominant are provided and help the solver of linear systems in de-ciding what algorithm to choose. This will be important and helpful as soon asthere are many algorithms to choose from in NOOCL INVERTER.

2.4. INTERPOLATION CLASSES 19

Just as vectors, matrices in NOOCLEUS are restricted to NUMERIC types.

NOOCL MATRIX and NOOCL VECTOR form the basis of the linear algebrain NOOCLEUS, and future work dealing with systems of equations, matrix-matrix, vector-matrix and vector-vector operations will have to extend the func-tionality of the two as by now they serve only very specific purposes.

2.3.6 NOOCL INVERTER

This class is initialized with a coefficient matrix; calling solution providing aright hand side vector returns a solution vector.

Again, this class is very specialized and requires the matrix passed as argumentto make to be tridiagonal and diagonally dominant. Turning this class into ageneral direct solver for all kinds of linear systems of equations would be subjectof a project of its own, possibly involving lots of external calls (see remarks onefficiency, section 3.3 and conclusions, section 4.1).

2.4 Interpolation classes

These are the classes taking care of the core business: They create nice functionsfrom a discrete set of points, more or less efficiently, but in any case offering auniform interface whose usage should be self-explanatory.

In the following sections some design considerations and remarks for each classare assorted.

2.4.1 INTERPOLATED FUNCTION

This class is heir of NOOCL FUNCTION—an interpolated function still isa function—and client of the two other effective function classes. The dis-

crete function attribute is never void because initialization attaches it to anobject of the FUNCTION GIVEN BY TABLE type; continuous version in con-trast is optional.

The idea behind offering a possibility to have also the closed form of the func-tion is the following: Certain tasks such as error estimate (see 2.1.1) need thecontinuous version of the function. It is also imaginable that a measurement ofthe interpolation error integrates the original and the interpolated function andcompares the integrals. For this purpose, having that continuous version in thesame place as the table-shaped one is convenient.

To ensure that this continuous version in fact corresponds to the discrete one, theclass offers a boolean function to check compliance (continuous is corresponding).

20 CHAPTER 2. THE INTERPOLATION CLUSTER

Figure 2.7: INTERPOLATED FUNCTION and its heirs, ancestor and suppli-ers

2.4.2 LAGRANGE INTERPOLATED FUNCTION

Lagrange interpolation is slow and the very same task is accomplished by New-ton interpolation much faster. So why bother with it at all?

Several times in the course of learning something (e. g. different methods tosolve a certain problem), it annoyed me that software always implements onlythe fastest mechanism. Where was the light bulb moment of seeing how much

better the best method was, if there were no slower ones at all? NOOCLEUSfills this gap and offers also suboptimal methods for educational purposes, so tospeak.

All the calculations are not performed until item is called because the Lagrangepolynomials include the x-position (see page 9).

2.4.3 NEWTON INTERPOLATED FUNCTION

Newton interpolation is much more efficient: Upon initialization, make alreadycomputes the coefficients of the polynomial. A call to item triggers only theevaluation of the polynomial using the given x position; this Horner scheme isquite fast. For the algorithm see page 9.

The coefficients are stored separately in an exported vector such that the poly-nomial can be constructed explicitly if desired.

2.4. INTERPOLATION CLASSES 21

Figure 2.8: NEWTON INTERPOLATED FUNCTION and its heir, ancestorand supplier

2.4.4 HERMITE INTERPOLATED FUNCTION

Hermite interpolation in NOOCLEUS is restricted to a special case (the mostcommon one): Interpolating first derivatives at every node. As seen on page 11,first derivatives can be put into the Newton scheme by using multiple nodes asin [yk, yk] = f ′(xk).

Implementing this was a bit more involved than the simple formula might leadto assume. The coefficients are computed as soon as the derivatives are set usingset derivatives. To this end, every entry in discrete function is doubled. Stor-ing this back to discrete function would violate the command/query-separationprinciple, so a new object, temporary discrete function, is created and used forthe Newton scheme.

Figure 2.9: HERMITE INTERPOLATED FUNCTION and its ancestor andsuppliers

22 CHAPTER 2. THE INTERPOLATION CLUSTER

Calculating the Newton scheme is done in two phases: The column with firstdivided differences is computed by inserting f ′(xk) whenever appropriate (i. e.in the [yk, yk]-case) and performing the standard operation in the other cases.From the next column on the procedure is as usual.

Evaluation of the polynomial again utilizes the double-sized temporary dis-

crete function, so after the somewhat lengthy calculations when assigning anew vector of derivatives, Hermite interpolation is just as fast as Newton (withthe corresponding number of nodes).

2.4.5 INVERSE INTERPOLATED FUNCTION

Inverse interpolation is very similar to Newton interpolation, just the first twocolumns in the Newton scheme are exchanged, and the polynomial is a functionof y and not of x. Correspondingly, item takes y value as argument.

The function to be interpolated must be monotone, a precondition that cannotbe checked formally but ensures the uniqueness of Result in item.

2.4.6 CUBIC SPLINE INTERPOLATED FUNCTION

Cubic splines are the most expensive of all methods in interpolation but alsothe only ones that can deal with situations where many nodes are given (seefigure 2.4 on the right).

Initialization with make puts up the linear system of equations for the firstderivatives ci and solves it using NOOCL INVERTER. The required two extraconditions are s′′(x0) = 0 and s′′(xn) = 0 so the result is always a naturalspline. Solving the system is also the time-critical part of spline computation,see section 3.4 for a comparison with a spline interpolation using a LAPACK-based solver.

A call to item triggers two things: First, the appropriate interval is searched for,and second, the cubic polynomial (2.5) of that interval is evaluated.

Chapter 3

Assessment of efficiency

The design of a numerics library can be brilliant, but nobody works with a li-brary if every computation takes ages. In this chapter, the interpolation clustercompetes against MATLAB functions performing similar tasks. Instead of hav-ing a problem, solving it a million times and dividing the time required to solvethem by a million, many versions of the same problem were created. Theseversions were of increasing size and provided insights into how an increasingsystem size affects the behavior of the performance.

The test system was the following:

Intel Pentium M CPU 1.7 GHz512 MB RAMMicrosoft Windows XP SP2ISE EiffelStudio 5 (5.5.1012 Free Edition)Borland C++ Compiler 5.5MATLAB Student Version 7.0.1.15 (R14) SP1

Why MATLAB? Firstly, MATLAB claims to let its users develop applicationsquickly, faster than in languages such as C, C++ and Fortran. This came in veryhandy as I once again had to provide more or less the complete functionalityof the interpolation cluster in MATLAB; the programming certainly was notdoubled when writing these functions. Secondly, execution speed can competewith the equivalent C or C++ code by using LAPACK and BLAS (see section1.1) for linear algebra and FFTW1 for discrete Fourier transforms.

MATLAB offers some O-O facilities, but for this assessment they were not usedat all (they would not improve performance anyway).

The assessment consisted of four test cases: Newton interpolation (section 3.1),Hermite interpolation (section 3.2), a linear system of equations as the time-critical part of cubic spline interpolation (section 3.3) and cubic spline interpo-lation (section 3.4).

The exact measurement for every system size was plotted and a fitted curveadded, be it quadratic or linear.

1“Fastest Fourier Transform in the West”, see http://www.fftw.org/

23

24 CHAPTER 3. ASSESSMENT OF EFFICIENCY

3.1 Newton interpolation

In this assessment, a given set of points had to be interpolated using New-ton interpolation. Starting with two points, every iteration a point was addedwith the x-value increased by a U(0, 3)-distributed value; the y-values were allU(0, 5)-distributed. To obtain a relatively smooth curve for the time required,for every system size the coefficients were computed 70 times, the polynomialwas evaluated 70 times for n− 1 points (with n the system size) and in the endeverything was averaged.

0 20 40 60 80 100 120 140−2

0

2

4

6

8

10

12

14

16x 10

−3

Number of nodes

Tim

e in

sec

onds

MATLAB

Eiffel

0 50 100 150 2000

1

2

3

4

5

6

7

8x 10

−3

Number of nodes

Tim

e in

sec

onds

MATLAB

Eiffel

Figure 3.1: Performance of NEWTON INTERPOLATED FUNCTION com-pared with the built-in (left) and a self-made MATLAB function (right)

The Eiffel implementation was compared with MATLAB’s built-in polynomialfitting function (figure 3.1 on the left) and a function I wrote myself (figure 3.1on the right). Comparing with the built-in function is not quite fair, because itprovides also curve approximation and checks if the x-values are close to eachother (and thus the polynomial is badly conditioned), so it does more than myEiffel class offers. This extra effort has its price as can be seen from the requiredtime compared with Eiffel. For systems larger than 130 nodes, MATLAB suffersa complete collapse of performance.

Using the self-made function (without any condition checking, bells and whis-tles), MATLAB shows its teeth: From a system of size about 120 on, it out-performs Eiffel. This is presumably due to garbage collection in Eiffel, and alsothe jump at about 190 nodes could have its explanation in memory manage-ment.

As shown above (see figure 2.4), Newton interpolation is not really interestingany more for systems with more than about 10 nodes, let alone 100 nodes.So the bottom line is that Eiffel is faster for systems of a size where Newtoninterpolation makes sense.

3.2 Hermite interpolation

The setting for this test was very similar to the Newton one; additionally, avector y prime with first derivatives (U(−1, 1)-distributed) was carried along.

3.3. LINEAR SYSTEM OF EQUATIONS 25

0 20 40 60 80 100 1200

1

2

3

4

5

6

7

8x 10

−3

Number of nodes

Tim

e in

sec

onds

MATLAB

Eiffel

Figure 3.2: Performance of HERMITE INTERPOLATED FUNCTION com-pared with a self-made MATLAB function

The results are similar as for Newton interpolation with the self-made MATLABfunction: For small systems, Eiffel is faster; for systems with more than about45 nodes (corresponds to about 90 nodes in Newton interpolation), MATLABwins. And again, as nobody wants to do Hermite interpolation with 40 nodesor more, NOOCLEUS is a useful alternative here.

3.3 Linear system of equations

Computing a spline requires solving a linear system of equations, and that partis also the computationally most expensive one. The coefficient matrix is tridi-agonal and diagonally dominant, so no row interchanges are necessary.

MATLAB relies on LAPACK to solve such problems, and LAPACK offers spe-cial solvers for band matrices. Although the Eiffel solver in NOOCL INVERTER

is also somewhat specialized (read “restricted”), it does not treat the coefficientmatrix as sparse, i. e. stores all the zeros around the three diagonals, whereasMATLAB stores only the non-zero elements.

The task was to solve a system of increasing size, 350 times per size. Thecoefficient matrix was tridiagonal with random elements; to ensure diagonal

26 CHAPTER 3. ASSESSMENT OF EFFICIENCY

dominance, the main diagonal’s elements were U(1.1, 2.1)-distributed and thesecondary diagonals’ elements U(0, 1).

0 50 100 150 200 250−2

0

2

4

6

8

10

12x 10

−4

Size of system

Tim

e in

sec

onds

MATLAB

Eiffel

Figure 3.3: Performance of NOOCL INVERTER compared with the built-inMATLAB function

Competing with LAPACK means running against quite the best there is, andcorrespondingly devastating is the result: Figure 3.3 shows that the Eiffelsolver’s time behaves like O(n2) (with n the system size), although the num-ber of operations equals 5n − 4 = O(n). This must be due to carrying alongthe full matrix which grows like O(n2). See section 4.1 for what could be aremedy.

3.4 Cubic spline interpolation

The last assessment was interpolating points constructed similarly as in theNewton interpolation assessment (section 3.1), but using cubic spline interpola-tion.

Now also large systems are interesting, because that is how these are dealtwith. Every system had to be interpolated 250 times before size was increasedby one, and still MATLAB timing behaves very oscillatory for my self-madecubic splines. But this cannot hide the fact the MATLAB is nearly everywhereslower than Eiffel (figure 3.4 on the left).

If MATLAB’s built-in spline is used, the effect of having an O(n)-solver for

3.4. CUBIC SPLINE INTERPOLATION 27

0 50 100 150 2000

0.5

1

1.5

2

2.5

3

3.5

4x 10

−3

Number of nodes

Tim

e in

sec

onds

MATLAB

Eiffel

0 50 100 150 2000

0.5

1

1.5

2

2.5

3

3.5x 10

−3

Number of nodes

Tim

e in

sec

onds

MATLAB

Eiffel

Figure 3.4: Performance of CUBIC SPLINE INTERPOLATED FUNCTION

compared with a self-made (left) and the built-in MATLAB function (right)

linear systems of equations becomes visible again (figure 3.4 on the right): Theelapse time grows linearly, although starting at a high level (and showing thisunexplainable peak for very small systems). This overhead is most likely dueto extended functionality: The function provides also complete splines and per-forms numerous additional if/then-checks.

This means that for systems of size about 180 and above, MATLAB is fasterthan Eiffel and will be much faster for very large systems.

Chapter 4

Final remarks

4.1 Summary and conclusions

The results found in this thesis confirm what was found in [3]: Object-Orientedtechnology is not a burden but an enrichment for computational science.

The performance of the implemented Eiffel cluster for interpolation is completelycapable of competing with MATLAB’s performance. The only exception con-stitutes the solution of linear systems of equations: Here we have to admitsuperiority to the LAPACK based solvers of MATLAB. As it is very unlikelyto be able to obtain a similar performance in these tasks by re-implementingthese algorithms from scratch, we should ask ourselves how pure (in the sense of“pure Eiffel”) NOOCLEUS has to be. Bertrand Meyer’s opinion is the following:Answering one of the two questions

• Is there any expectation that the element might change?

• Is there anything else to be gained by using an object-oriented techniquefor the element?

with “yes” means that the element should be re-implemented in Eiffel. If theanswer is “no” for both questions, the element can be implemented using ex-ternal calls, but with Bertrand Meyer’s words: “I would still treat this as amatter of expediency, to save time in favor of working on software elements forwhich a rewrite will yield more visible benefits.” So ultimately, NOOCLEUSshould be purely Eiffel, but external functions may be considered a remedy untilthen.

In my opinion, for an element performing the task of solving a tridiagonal lin-ear system of equations, both of the above questions can be given a negativeresponse. This means, that NOOCL INVERTER should not try and reinventthe wheel, but call external libraries. I see two options:

1. Calling a C library (open source, if possible) that wraps LAPACK or is aFortran to C conversion of LAPACK, e. g. CLAPACK1;

1See http://www.netlib.org/clapack/

28

4.2. OUTLOOK 29

2. writing a C wrapper for LAPACK and wrap this wrapper in Eiffel (thiswould be the more demanding way).

One last remark concerns Eiffel’s “Design by contract”: Having defined accuratepre- and postconditions significantly facilitated the task of dealing with indicesin arrays and tables and the like and will help authors of the cluster’s clientclasses to write correct software. Furthermore, the performance cost will beeliminated due to the fact that all the assertion checking will be inactive in aproduction version.

4.2 Outlook

There are some details in the cluster treated by this thesis that could be com-plemented:

• A possibility to initialize a FUNCTION GIVEN BY TABLE using a con-tinuous function and an instruction or a vector of nodes (see section 2.3.2),

• Hermite interpolation with an arbitrary number of derivatives at everynode,

• Cubic spline interpolation offering a choice between natural, complete andperiodic spline,

• B-splines and

• two dimensional interpolation and curve interpolation,

whereas the last two are probably not just small details.

NOOCLEUS is still far from being a complete library for numerical mathemat-ics, and I was the first to work on the subject after the initial diploma thesisby Peter Hafliger ([3]) which was completed in March 2003. In the future, theintervals between two projects dealing with NOOCLEUS is likely to be muchshorter, as there is an advertised project at the chair of software engineeringengaging in the development of a web-site for the project, hopefully attractingfurther developers.

Acknowledgements

I would like to thank the following people (in alphabetical order):

Peter Hafliger for taking his time, providing useful comments and sending meany material (even if I only just mentioned it in passing).

Bertrand Meyer for answering arbitrary questions at any time of the day fromanywhere in the world.

Bernd Schoeller for providing a single solution for any problem ever occurring:the one that worked.

Carolyn Wardle-Davies for proofreading my report; any idiosyncratic con-structions are thanks to her and any remaining linguistic flaws are an oversighton my part.

30

Bibliography

[1] Bertrand Meyer. Object-Oriented Software Construction, 2nd edition. Pren-tice Hall, New Jersey, 1997

[2] Paul F. Dubois. Object Technology for Scientific Computing. Prentice Hall,New Jersey, 1996

[3] Peter Hafliger. Exploration of the Suitability of O-O Techniques for the

Design and Implementation of a Numeric Math Library using Eiffel. Diplomathesis, ETH Zurich, 2003

[4] Hans-Rudolf Schwarz, Norbert Kockler. Numerische Mathematik, 5. Auf-lage. Teubner, Wiesbaden, 2004

31