SC-95-42

8/4/2019 SC-95-42

1/17

Konrad-Zuse-Zentrum fr Informationstechnik Berlin

Heilbronner Str. 10,D-10711 Berlin - Wilmersdorf

Wolfram Koepf

Efcient Computation of

Orthogonal Polynomials

in Computer Algeb

Preprint SC 95-42 (December 1995)

8/4/2019 SC-95-42

2/17

8/4/2019 SC-95-42

3/17

Efcient Computation of

Orthogonal Polynomials

in Computer Algeb

Wolfram Koepf

[email protected]

Introduction

rthogonal polynomials can be calculated by computation of determinants, by the use of

generat ing functions, in ter ms of Rodri gues formulas, by ter ati ng recurrence equati ons, calcu

lating the polynomial solutions of differential equations, through closed form representations

and by other means.

In compuer algebra sysems all these methods can be implemened. Depending on the appli

cation on might need

1. on (or many ) of thes e poly nomi als in any form or spec ifca lly in exp and ed form,

2. th exact ration al value of one of thes polynomi als at a certai n rati onal poi nt,

. or a decimal appr oxim atio n of the value of one of these po l no mi al s at a certai point .

In thi s arti cle, we give an overview abo u th efici ency of th e above me th od s in Mapl e,

Mathem atica, and REDU CE. As a noncommercial package we nclude the MuPAD sy se m.

MuPAD is freely distribued for non-commercial use within the scientific community.

Pri mari ly we stu dy th plem entat ion of the Chebyshev polynomi als of the fr st kind as an

exampl case.

irst, we consider the builtin implementations of the Chebyshev polynomials in these systems.

Next we stu dy the classical algor ithms begi nning with the slow ones, and leading to the ef ci en

ones. Finally, we finish with a new algorithm based on a divide and conquer approach which

has a remarkable complexty.

In particular, we will show tha

to obtai n the expan ded form of one of th Chebyshev polynom ials an iterative use of ts

power series repr esent atio n is most ef ci en t,

for numerical purposes (both rationally exact, and decimal approximation) a divide and

conquer appro ach th at is available for Chebyshev polynom ials is much preferable. Thi s

appro ach, however, is not efc ie nt if th expa nded form of th polyn omial is needed.

We present all algorithms as short Maple programs. The other implementations of this articl

may be obtained from the author.
mailto:[email protected]:[email protected]

8/4/2019 SC-95-42

4/17

The C h ey s he v P l y n o i a

Th Chebyshev polynom ials Tn(x) of the frst knd are defned by

Tn(cosi) = cos(nt) , hence Tn(x) = cos(narccosx) . 1)

They form a family ofpolynomials that ar e orthogonal with respect to the scalar product

8/4/2019 SC-95-42

5/17

Mapl Mathematic REDUCE MuPAD

10 000 00 1 00 5 014

100 20 11 83 10

500 2850 433 1 11600

1000 34700 16103 28800 50600

5000 647003

Table 2 Buil tin Chebyshev Polynomi als Calculat ion of Tn{x

Not th at th invocation of th calculati on Tn(x) has q u i e differen conseque nces in th e fousystems:

Mapl e calculates all consecutive Chebyshev polynomials T^ (x) (k 0 . . , n) in ex

panded form ifTn(xo) is issued for some xo, and puts these in memory by the remember

function. Hence the comput ati on times are almos equal in any of th thr ee different

situations.

This procedure has the obvious advantage that all compued functions are immediately

available af er wa rd s. n the other hand, as a disadva ntage the memory is full as soon as

one has issued a single comput ation with high enough B N even if only this particul a

resu l is needed .

Mat hem at i ca calculaes a particular Tn[x) if issued, and uses no remember tables.

For numerical computations, both exact and approximae, Mathematica uses a different

algorithm which is much faster.

R E D U C E calculaes a single Tn(x if issued, and uses no remember tables.

M u P A D also calculaes a single Tn(x if issued, and uses no remember tables.


10 002 000 005 012

100 28 00 40 34

500 2790 00 28 121001000 35300 2490 140050

104 13

105 12 8

106 1283

107 12700

Table Built in Chebyshev Polynomi als Cal culat ion of l)

As a consequence of thes considera tions, Mat hem ati ca seems to have the most ef ci en t buil ti

implementation of the Chebyshev (and other families of orthogonal) polynomials. On the other

hand, as we will see, appropriate implementations enable Maple, REDUCE and MuPAD to

calculate Tn(x) for large n much faster than Mathematica.

3with the setting $Recur sionLi mit=In f in it y. If the user doesn't redefine $RecursionL imit, for n > 494no results are obtained

8/4/2019 SC-95-42

6/17

Maple uses th t hr ee -e rm recurrence equat ion to obta in the polynom ial list. Tabl 8 of


10 000 00 1 005 00710

500 2870 11 12500

1000 34700 2 1000

5000 103

104 205

105

10 0106 10004

Tabl Built in Chebyshev Polynomi als: Appr oxim atio n of l)

gives a fair comparison for this approach between the four sysems, which shows that for large

n N Mat hem ati ca is faster for this approach and can co m p ue a larger list tha n Maple .

However, since the memory and storage requirements are so immense, we think that an eficien

computation of a single Tn(x) is the most imp ort an task. Hence we are mainly i n e r e s e d

to compare the efciency of the computation of Tn(x) for large n (as large as the computer

memory of today's computers allow), and we do not deal with the computation of lists of all

Tk(x) (k = 0 , . . . , n), but mainly with th computat ion of a single Tn(x).

In th e following sectio ns, we will consider th ef ci en cy of different ap pro ache s to calcul aTn[x).

etermnants

Th Chebyshev polynom ials have th repre sent atio n

Tn[x)

as the determinant of an n x n almost) ba nd -m ar ix . In Maple, this is given as

with(linalg):

hebyshevT:=proc(n,x)

local f,A;

A:=band([-l,2*x,-l],n);

A[l,l]:=x;

RETURN(det(A));

end:

Th codes in Mat hema tic a, RED UC E and MuP AD can be def ned analogously.

- 1 2x - 1

2x - 1

- 1 2x - 1- 1 2x

lt 0.

8/4/2019 SC-95-42

7/17

All classical families of orth ogonal polynomi als have similar represen tati ons. Exp and ing th

above determinant yelds the wellknown three-erm recurrence equation for Tn(x) which w

consider in 6.

To calculate Tn{x) via the above determinant is inherently ineffective since the computation

of det erm ina nts of large matri ces is very expensive. Obviously the special s r uc t u r e of th

Chebyshev polynomials is not sufciently utilized by this approach.

Mapl Mathem REDUCE MuPAD

10 04 5 016 0 03 21006

50 23000 0710 47

15

200 64600

Table 5 Determi nant Compu tati on ofTn(x

For the sake of completeness, we give the timings for the determinant approach in Table 5.

Determinant computations are very slow in Maple, Mathematica, and MuPAD, whereas RE

DUCE calculates Tiooo(l) m 78 seconds by this approach. Tn(x) cannot be computed for

generic x with any of the four sysems for n > 00.

enerating Functions

The function

/ n=0

is the generatin g function of the Chebyshev polynomial s. By Taylor' theor em, one can there

fore compue Tn(x) as

T. (. ) = ^n!

In Map l thi s is given as


local F,z,Dn;

F:=(l-x*z)/(l-2*x*z+z~2);

Dn:=diff(F,z$n);

RETURN(subs(z=0,Dn)/n!)

end:

Note that other than the determinant approach the generating functions approach in principl

is capable to calculae the polynomial system iteratively.

Table 6 gives the timings for the calculation of a single Tn(x) with this approach. RE DU CE

brings each ter ate d derivative of F(z to a rational normal representation which is qui

expensive. Maple and Mathematica do not use such normal representations, hence they are

much fa se r. On th e other han d, Mapl e fails very soon beca use of emor y overflow: th

5 with on cramer;.6MuPAD's out put is not in normalized polynomial form. This normalization can be done by normal, bu t needs

exta time.

this time with off crame r;

8/4/2019 SC-95-42

8/17

era ted derivative s are too large obje cts . T hi s is even wors if one defines

: = ( ( l - z ~ 2 ) / ( l - 2 * x * z + z ~ 2 ) + l ) / 2 ,

and hence in this case the timings are worse, too.

The generating functions approach is littl bett er than the determi nant approach in co mp ue r

algebra sy se ms without rational normal representation, but still is qui ine fci ent .

Mapl Mathematic REDUCE8 MuPAD9

10 00 3 03 8 02 2 18 8

50 93 70 11100

1020 16

300 37100400 68200

500 110100

Tabl 6: Generating Function Compu tat ion of Tn(x

odrigues Formulas

Th Chebyshev polynom ials have the Rodri gues repres entat ion

(x)l)

2n yr

- ^

In Map l thi s is given as


normal(( -2 )"n*n! / (2*n)!*sqr t ( l -x~2)*d if f ( ( l -x~2)*(n-1 /2) ,x$n))

end:

All classical families of ortho gonal polyno mials have a similar Rodrig ues repres entat ion. Th

complexty is comparable to the one of the last section.

The iterated derivatives of 1 2 ) 1 ' 2 , however, are simpler functions than the derivatives of

(z) so tha the timings are be t er . I particular , this time the rational normal represent ation

in RE DU CE is useful since t keeps th e mem ory size small , see Tab l 7.

Mapl Mahe R E C E MuPAD

10 00 5 01 5 00 5 21 2

100 70 1360 85 249 0

200 239 8 6010 1960 12700

300 856 0 13800 498 0 4090054 10

500 43100 19000

1000 200000 137500 10

Tabl : Rodrigues Formula Compu tati on ofTn (x)

with off exp ;.9MuPAD's out put is not in normalized polynomial form. This normalizat ion can be done by normal, bu t needs

extra time.10 with set _he ap_ size 3000000;

8/4/2019 SC-95-42

9/17

ecurrence Equations

Now, we start to discuss the methods that are more efficient. The frst such method is the us

of the recurrence equation

Tn{x) = 2xTn(x) - Tn(x 2)

with the initial functions

T0(x) = and {x) = x

Not that via 1) this recurrence equation is equivalent to th rigonometric identity

cos(nt) = 2cosi cos(n l)t) cos(n 2)i) .


10 00 1 00 5 002 00 5100 2 11 55500 29.10 53.60 28.2011 90.30

1000 344.00 173.00

2000 124600

Tabl : Recursive Computation ofTn(x

With r e m e m r , w can use 2) recursivel by the Mapl procedure

hebyshevT:=proc(n,x)option remember;

if n=0 th en 1 el i f n=l th en

else expand(2*x*ChebyshevT(n-1,x)-hebyshevT(n-2,x))f i

end:

The remember option gives recursive rograms linea complexity since all calculations are don

exactly once.

Table 8 shows the timings for this approach. REDUCE generaes variable stack overflow.

The timings for Maple are comparabl to those in Tabl 2, since this is Maple's builtin stra tegy. As already mentioned, the remember feature has the disadvantage that all previously

calculaed T^(x) have to be stored. Therefore the memory requirements are immense.One might have the idea to us th recurrence equation without expanding intermediaresults. Indeed, this decreases th cost by the cost of the expansion, but it genera es so huge

expressions that t turns out not to be a good idea at all, and the resulting expression is

di fc ul t to handl even for small n. Already T2o eeds more than 80kB of storage space withthis approach, compare Tabl 1. Their complicaed ese d structure makes an evaluation of

these objects very time consuming.

The following erative approach


loca l T , i ;

if n=0 then el if n=l then el se

T [ -2 ] := l ; T [ - l ] :=x ;

11 with set_bndstk_size(100000); li sp setq(simplimi t!*,100000);.12 forthcoming vesion of REDUCE will include a remember option like the other systems

8/4/2019 SC-95-42

10/17

fo r from to n doT[0]:=expand(2*x*T[-l]-T[-2]);T[- 2]: =T[ -1] ; T[- l] :=T[0] ;

od;

f i ;RETURN(T[0]);

end:

remembers only the last two polynomials and does therefore not generae memory overflow.

Hence th timings are much be t er .


10 00 1 00 5 0 00 00 7

100 26 16 44 74

1000 18900 1600 393 0 54400

2000 124600 108700 20700 28140024 54

4000 117700

Table 9: Ierative Computation oTn(x)

Thi s is unt il now by fa th e most successful appro ach. All th e sys tems do ra th er well, wi th

RE DU CE being most successful On the other hand , with none of the sys em s one can

calc ulat e iooooOs) using thi s app roa ch. In th e following sections, w consider ethod s wi th

which this is possible.

ifferential Equations

The Chebyshev polynomial Tn(x is the uniqu polyno mial solution of the differential equat ion

l - x f " ( x ) - x f ' ( x ) ( x ) = 0

with the initial valu

if n is odd

(_1) jfn i s e v e n

In [1] a very efcient algorithm to calculate the polynomial and rational solutions of certain

operator equations was published, in particular for liear ordinary differential equations with

polynomial coeffcients like (3).

Using the Maple implementation ratlode of this algorithm, written by M. Bronsein, and

availabl the Maple share libra ry [ , one gets th timi ngs of Table 10.

Mapl

10 050

100 60

1000 73610000 1200

Table 1 : Differential Equatio ns Compu tati on oTn{x)

8/4/2019 SC-95-42

11/17

Th resul ts are again given as expan ded polyno mials .

Note th at th is algor ithm is the frst one to break th complexity barrier in calculati ng Tn(x)

for n > 10000. Moreover TIQQQ(X) is calculaed in no more than a few seconds!

In the next section, we will see th at with more direct a p r o a c h even b e t e r timings are

possible.

Serie epresentations

Since Tn(x) for fixed n N is a polynomial, any closed form series representation might behelpful to calculate it. Several closed form series representations for T(x) are known of which

we only utilize the Taylor expansion at x = 0

2] ( - A - - 1 VT-

8/4/2019 SC-95-42

12/17

2]

ofTn(x) ^ is rather expensive: For any k = 0 [n/2] large factorials have to bek=o

calculaed in both numerator and denominator, and fnally the fraction has to be converted to

lowest terms. Since the coeffcients ^ are integers, this procedure has a large overhead whichcan be o m i e d if one calculaes eratively. Since th erm ratio is given by

(n + 2)(n l)

Akx{n-k) )

the series computation can be done ale rna tively by the Mapl procedure


local k,tmp,result;

if n=0 then RETURN(1) fi;

if n=l then RETURN(x) fi;

mp:=(2*x)~/2;

result:=tmp;

for k from 1 to /2 do

tmp:=-tmp/4/k*(n-2*k+2)*(n-2*k+l)/x~2/(n-k);

result:=result+tmp

od;

RETURN(result);

end:

Note that this approach can always be used if polynomials are given as hypergeometric series,

which applies to all classical orthogonal polynomials.

t turns out that this implementation by far is the most efcient way to calculat the expanded

polynomial Tn(x) for large n N. Maple, REDUCE as well as MuPAD are very efcient indoing so, and leave Mathematica far behind them.

On the other hand, the timings of Tables 2 and 12 suggest tha this is exactly th way how

Mathematica's builtin implementation calculaes th Chebyshev polynomials.15


10 00 0 00 1 00 3 0.02

100 05 25 18 13

10 16510000 30400 302700 20300 50000

20000 176100 1600 22820028 127

Table 12: Iterative Series Computation of Tn(x

The given iterative algori th gives a clue why the algorithm of Abramov, Bronstein and

Petkovsek [1] presented i the last section is so fast: Their algori thm is based on the iterative

computation of series representations, and it calculaes Tn{x) similarly as considered here.

ivde and Conquer Aproach

In this section, we leave the road of trying to find the polynomials in expanded form. Since

) forms an alerna ti ng series with huge integer coefficients, by cancellation t cannot be used

15Actually, Mathematica seems to calculat ak recursivel athe r tha n it ea ti ve l by means of (5) since$RecursionLimit is involved

8/4/2019 SC-95-42

13/17

for numerical purpo ses when using decimal represen tati ons of fixed precision, and t is rathe r

ineffcient when using exact integer arithmetic.

We will find a way to calculae Tn (x) very efciently in a non-expanded form which furthermore

elds also an efficient representation for numerical purposes. Therefore we utilize the formula

ee e.g. [2] 22 .7 .2 4)

2 Tn(x) T(x) = Tn+(x+Tn(x (n > TO) . (6)

Using (6) for T = n and T = n 1, we get the Mapl mpl emen tat ion

hebyshevT:=proc(n,x)option remember;

if n=0 then 1elif n=l then

elif type(n,even) then 2*ChebyshevT(n/2,x)~2-lelse 2*ChebyshevT((n-1)/2,x)*ChebyshevT((n+1)/2,x)-x

f iend:

This is a typical divide and conquer approach since the problem of size n is carried out by th

comp uta tio n of (at most) 2 subp robl ems of size n/ 2. W t h th is approach it is necessary to

us th reme mber feature since otherwi s in te rm ed ia e comp uta tio ns have to be carried ou

several times , result ing in exponent ial com ple xty . the other han d for n = 10 15 e.g., onl50 iterations are necessary.

Table f shows th timings for this ap r oa ch .


1000 000 00 5 2140 00 4106 10 07

10 03 16 11

1012 06 2 15

10 05 25 20

Table 1 : Divide and Conquer Compu tati on of (x)

The effciency of the metho d is due to the fact tha yields very small repre senta tions of

Tn(x) for large n. For Tooo^)) we have for exampl

()oo(20 = 2 ( 2 ( 2 ( 2 ( 2 (

(2 x (2 x 2 ) - x (2 x - 1 l ) - x) y - x)

( 2 y ) - x ) ( 2 (

2 (2 (2 x (2 x ) - x) ( (2 x - 1 - x - x

(2y)-x(2y2-l - x - xf - l - 1

where y is an abreviation for

y = 2 ( ( 2 ; l .

with off exp ;

11

8/4/2019 SC-95-42

14/17

Thi s obviously is a very compac t way to writ e Tiooo(^)> comp are wit h Table 1. Note t ha

expansio n of these expressions cannot be done with similar ef ci en cy as th direct a ppro ach

that we considered in the preceding section.

RE DU CE internal representation makes many evaluations of the expressions co mp ue d nec

essary, hence the timings are bad.

Note that the given representations furthermore enable the fast rationally exact calculation

of Tn(xo) for XQ , and not too large n N , compar e Table 16, e.g.18

/ 1 \ 25121622714275047687831715137700 \J 2 5 1 2 0 0 4 5 6 4 5 8 8 0 2 9 9 4 0 6 4 1 0 7 5 2

Tables 14-15 give th timi ngs of the exact and appr oxima tive calcul ations of T n (l) with th

current approach.

These show that this is a very efcient way to calculae the Chebyshev polynomials accuraely,

in part icul ar wit h ratio nally exact result s. O the other hand, th complexity of the calculat ion

depen ds heavily on the complexity of the out put . Since T n ( l ) = 1 was very simpl e, th

calculati on was done almost insta ntly. If we ca lc ul ae Tn(xo) for rat ional i o / 1, the n th

result typically is a rati onal number with huge nume rat ors and denomin ator s. Hence th

timings are much slower in these cases, the reason of which is the complexty of the result and

not of the algorithm, though.


1000 00 3 00 5 0 06 004

10 03 10 20 0810 18 1

1012 05 2 71 110 2 19

Table 14 ivide and Conquer Com put at ion of T n ( l )

Mapl Mathem R E C E MuPAD

1000 00 3 00 6 008 0.03

106

16 27 0810 05 20 56 10

1012 05 28 112 110 06 1 20

Table 15: Divide and Conquer Appr oxim ati on of l)

In Table 16, we present the timings for the calculation of T n (l /4 ), and in Tabl 17, the number

of digits of both numerators and denominators of the corresponding results are given.

Furthermore, the method gives a very fast algorithm to compute high precision approximations

for high n, e.g. 0

025) = 07208079782290876405505280948925418987994968000. .

Note tha t the algorithm is much fa se r t han Mathe matic a's builti n approach, see Tables -4 .

18 The numerator s and de nomina tors of TioooC o) are too large to be present ed here compare Table 1719 Mathematica returns the wrong result 0.0 for n > 10

Try to calculate this with any other method!

12

8/4/2019 SC-95-42

15/17


1000 00 3 00 5 02 7 00 7

104 48 13 1010 185

10 3970 208 1 7 0 0

106 7610

Table 1 : Divide and Conquer Computation of l /

mer. digits enom. digits

1000 300 301

104 3010 3010

105 30103 30103

106 301029 301030

Table 1 : Numerator and Denominator Size of l /

How accurate are these computations? Table 18 gives the number of correct digits of th

calculations ofTn (025), done with a precision of16 digits, and the system specific approximatemodes (evalf in Maple, N in Mathematica, on rounded in REDUCE, and f loa t in MuPAD).


1000 14 15 18 18106 11 11 15 15

10 11 11

1012 10

10

Table 1 : Accuracy of Approximations ofTn ( 2 5 )

The table shows that the presened divide and conquer algorithm s rather wellconditioned

(ee e.g. [4]), hence the algorithm can be applied for quit large n N, up to n 106, say,without any further precautions.

Unfortunately, such a divide and conquer approach is not available for all classical orthogonal

polynomials. The Chebyshev polynomials of the second type Un(x), however, can be calculaedin a similar way by the identities ( ee e.g. [2] 22.6.26), 22.6.28)

(x) = U+{x) + U { x > t

for TO = 1 and

2Tn(x)Un{x) = U(x)

These give the Mapl implementation

hebyshevU:=proc(n,x)

option remember;

if n=0 then 1

elif n=l then 2*x

elif type(n,even) then 2*hebyshevT(n/2,x)*ChebyshevU(n/2,x)-l

else 2*ChebyshevU((n-l)/2,x)*ChebyshevT((n+l)/2,x)

fi

end:

8/4/2019 SC-95-42

16/17

10 Conclusion

O r article present s algori thms for th comp uta ti on of ortho gonal polynom ials, especially

Chebyshev polynomials, with which one can break the complexty barrier, and receive results

that are not available with previously implemened algorithms.

r considerations show:

1. Th e emcieny of a specific ethod does not only dep end on th e und erl yin g algor ith m,

but also heavily on the specifics of the compuer algebra system used. Here in particula

the inter nal represen tati on plays an mpo rt ant role, but also the efficiency of utilized

suba lgor ithm s (determ inan t comput ati on in Tabl 5, com put ati on of factorials of large

in eg er s in Table 11, . is an issue.

2. Ef ci en t symbolic and efficient numeric com put ati on most l requir e different al gorit hms.

. Remember options can enhance efficiency in specific situations, but often iterative pro

grams are more adequate and faser since memory should be used carefully in compuer

algebra to avoid overflow.

4. For the computation of numerical values of the Chebyshev polynomials, both rationally

exact, and decimal approximation, the presented divide and conquer algorithm is mos

eficient. The same applies to the compu tat ion ofTn(x) and Un(x) if the expanded formis not required.

5. If the expanded form of an orthogonal polynomial is needed, then the iterative use of th

closed form series repr esen tati on is most efficient. Thi s applies also to the comp uta ti on

of orthogonal polnomials for which no divide and conquer approach is available.

cknowledgments

I would like to thank Peter Deuflhard who initiated my studes on the given topic for his

encouragement and support, and Winfried Neun for his help with REDUCE.

14

8/4/2019 SC-95-42

17/17

ferenc

Abramow, S. A., Bronstein, M., Petkovsek, M.: On polynomial solutions of linear oper

ator equations. Proc. of ISSAC 95, ACM ress, New York, 1995, 290296 .

Abramowitz, M. Stegun, I. A. 1964). Handbook of Mathematical Functions. DoverPubl, New York.

ronsein, M.: ratlode. Maple share library, 1995.

Deuflhard, P., Homann, A.: Numerical Analysis. A First Course in ScientiEc Computation. Walter d ru ye r, Berlin-New York, 1995.

Melenk, H.: Th complexty barrier in REDUCE - a case study. Konrad-Zuse-Zenrum

fr Informationsechnik Berlin (ZIB), Technical Report TR 94-06, 1994.

Rivlin, Th. J.: The Chehyshev Polynomials. Pure & Applied Mathematics. John Wiley& Sons, New York-London-Sydney-Toronto, 1974.

Szeg, G.: Orthogonal Polynomials. Amer. Math. Soc. Coll Publ. Vol. 23 , New Yorkty, 199.

Tricomi, F. G: Vorlesungen ber Orthogonalreihen. Grundlehren der Mathematischenissenschafen 76, Sringer-Verlag, Berlin-Gttingen-Heidelberg, 1955.

15

SC-95-42

Documents

Transcript of SC-95-42