Post on 06-Oct-2018
1Orthogonal functions
Function approximation: Fourier, Chebyshev, Lagrange
Orthogonal functionsFourier SeriesDiscrete Fourier SeriesFourier Transform: propertiesChebyshev polynomialsConvolutionDFT and FFT
Scope: Understanding where the Fourier Transform comes from. Moving from the continuous to the discrete world. The concepts are the basis for pseudospectral methods and the spectral element approach.
2Orthogonal functions
Fourier Series: one way to derive them
The Problem
we are trying to approximate a function f(x) by another function gn (x) which consists of a sum over N orthogonal functions Φ(x) weighted by some coefficients an .
)()()(0
xaxgxfN
iiiN ∑
=
Φ=≈
3Orthogonal functions
... and we are looking for optimal functions in a least squares (l2 ) sense ...
... a good choice for the basis functions Φ(x) are orthogonal functions. What are orthogonal functions? Two functions f and g are said to be
orthogonal in the interval [a,b] if
∫ =b
a
dxxgxf 0)()(
How is this related to the more conceivable concept of orthogonal vectors? Let us look at the original definition of integrals:
The Problem
{ } !Min)()()()(2/1
2
2=⎥
⎦
⎤⎢⎣
⎡−=− ∫
b
aNN dxxgxfxgxf
4Orthogonal functions
Orthogonal Functions
... where x0 =a and xN =b, and xi -xi-1 =Δx ...If we interpret f(xi ) and g(xi ) as the ith components of an N component
vector, then this sum corresponds directly to a scalar product of vectors. The vanishing of the scalar product is the condition for orthogonality of
vectors (or functions).
⎟⎠
⎞⎜⎝
⎛Δ= ∑∫
=∞→
N
iii
b
aN
xxgxfdxxgxf1
)()(lim)()(
figi
0==• ∑ ii
iii gfgf
5Orthogonal functions
Periodic functions
-15 -10 -5 0 5 10 15 200
10
20
30
40
Let us assume we have a piecewise continuous function of the form
)()2( xfxf =+ π
2)()2( xxfxf ==+ π
... we want to approximate this function with a linear combination of 2π periodic functions:
)sin(),cos(),...,2sin(),2cos(),sin(),cos(,1 nxnxxxxx
{ }∑=
++=≈⇒N
kkkN kxbkxaaxgxf
10 )sin()cos(
21)()(
6Orthogonal functions
Orthogonality
... are these functions orthogonal ?
0,00)sin()cos(
0
0,,0)sin()sin(
002
0)cos()cos(
>≥=
>=
>≠
⎪⎩
⎪⎨
⎧=
>===
≠
⎪⎩
⎪⎨
⎧=
∫
∫
∫
−
−
−
kjdxkxjx
kj
kjkjdxkxjx
kjkj
kjdxkxjx
π
π
π
π
π
π
π
ππ
... YES, and these relations are valid for any interval of length 2π.Now we know that this is an orthogonal basis, but how can we obtain the
coefficients for the basis functions?
from minimising f(x)-g(x)
7Orthogonal functions
Fourier coefficients
optimal functions g(x) are given if
{ } 0)()(!Min)()(22=−=− ∂
∂ xfxgorxfxg nan k
leading to
... with the definition of g(x) we get ...
{ }⎥⎥⎦
⎤
⎢⎢⎣
⎡⎥⎦
⎤⎢⎣
⎡−++
∂∂
=−∂∂
∫ ∑− =
dxxfkxbkxaaa
xfxga
N
kkk
kn
k
2
10
2 )()sin()cos(21)()(
2
π
π
{ }
∫
∫
∑
−
−
=
==
==
++=
π
π
π
π
π
π
Nkdxkxxfb
Nkdxkxxfa
kxbkxaaxg
k
k
N
kkkN
,...,2,1,)sin()(1
,...,1,0,)cos()(1
with)sin()cos(21)(
10
8Orthogonal functions
Fourier approximation of |x|
... Example ...
.. and for n<4 g(x) looks like
leads to the Fourier Serie
⎭⎬⎫
⎩⎨⎧ +++−= ...
5)5cos(
3)3cos(
1)cos(4
21)( 222
xxxxgπ
π
ππ ≤≤−= xxxf ,)(
-20 -15 -10 -5 0 5 100
1
2
3
4
9Orthogonal functions
Fourier approximation of x2
... another Example ...
π20,)( 2 <<= xxxf
.. and for N<11, g(x) looks like
leads to the Fourier Serie
∑= ⎭
⎬⎫
⎩⎨⎧ −+=
N
kN kx
kkx
kxg
12
2
)sin(4)cos(43
4)( ππ
-10 -5 0 5 10-10
0
10
20
30
40
10Orthogonal functions
Fourier - discrete functions
iN
xiπ2
=
.. the so-defined Fourier polynomial is the unique interpolating function to the function f(xj ) with N=2m
it turns out that in this particular case the coefficients are given by
,...3,2,1,)sin()(2
,...2,1,0,)cos()(2
1
*
1
*
==
==
∑
∑
=
=
kkxxfN
b
kkxxfN
a
N
jjj
N
jjj
k
k
{ } )cos(21)sin()cos(
21)( *
1
1
****0 kxakxbkxaaxg m
m
km kk
+++= ∑−
=
... what happens if we know our function f(x) only at the points
11Orthogonal functions
)()(*iim xfxg =
Fourier - collocation points
... with the important property that ...
... in our previous examples ...
-10 -5 0 50
0.5
1
1.5
2
2.5
3
3.5
f(x)=|x| => f(x) - blue ; g(x) - red; xi - ‘+’
12Orthogonal functions
Fourier series - convergencef(x)=x2 => f(x) - blue ; g(x) - red; xi - ‘+’
13Orthogonal functions
Fourier series - convergencef(x)=x2 => f(x) - blue ; g(x) - red; xi - ‘+’
14Orthogonal functions
Gibb’s phenomenonf(x)=x2 => f(x) - blue ; g(x) - red; xi - ‘+’
0 0.5 1 1.5-6
-4
-2
0
2
4
6
N = 32
0 0.5 1 1.5-6
-4
-2
0
2
4
6
N = 16
0 0.5-6
-4
-2
0
2
4
6
0 0.5 1 1.5-6
-4
-2
0
2
4
6
N = 128
0 0.5 1 1.5-6
-4
-2
0
2
4
6
N = 256
The overshoot for equi- spaced Fourier
interpolations is ≈14% of the step height.
15Orthogonal functions
Chebyshev polynomials
We have seen that Fourier series are excellent for interpolating (and differentiating) periodic functions defined on a regularly spaced grid. In many circumstances physical phenomena which are not periodic (in space) and occur in a limited area. This quest leads to the use of Chebyshev polynomials.
We depart by observing that cos(nϕ) can be expressed by a polynomial in cos(ϕ):
1cos8cos8)4cos(cos3cos4)3cos(
1cos2)2cos(
24
3
2
+−=
−=
−=
ϕϕϕ
ϕϕϕ
ϕϕ
... which leads us to the definition:
16Orthogonal functions
Chebyshev polynomials - definition
NnxxxTTn nn ∈−∈=== ],1,1[),cos(),())(cos()cos( ϕϕϕ
... for the Chebyshev polynomials Tn (x). Note that because of x=cos(ϕ) they are defined in the interval [-1,1] (which - however - can be extended to ℜ). The first polynomials are
0
244
33
22
1
0
and]1,1[for1)(where188)(
34)(
12)(
)(1)(
NnxxTxxxT
xxxT
xxT
xxTxT
n ∈−∈≤
+−=
−=
−=
==
17Orthogonal functions
Chebyshev polynomials - Graphical
The first ten polynomials look like [0, -1]
The n-th polynomial has extrema with values 1 or -1 at
0 0.2 0.4 0.6 0.8-1
-0.5
0
0.5
1
x
T_n(
x)
nkn
kx extk ,...,3,2,1,0),cos()( ==
π
18Orthogonal functions
Chebyshev collocation points
These extrema are not equidistant (like the Fourier extrema)
nkn
kx extk ,...,3,2,1,0),cos()( ==
π
k
x(k)
19Orthogonal functions
Chebyshev polynomials - orthogonality
... are the Chebyshev polynomials orthogonal?
02
1
1
,,0
02/0
1)()( Njk
jkforjkforjkfor
xdxxTxT jk ∈
⎪⎭
⎪⎬
⎫
⎪⎩
⎪⎨
⎧
==>=
≠=
−∫− π
π
Chebyshev polynomials are an orthogonal set of functions in the interval [-1,1] with respect to the weight functionsuch that
21/1 x−
... this can be easily verified noting that
)cos()(),cos()(sin,cos
ϕϕϕϕϕ
jxTkxTddxx
jk ==−==
20Orthogonal functions
Chebyshev polynomials - interpolation
... we are now faced with the same problem as with the Fourier series. We want to approximate a function f(x), this time not a
periodical function but a function which is defined between [-1,1]. We are looking for gn (x)
)()(21)()(
100 xTcxTcxgxf
n
kkkn ∑
=
+=≈
... and we are faced with the problem, how we can determine the coefficients ck . Again we obtain this by finding the extremum
(minimum)
{ } 01
)()(2
1
1
2 =⎥⎦
⎤⎢⎣
⎡
−−
∂∂∫− x
dxxfxgc n
k
21Orthogonal functions
Chebyshev polynomials - interpolation
... to obtain ...
nkx
dxxTxfc kk ,...,2,1,0,1
)()(2 1
12
=−
= ∫−π
... surprisingly these coefficients can be calculated with FFT techniques, noting that
nkdkfck ,...,2,1,0,cos)(cos2
0
== ∫ ϕϕϕπ
π
... and the fact that f(cosϕ) is a 2π-periodic function ...
nkdkfck ,...,2,1,0,cos)(cos1== ∫
−
ϕϕϕπ
π
π
... which means that the coefficients ck are the Fourier coefficients ak of the periodic function F(ϕ)=f(cos ϕ)!
22Orthogonal functions
Chebyshev - discrete functions
iN
xiπcos=
... leading to the polynomial ...
in this particular case the coefficients are given by
2/,...2,1,0,)cos()(cos21
* NkkfN
cN
jjjk
== ∑=
ϕϕ
∑=
+=m
kkkm xTcTcxg
1
**0
* )(21)( 0
... what happens if we know our function f(x) only at the points
... with the property
N0,1,2,...,jj/N)cos(xat)()( j* === πxfxg m
23Orthogonal functions
Chebyshev - collocation points - |x|
f(x)=|x| => f(x) - blue ; gn (x) - red; xi - ‘+’
-1 -0 .8 -0 .6 -0 .4 -0 .2 0 0 .2 0 .4 0 .6 0 .80
0 .2
0 .4
0 .6
0 .8
1N = 8
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.80
0.2
0.4
0.6
0.8
1N = 16
8 points
16 points
24Orthogonal functions
Chebyshev - collocation points - |x|
f(x)=|x| => f(x) - blue ; gn (x) - red; xi - ‘+’
32 points
128 points
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.80
0.2
0.4
0.6
0.8
1N = 32
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.0
0.2
0.4
0.6
0.8
1N = 128
25Orthogonal functions
Chebyshev - collocation points - x2
f(x)=x2 => f(x) - blue ; gn (x) - red; xi - ‘+’
8 points
64 points
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.80
0.2
0.4
0.6
0.8
1
1.2N = 8
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 10
0.2
0.4
0.6
0.8
1
1.2N = 64
The interpolating function gn (x) was shifted by a small amount to be visible at all!
26Orthogonal functions
Chebyshev vs. Fourier - numerical
f(x)=x2 => f(x) - blue ; gN (x) - red; xi - ‘+’
This graph speaks for itself ! Gibb’s phenomenon with Chebyshev?
0 0.2 0.4 0.6 0.8 10
0.2
0.4
0.6
0.8
1 N = 16
0 2-5
0
5
10
15
20
25
30
35
N =
Chebyshev Fourier
27Orthogonal functions
Chebyshev vs. Fourier - Gibb’s
f(x)=sign(x-π) => f(x) - blue ; gN (x) - red; xi - ‘+’
Gibb’s phenomenon with Chebyshev? YES!
Chebyshev Fourier
-1 -0.5 0 0.5 1-1.5
-1
-0.5
0
0.5
1
1.5 N = 16
0 2-1.5
-1
-0.5
0
0.5
1
1.5N = 16
28Orthogonal functions
Chebyshev vs. Fourier - Gibb’s
f(x)=sign(x-π) => f(x) - blue ; gN (x) - red; xi - ‘+’
Chebyshev Fourier
-1 -0.5 0 0.5 1-1.5
-1
-0.5
0
0.5
1
1.5 N = 64
0 2 4-1.5
-1
-0.5
0
0.5
1
1.5N = 6
29Orthogonal functions
Fourier vs. Chebyshev
Fourier Chebyshev
iN
x iπ2
= iN
xiπcos=
periodic functions limited area [-1,1]
)sin(),cos( nxnxϕϕ
cos),cos()(
==
xnxT n
{ }
)cos(21
)sin()cos(
21)(
*
1
1
**
**0
kxa
kxbkxa
axg
m
m
k
m
kk
+
++
=
∑−
=
∑=
+=m
kkkm xTcTcxg
1
**0
* )(21)( 0
collocation points
domain
basis functions
interpolating function
30Orthogonal functions
Fourier vs. Chebyshev (cont’d)
Fourier Chebyshev
coefficients
some properties
∑
∑
=
=
=
=
N
jjj
N
jjj
kxxfN
b
kxxfN
a
k
k
1
*
1
*
)sin()(2
)cos()(2
∑=
=N
jjj kf
Nc
k1
* )cos()(cos2 ϕϕ
• Gibb’s phenomenon for discontinuous functions
• Efficient calculation via FFT
• infinite domain through periodicity
• limited area calculations
• grid densification at boundaries
• coefficients via FFT
• excellent convergence at boundaries
• Gibb’s phenomenon
31Orthogonal functions
The Fourier Transform Pair
∫
∫∞
∞−
−
∞
∞−
=
=
ωω
πω
ω
ω
deFtf
dtetfF
ti
ti
)()(
)(21)( Forward transform
Inverse transform
Note the conventions concerning the sign of the exponents and the factor.
32Orthogonal functions
Some properties of the Fourier Transform
Defining as the FT: )()( ωFtf ⇒
Linearity
Symmetry
Time shifting
Time differentiation
)()()()( 2121 ωω bFaFtbftaf +⇒+
)(2)( ωπ −⇒− Ftf
)()( ωω Fettf ti Δ⇒Δ+
)()()( ωω Fit
tf nn
n
−⇒∂
∂
33Orthogonal functions
Differentiation theorem
Time differentiation )()()( ωω Fit
tf nn
n
−⇒∂
∂
34Orthogonal functions
Convolution
∫∫∞
∞−
∞
∞−
−=−=∗ ')'()'(')'()'()()( dttgttfdtttgtftgtf
The convolution operation is at the heart of linear systems.
Definition:
Properties: )()()()( tftgtgtf ∗=∗
)()()( tfttf =∂∗
∫=∗ dttftHtf )()()(
H(t) is the Heaviside function:
35Orthogonal functions
The convolution theorem
A convolution in the time domain corresponds to a multiplication in the frequency domain.
… and vice versa …
a convolution in the frequency domain corresponds to a multiplication in the time domain
)()()()( ωω GFtgtf ⇒∗
)()()()( ωω GFtgtf ∗⇒
The first relation is of tremendous practical implication!
36Orthogonal functions
Summary
The Fourier Transform can be derived from the problem of approximating an arbitrary function.
A regular set of points allows exact interpolation (or derivation) of arbitrary functions
There are other basis functions (e.g., Chebyshev polynomials, Legendre polynomials) with similar properties
These properties are the basis for the success of the spectralelement method