Algorithms for Uncertainty Quantiﬁcation - TUM · 2018. 6. 4. · Polynomial Chaos Methods (2)...

Algorithms for Uncertainty Quantification

Lecture 6: Polynomial Chaos Approximation 1: ThePseudo-spectral Approach

ST 2018

Tobias NeckelScientific Computing in Computer ScienceTUM

Repetition of Previous Lecture

• Interpolation concepts• every continuous function can be interpolated uniquely• interpolation with Lagrange cardinal polynomials• interpolation error⇔ Lebesgue constant• uniform grid not always a good choice

• Quadrature concepts• weighted inner product spaces• orthogonal polynomials• examples: Lagrange (uniform weight), Hermite (Gaussian weight)

polynomials• Gaussian quadrature

• nodes: zeros of the underlying orthogonal polynomial• weights: integral of the Lagrange cardinal polynomial evaluated at the nodes

T. Neckel | Algorithms for Uncertainty Quantification | L6: PC approx. 1: pseudo-spectral approach | ST 2018 2

Concept of Building Block:• Time: ≈ 90 minutes• Content

• Polynomial Chaos: Basic concept & pseudo-spectral approach• Example: Damped linear oscillator

• Expected Learning Outcomes• The participants can describe the basic idea of generalized

polynomial chaos (gPC) methods and underlying reasons.• They are able to list several ways to compute the gPC coefficients

and to describe and relate the pseudo-spectral approach in thiscontext.

• They can give rough estimates for the dependency of theapproximation orders N and K .

• The participants are able to derive the formulas relating expectationand variance to the coefficients of the gPC approach.

• They can indicate the relevant changes in the concept if severalinstead of one parameters are uncertain, i.e. in the multivariatecontext.


Agenda

Topic

Methods based on polynomial chaos approximation

Content• polynomial chaos expansion• orthogonal polynomials• the pseudo-spectral approach• example: damped linear oscillator• extension to multivariate polynomials• summary


Forward Propagation of Uncertainty

stochastic input Ω

stochasticmodel f (t , ω)

stochastic output Y

Problem• assumption: f computationally expensive or available as a black-box• deterministic independent variable: t (placeholder for t , x , . . .)

What we want• a good approximation of f (or statistical moments of its output Y ) that is

cheap to evaluate


Forward Propagation of Uncertainty (2)

stochastic input Ω

stochasticmodel f (t , ω)

stochastic output Y

Which method to use?• remember: standard or improved Monte Carlo sampling

1. sample from the input distribution2. solve system for each sample3. compute statistical output properties

• slow convergence in general• computationally expensive (many samples)


Polynomial Chaos Methods

Analogy: Fourier series• series of trigonometric functions for periodic functions s(t)

s(t) =∞∑

n=0

ŝn sin(. . . )

Polynomial chaos expansion• idea: approximate f (t , ω) by series of polynomials

f (t , ω) =∞∑

n=0

f̂n(t)φn(ω)

• φn(ω) polynomials of degree n, f̂n(t) coefficients


Polynomial Chaos Methods (2)

Polynomial chaos expansion• truncate series after N terms

f (t , ω) ≈N−1∑n=0

f̂n(t)φn(ω)

• φn(ω) orthogonal• type of polynomials chosen w.r.t. input distribution ρ(ω)


Polynomial Chaos Methods – Checklist

Polynomial chaos expansion

f (t , ω) ≈N−1∑n=0

f̂n(t)φn(ω)

What we need to do• specify type of polynomials φn(ω)

• compute coefficients f̂n(t)• choose maximum order N• compute statistical properties of f (t , ω) based on this approximation


Inner Product of Functions

Remember: Scalar product of vectors• vectors in Euclidean space: a, b

< a,b >= a · b =N∑

i=1

ai bi

(Weighted) inner product of functions• Euclidean space→ Hilbert space• vectors→ functions: p(ω), q(ω)• sum→ integral with weight function ρ(ω)

< p(ω),q(ω) >ρ =∫

supp(ρ)p(ω) q(ω) ρ(ω) dω

T. Neckel | Algorithms for Uncertainty Quantification | L6: PC approx. 1: pseudo-spectral approach | ST 201810

Orthogonality

Remember: Orthogonal vectors• scalar product is zero

< a,b >= 0 ⇐⇒ a ⊥ b

Orthogonal functions• inner product is zero

< p(ω),q(ω) >ρ =∫supp(ρ)

p(ω) q(ω) ρ(ω) dω = 0 ⇐⇒ p(ω) ⊥ q(ω)


Univariate Orthogonal Polynomials

Orthogonal basis• degree 0 to N − 1: φ0, φ1, . . . , φN−1• orthogonal w.r.t. weight ρ(ω)

< φi (ω), φj (ω) >ρ =

∫φi (ω)φj (ω) ρ(ω) dω = γi δij

• Kronecker delta δij =

{1, if i = j ,0, if i 6= j .

• normalization constants γi =< φi (ω), φi (ω) >ρ


Univariate Orthogonal Polynomials (2)

Orthogonal basis

< φi (ω), φj (ω) >ρ =

∫φi (ω)φj (ω) ρ(ω) dω = γi δij

Orthonormal basis• Normalization constants are 1

φ̃i =1√γiφi

• from now on: assume φi (ω) are normalized

< φi (ω), φj (ω) >ρ= δij


Type of Polynomials


f (t , ω) ≈N−1∑n=0

f̂n(t)φn(ω)

What type of polynomials to use?• chosen w.r.t. probability distribution ρ(ω)• examples:

• Ω ∼ U(−1,1)→ Legendre polynomials• Ω ∼ N (0,1): → Hermite polynomials

• the idea can be generalized to all probability distributions


Legendre Polynomials

• orthogonal w.r.t. integral from -1 to 1 with weight function ρ(ω) = 12∫ 1−1φi (ω)φj (ω) ρ(ω) dω =

22i + 1

δij

• φ0 = 1• φ1 = ω• φ2 = 12

(3ω2 − 1

)• φ3 = 12

(5ω3 − 3ω

)• φ4 = 18

(35ω4 − 30ω2 + 3

)• ...

−1.0 −0.5 0.0 0.5 1.0

x

−1.0

−0.5

0.0

0.5

1.0

Legendre polynomials

φ0

φ1

φ2

φ3

φ4


Hermite Polynomials• orthogonal w.r.t. integral from −∞ to∞• weight function ρ(ω) = 1√

2πexp(−ω

2

2 )∫ ∞−∞

φi (ω)φj (ω) ρ(ω) dx = i! δij

• φ0 = 1• φ1 = ω• φ2 = ω2 − 1• φ3 = ω3 − 3ω• φ4 = ω4 − 6ω2 + 3• ...

−3 −2 −1 0 1 2 3

x

−8

−6

−4

−2

0

2

4

6

8

Herm

ite polynomials

φ0

φ1

φ2

φ3

φ4


Three-Term Recursion

Computation of polynomials• Stieltjes’ three-term recursion relation

φ−1(ω) ≡ 0φ0(ω) ≡ 1

φn+1(ω) = (Anω + Bn)φn(ω) − Cn φn−1(ω) n ≥ 0

• An, Bn, Cn constants (computed by recursion, depending on weight ρ viainner product)

• satisfied by all orthogonal polynomials• numerically stable method for computation of polynomials• other schemes possible, e.g. Gram-Schmidt algorithm




f (t , ω) ≈N−1∑n=0

f̂n(t)φn(ω)

What we need to do• specify type of polynomials φn(ω) X

• compute coefficients f̂n(t)• choose maximum order N• compute statistical properties of f (t , ω) based on this approximation


Computation of Coefficients

The pseudo-spectral approach• exploit orthonormality of underlying basis

N−1∑n=0

f̂n(t)φn(ω) = f (t , ω)

<

N−1∑n=0

f̂n(t)φn(ω), φm(ω) >ρ =< f (t , ω), φm(ω) >ρ

N−1∑n=0

f̂n(t) < φn(ω), φm(ω) >ρ︸︷︷︸δnm

=< f (t , ω), φm(ω) >ρ

f̂n(t) =< f (t , ω), φn(ω) >ρ

• other approaches: least squares, stochastic Galerkin (next lecture) . . .


The Pseudo-spectral Approach

Computation of coefficients

f̂n(t) =< f (t , ω), φn(ω) >ρ=∫

Ω

f (t , ω)φn(ω) ρ(ω) dω

• possible difficulties:• f (t , ω) computationally expensive• f (t , ω) available only as a black box

• solution: use quadrature• Gaussian quadrature optimal in one dimensional settings

f̂n(t) =K−1∑k=0

f (t , xk )φn(xk ) wk


The Pseudo-spectral Approach (2)

Nodes and weights• quadrature rule→ nodes, weights {xk ,wk}K−1k=0• chosen w.r.t. the input probability distribution ρ• the underlying model f (t , ω) needs be evaluated only once at xk

Algorithm 1: compute coefficientsRequire: N, K , ρ

generate polynomials φifor k = 0 to K − 1 do

generate xk , wkevaluate f (t , xk )

for n = 0 to N − 1 docompute f̂n(t) =

∑K−1k=0 f (t , xk )φn(xk ) wk



Polynomial chaos expansion and the pseudo-spectral approach

f (t , ω) ≈N−1∑n=0

f̂n(t)φn(ω)

f̂n(t) =K−1∑k=0



• compute coefficients f̂n(t) X• choose maximum order N and number of number of quadrature terms K• compute statistical properties of f (t , ω) based on this approximation


Number of Terms and Nodes

Number of quadrature nodes K• expensive: evaluate f (t , xk )• K determines computational effort

Number of expansion terms N

• f̂n(t) decay exponentially• few coefficients sufficient• low computational effort once

f (t , xk ) known• rule of thumb: use N ≈ 12 K 0 1 2 3 4 5 6 7

i

10-12

10-11

10-10

10-9

10-8

10-7

10-6

10-5

10-4

10-3

10-2

10-1

100

|ci|

coefficients of oscillator example



Polynomial chaos expansion and pseudo spectral approach

f (t , ω) ≈N−1∑n=0

f̂n(t)φn(ω)

f̂n(t) =K−1∑k=0



• compute coefficients f̂n(t) X• choose maximum order N and number of quadrature terms K X• compute statistical properties of f (t , ω) based on this approximation


Expectation and Variance

Expectation• remember: φ0(ω) ≡ 1• E [φ0(ω)φn(ω)] =< φ0(ω), φn(ω) >ρ

E [f (t , ω)] ≈ E

[N−1∑n=0

f̂n(t)φn(ω)

]

=N−1∑n=0

f̂n(t) E [1 · φn(ω)]

=N−1∑n=0

f̂n(t) E [φ0(ω)φn(ω)]︸︷︷︸=δ0n

= f̂0(t)


Expectation and Variance

Variance

Var [f (t , ω)] = E[(f (t , ω)− E [f (t , ω)])2

]≈ E

(N−1∑n=0

f̂n(t)φn(ω)− f̂0(x)

)2 = E(N−1∑

n=1

f̂n(t)φn(ω)

)2=

N−1∑n=1

f̂ 2n (t) E[φn(ω)

2]︸︷︷︸=1

=N−1∑n=1

f̂ 2n (t)

• squared sum→ sum of squares• mixed terms = 0 due to orthogonality: E [φn(ω)φm(ω)] = δnm



Polynomial approximation and the pseudo-spectral approach

f (t , ω) ≈N−1∑n=0

f̂n(t)φn(ω)

f̂n(t) =K−1∑k=0



• compute coefficients f̂n(t) X• choose maximum order N and number of quadrature terms K X• compute statistical properties of f (t , ω) based on this approximation X


Model Problem – Damped Linear Oscillator

d2ydt2 (t) + c

dydt (t) + ky(t) = f cos(ωO t)

y(0) = y0dydt (0) = y1

• c – damping coefficient• k – spring constant• f – forcing amplitude• ωO – frequency• y0 – initial position• y1 – initial velocity


Damped Linear Oscillator (2)• c = 0.100, k = 0.035, f = 0.100, ωO = 1.000, y0 = 0.500, y1 = 0.000• t ∈ [0,30]

0 5 10 15 20 25 30time

0.4

0.2

0.0

0.2

0.4

0.6

0.8

displacementvelocity


The Pseudo-spectral Approach – Example

• assume c ∼ U(0.08,0.12)

y(T , ω) ≈N−1∑n=0

ŷn(T )φn(ω)

ŷn(T ) =K−1∑k=0

y(T , xk )φn(xk ) wk

• ŷn(T ) – coefficients• φn(ω) – Legendre polynomials• xk , wk – Gauss-Legendre quadrature nodes and weights


The Pseudo-spectral Approach – Example (2)

• T = 15

Deterministic result• y(T ) = −1.51e − 01

Stochastic results – Monte Carlo sampling• 100000 samples→ E [y(T , ω)] ≈ −1.53e − 01, Var[y(T , ω)] ≈ 7.83e − 04

Stochastic results – the pseudo-spectral approach• 5 nodes→ E [y(T , ω)] ≈ −1.52e − 01, Var[y(T , ω)] ≈ 7.80e − 04


The Pseudo-spectral Approach – Example (3)

Convergence• sufficient to use small K and N = 12 K• oscillator example: stoch. Galerkin with 10 terms as reference

3 4 5 6 7

K

10-8

10-7

10-6

10-5

10-4

10-3

10-2

rela

tive e

rror

in v

ari

ance

number of quadrature nodes

2 3 4 5 6 7

N

10-8

10-7

10-6

10-5

10-4

10-3

10-2

rela

tive e

rror

in v

ari

ance

number of expansion terms (K=10)


Multivariate Polynomial Chaos Expansion

• random vector Ω consisting of independent random variables Ωi ,i = 1, . . . ,d

• multi-index n = (n1, . . . ,nd ) ∈ Nd0• multivariate polynomials: product of univariate polynomials

φn(ω) = φn1 (ω1) · · ·φnd (ωd ),< φn(ω), φm(ω) >w = δnm, δnm = δn1m1 · · · δnd md

• multivariate polynomial chaos expansion:

f (t ,ω) ≈∑

n

f̂n(t)φn(ω)

• use multivariate pseudo-spectral approach to obtain f̂n:

f̂n(t) =K−1∑k=0

f (t ,xk )φn(xk ) wk


Multivariate Polynomial Chaos Expansion (2)

• multivariate polynomial chaos expansion

f (t ,ω) ≈∑

n

f̂n(t)φn(ω)

• n typically chosen such as n1 + . . . nd ≤ N for a given N• in this situation, the number of elements of{n ∈ Nd0 : n1 + . . . nd ≤ N} =

(d+Nd

):= P

• example• if d = 2,N = 4→ P = 15• if d = 3,N = 4→ P = 35• if d = 4,N = 4→ P = 70• if d = 5,N = 4→ P = 126• ...

• computational cost: K , only indirectly related to P


Multivariate Polynomial Chaos Expansion (2)

• multivariate polynomial chaos expansion

f (t ,ω) ≈∑

n

f̂n(t)φn(ω)

• n typically chosen such as n1 + . . . nd ≤ N for a given N• in this situation, the number of elements of{n ∈ Nd0 : n1 + . . . nd ≤ N} =

(d+Nd

):= P

• example• if d = 2,N = 4→ P = 15• if d = 3,N = 4→ P = 35• if d = 4,N = 4→ P = 70• if d = 5,N = 4→ P = 126• ...

• computational cost: K , only indirectly related to PT. Neckel | Algorithms for Uncertainty Quantification | L6: PC approx. 1: pseudo-spectral approach | ST 201834

Literature

• Chapter 10 in R. C. Smith, Uncertainty Quantification – Theory,Implementation, and Applications, SIAM, 2014

• D. Xiu, Numerical Methods for Stochastic Computations – A SpectralMethod Approach, Princeton Univ. Press, 2010


Summary

Polynomial chaos methods• polynomial chaos expansion

• approximate quantity of interest by polynomial series• f (t , ω) ≈

∑N−1n=0 f̂n(t)φn(ω)

• orthogonal polynomials and polynomial chaos• inner product 0 for orthogonal polynomials• < φi (ω), φj (ω) >ρ = δij• choose polynomial type according to input distribution

• the pseudo-spectral approach• use quadrature rule to compute coefficients• f̂n ≈

∑K−1k=0 f (t , xk )φn(xk ) wk

• model problem: damped linear oscillator• multivariate polynomial chaos expansion


Repetition of Previous LectureIntroductionPolynomial Chaos - Basic ConceptsPolynomial Chaos - Orthogonal PolynomialsPolynomial Chaos - Computation of CoefficientsPolynomial Chaos - Choice of TruncationPolynomial Chaos - Computation of Statistical Moments

Example: Damped Linear OscillatorPolynomial Chaos in Higher Dimensions

Algorithms for Uncertainty Quantiﬁcation - TUM · 2018. 6. 4. · Polynomial Chaos Methods (2)...

Documents

Transcript of Algorithms for Uncertainty Quantiﬁcation - TUM · 2018. 6. 4. · Polynomial Chaos Methods (2)...