Chapter 3 Foundation of Mathematical AnalysisChapter 3 Foundation of Mathematical Analysis

§ 3.1 Statistics and Probability

§ 3.2 Random Variables and Magnitude Distribution

§ 3.3 Probability Density Functions and Usages

§ 3.4 Stochastic Processes and Representation

R. J. ChangDepartment of Mechanical Engineering

NCKU

§ § 3.1 3.1 Statistics and Probability(1)Statistics and Probability(1)

1. Introduction (1) Terminologies (a) Statistics Randomization, Outcome, Events

(b) Probability Axioms, Probability space, Random processes

(c) Experiments Sample space, Probability measure

Ex: Die experiment, Coin experiment

§ § 3.1 3.1 Statistics and Probability(2)Statistics and Probability(2)

(2)Mathematical Model

UncertainPhysical

World

MathRepresentation

StatisticalExperiment

ProbabilitySpace

Randomizationdesign and test

Probabilityaxioms

Statisticalmodeling

Probabilitymodeling

Modeling and mapping

§ § 3.1 3.1 Statistics and Probability(3)Statistics and Probability(3)

2. Probability Space

(1) Definition In probability theory, the probability space is defined as a triplet {Ω, £, P}. Ω is called the sure event which is a space of points ωi. £ is

called the sets of events and are subsets of Ω. P is called a probability measure.

§ § 3.1 3.1 Statistics and Probability(4)Statistics and Probability(4)

Ex: Coin experiment-Two consecutive tosses of fair coin Outcomes (“Points”): ω1 (H, H);

ω2 (H, T);

ω3 (T, H);

ω4 (T, T).

Tail(T)

Head(H)

§ § 3.1 3.1 Statistics and Probability(5)Statistics and Probability(5)

Specific Events: Ai : Subsets of points.

A1 : At least a tail was thrown ω2, ω3, ω4.

A2 : Exactly a tail was thrown ω2, ω3.

A3 : Exactly two tail were thrown ω4.

23

41

A1

A2 A3

§ § 3.1 3.1 Statistics and Probability(6)Statistics and Probability(6)

(2) Probability measure Define P as a function mapping P: £ R, and P satisfy the following axioms.

(a) P(A) 0, where A is an event and P(A) is called ≧ the probability of the event.

(b) P(Ω)=1.

(c) P(A B)=P(A)+P(B) provided that A,B £ and ∪ A∩B= Φ,Φ is called the impossible event.

§ § 3.1 3.1 Statistics and Probability(7)Statistics and Probability(7)

(3) Structure of Probability Space

ProbalityAxiom PP [0,1]

* Sample space (Ω )* Outcome Ω

Algebra ￡ ￡

Events

§§ 3.2 3.2 Random Variables and Magnitude Distribution (1)Random Variables and Magnitude Distribution (1)

1. Random Variables and Distribution Function (1) Random variables: In a probability space (Ω， £， P), is a random variable if and only if X is measurable w.r.t the field £.

(2) Distribution function: The function is defined as the distribution function of X(ω).  

( ) { | ( ) } ωF x P X x

: nX R

§§ 3.2 3.2 Random Variables and Magnitude Distribution (2)Random Variables and Magnitude Distribution (2)

(a) Fundamental properties of F(x) A non-decreasing function with the properties:

1

( )F x

x

( )lim x

F x( )lim x

F x

1. lim ( ) 0

2. lim ( ) 1

x

x

F x

F x→

→

0

§§ 3.2 3.2 Random Variables and Magnitude Distribution (3)Random Variables and Magnitude Distribution (3)

(b) Function Mapping

iA

Samplespace

Realization ofrandom

variables inRn

Probabilityin [0,1]

X(£s) F(x)

P(A)

§§ 3.2 3.2 Random Variables and Magnitude Distribution (4)Random Variables and Magnitude Distribution (4)

1. Probability Density Function (1) Definition p(x) is called the probability density function of x(ω) if

If F(x) is differentiable w.r.t. x then

( ) ( )

∞

xF x p y dy

( ) ( ) /p x dF x dx

§§ 3.2 3.2 Random Variables and Magnitude Distribution (5)Random Variables and Magnitude Distribution (5)

Ex: Die Experiment Outcome: Six faces of the Die Ω = { f1, f2, f3, f4, f5, f6 }

Subsets(total number): 26 Events: “Even”, Outcomes are f2 , f4 , f6

Probability=3/6=1/2

Random Variable: Define X (fi)=10 * i

Face1, X (f1)=10

Face2, X (f2)=20

Face6, X (f6)=60

…

…

§§ 3.2 3.2 Random Variables and Magnitude Distribution (6)Random Variables and Magnitude Distribution (6)

Distribution FunctionF(100) = P{ } = 1

F(35) = P{ } = 1/2

Probability Density Function—Uniform discontinuous function

F(x)

x10 20 30 40 50 60

1/6

2/6

3/6

4/6

5/6

1

10 20 30 40 50 60

1/6

p(x)

x

100

Discrete uniformdistribution

Nondifferentiable in classicalsense

| ( ) 100i if X f( ) 35i if | X f

§§ 3.3 3.3 Probability Density Functions and Usages (1)Probability Density Functions and Usages (1)

1. Densities and Distributions

Type

Uniform(bounded)

a b

1

b a

p(x)

x

p(x)

x

x

GaussianNormal

(Error function)

ProbabilityDensity p(x)

Distribution Function F(x)

F(x)

1

a b

x

F(x)

x

1

0.5

xx

§§ 3.3 3.3 Probability Density Functions and Usages (2)Probability Density Functions and Usages (2)

2. Fundamental Usages

2 2 2

2 2 2

( ) 1

( ) [ ]

( ) [ ]

( )( ) [( ) ]

( . .

Total probability :

Mean value (1st -Moment) :

Mean square (2nd-Moment) :

Variance :

domain

x

domain

x

domain

x x x

domain

p x dx

p x xdx E x

p x x dx E x

p x x dx E x

Root Mean Square R M S ) Mean square

§§ 3.3 3.3 Probability Density Functions and Usages (3)Probability Density Functions and Usages (3)Ex: Gaussian density and probability distribution

2

2

( )1( ) exp[ ]

22

Mean value

Standard deviation

x

xx

x

x

xP x

[ ] ( )

b

aP a x b p x dx Probability

a b

p(x)

x x3x 3x

p(x)

x

P xx μ2

[| | 3 ] 99 %3

§§ 3.4 3.4 Stochastic Processes and Representation (1)Stochastic Processes and Representation (1)1. Time Domain Representation (1) Stochastic processes A stochastic process X(t, ω) is a family of random

variables defined on the probability space {Ω， £，

P} and indexed by time t .

For fixed time, we obtain a random variable which is measurable w.r.t. £.

For each ω, we obtain a function mapping X: t R called a sample function.

§§ 3.4 3.4 Stochastic Processes and Representation (2)Stochastic Processes and Representation (2)

t

iX( ,t)ω Random variable

it (fixed) i+jt

1

2

3

§§ 3.4 3.4 Stochastic Processes and Representation (3)Stochastic Processes and Representation (3)(2) Gaussian process A random process is a Gaussian process if for any

finite collection of n parametric values at t1, t2…tn, the corresponding n random variables X(t1), X(t2)…X(tn) are jointly Gaussian. The probability density function of the random variables X1, X2, X3…Xn can be given by p(x1, t1; x2, t2 …xn, tn):

1 1 2 2 n n x x

x

x ,t ; x ,t ;...; x ,t exp x x

x x x x

x

xx

μ μ

μ

11

2 2

1 2

1 1( ) ( ) ( )

2(2 )

where , ,..., states as a vector space

[ ] mean vector

[ ] , covariance matrix

T

n

T Tx

p

n

E

E

§§ 3.43.4 Stochastic Processes and Representation (4) Stochastic Processes and Representation (4)

(3) Properties of Gaussian process (a) Invariant property

(b)Ergodicity property For weakly stationary (up to 2nd moment) process

Linear Time-invariant Systemx y

Input Guassian Output Guassian

1lim ( ) 0

T

xxTTR d

T

( )xxR

Envelop decay !

-T T£n

§§ 3.43.4 Stochastic Processes and Representation (5) Stochastic Processes and Representation (5)

2. Frequency Domain Representation (1) Magnitude representations (a) Amplitude spectrum (b) Energy spectrum (c) Power spectrum

(d) Power spectral density

sin i i ix x t

ix

2

0

TE x t dt

iE

20 0

1 TP x t dt

T

ioP

0s

dPP

d

isP

§§ 3.43.4 Stochastic Processes and Representation (6) Stochastic Processes and Representation (6)

(2) Mathematical and physical spectrum

t

Gxx() = 2Sxx()for

Math

xxS

xxS

x

Phy.

ln ln2 lnxx xxG S

20ln 20log2 20lnxx xxG S

Magnitude in dB:

§§ 3.43.4 Stochastic Processes and Representation (7) Stochastic Processes and Representation (7)

3. Gaussian white process (1) Definition A Gaussian process v(t) define on {Ω， £， P} is a white process if its mean and covariance functions are given by (a) E[v(t)]=0 (b) E[v(t) v(s)]=Q‧ δ(t-s) (2) Interpretation of whiteness Frequency–domain interpretation

Power spectrum is Fourier transform of autocorrelation function.

£s

constant

t

( )t

£s

( )xxS ( )xxR

£n

§§ 3.43.4 Stochastic Processes and Representation (8) Stochastic Processes and Representation (8)

(3) Role of Gaussian white process (a) Mathematics: a model of ideal random signal source (b) Physics: a model of physical noise (c) Engineering: signal for dynamic testing

Physics Math Eng.

Phenomenon Ideal Analytical Model Testing signal

£s0

( )xxS

Bandwidth

Local (Band limited) white

§§ 3.43.4 Stochastic Processes and Representation (9) Stochastic Processes and Representation (9)

Gaussian white and colored process

System

Desired coloroutput

White

Desired LTI Model(Event include nonlinearity)