COMM 1004: Detection & Estimation

Prof. Ahmed El-MahdyDean of the faculty of IET

The German University in Cairo

Text Books

• H.L. Van Trees, Detection, Estimation, and Linear Modulation Theory, vol. I. John Wiley& sons, New York, 2001.

• Don. H. Johnson, Statistical Signal Processing: Detection Theory, Houston, TX, 2013.

• S. Kay, Fundamentals of Statistical Signal Processing: Estimation Theory, Prentice Hall, 1993.

• S. Kay, Fundamentals of Statistical Signal Processing: Detection Theory, Prentice Hall, 1993.

Grading

• Quizzes (2Quizzes) 15%

(No Compensation Quizzes)

• Assignments 15%

• Project 30%

• Final Exam 40%

Course Contents1-Estimation Theory:

2-Detection: Simple binary hypothesis testing, likelihood ratio, Bayes criterion, Neyman-Pearson Criterion, Min-Max Performance

Parameter Estimation

random

Applications: Communication channel estimation, Range Estimation,

Sinusoidal Parameter Estimation, communication receivers, Noise Canceller

COMM 1004: Detection & Estimation

Lecture 1

- Introduction- Estimation Theory

Introduction to Detection & Estimation

Goal: Extract useful information from noisy signals

Detection: Decision between two (or a small number of) possible hypothesis to choose the best of the two hypothesis.

Parameter Estimation: Given a set of observations and given an assumed probabilistic model, we get the best estimate of the parameters of the model.

What is the detection and estimation??

Detection: example 1: digital Communications

Detection example 3: In a speaker classification problem we know the speaker is German, British, or American. There are three possible hypotheses Ho, H1, H2.

Decision: After observing the outcome in the observation space, we guess which hypothesis is true.

Examples for Estimation

Estimation of the phase of the signal:

Estimation of a DC level of a signal:

Useful in coherent modulation:

• Estimation of fading Channel:

• Parameter estimation of a signal:

Estimate h[m]???

Difference between Detection & Estimation?

Detection:

Estimation:

Try to extract a parameter from them

Estimation theory

Definitions

Parameter Estimation

random

Performance of Estimators1- Unbiased Estimators:

- For an estimator to be unbiased we mean that on the averagethe estimator will yield the true value of the unknown parameter.

- Since the parameter value may in general be anywhere in the interval , unbiasedness asserts that no matter what the true value of θ, our estimator will yield it on the average.

𝐸[ 𝜃]=𝜃

Otherwise, the estimate is said to be biased: 𝐸[ 𝜃]≠ 𝜃

a b

The bias 𝑏[𝜃] is usually considered to be additive, so that:

𝐸[ 𝜃]=𝜃 + 𝑏[𝜃].

When we have a biased estimate, the bias usually depends on the number of observations N. An estimate is said to be asymptotically unbiased if the bias tends to zero for large N: lim

𝑁→∞𝑏=0

Variance of Estimator: The variance of an estimator 𝜃 is defined as:

𝑣𝑎𝑟( 𝜃)=𝐸[( 𝜃 − 𝐸[ 𝜃])2]

Expectations are taken over x (meaning 𝜃 is random but not 𝜃).

An estimate’s variance equals the mean-squared estimation error only if the estimate is unbiased.

Performance of Estimators

Example:

Unbiased Estimators

• An estimator is unbiased does not necessarily mean that it is a good estimator. We need to Check some other performance measure.

• It only guarantees that on the average it will attain the true value.

• A continuous bias will always result in a poor estimator.

21

2-Efficiency:An unbiased estimator is said to be efficient if it has lower variance than all other estimators. Example: If we compare two unbiased estimators .

Cramer-Rao bound is a lower bound of the variance of any unbiased estimators. Then:

An estimator is said to be efficient if:

-It is unbiased-It satisfies Cramer-Rao bound.

If an efficient estimate exists, it is optimum in the mean-squared sense: No other estimate has a smaller mean-squared error.Efficiency states that the estimator is “best”

21ˆandˆ

)ˆ()ˆ(ˆthanefficientmoreisˆ2121 VarVarif

3- Consistency:

• An unbiased estimator is consistent if its variance decreases as sample size increases.

• In consistent unbiased estimator, the distribution of the estimator converges to the true value as the sample size increases.

0)ˆ(lim 1

Varn

• Consistency is a relatively weak property in contrast to optimal properties such as efficiency. Unbiased and

Consistent Estimator

Thus, a consistent estimate must be at least asymptotically unbiased.

Appendix A :Revision of Matrices

Revision of Matrices

)/1(0

0)/1()()(

:then,0

0matrixdiagonoalFor)8(

and

matrixunitarycalledisthen,if)7(

)6(

)5(

)()4(

constantfor(3)

matrixsymmetricisthen,if(2)

)1(

:C and B, A, matrices For the

2

11

2

1

1

1

b

bBbBBa

b

bB

AA

AAA

ABCCBA

ABBA

BABA

AA

AAA

AA

T

T

T

TTTT

TTT

TTT

TT

T

TT

Determinant of matrices

Inverse of matrices

There exist an inverse of the matrix A when det (A) does not equal to zero.

For the matrix A:

Eigen values and Eigen vectors of a matrix :

. of each valuefor 0)-( :solve rs,eigenvecto thedetermine To-

. of for values 0)-det(

:equation sticcharacteri thesolve s,eigenvalue thedetermine To

.eigenvaluethecalledis

.= such that , any vector isr eigenvectoan ,matrix square aGiven

vIA

IA

vAvvA

Example to find the Eigen values and vectors of a matrix :

)(7/3

7/1

3

13Repeat

5/1

5/2

512:vectoroflengthbydivideunitbetovectortheFor

.1

2then,2Assume.

5.02

1063

02

0

0

63

21:getwe,04Solving

63

21

40

04

23

234:4For

:vectorseigenthefindTo

3,4:arevalueseigentheThen

0340120-2-3

2-3

0)-det(:isequation sticcharacteriThe

23

23:matrixtheofvectorseigeningcorrespondtheandvalueseigentheFind

222

1

22

1

111

11

11

111121211

1211

12

11

1

11

21

2

vectorunitfor

vv

vvvvv

vv

v

v

VV

VV

VV

VIA

IAIA

IA

A

Appendix B :Revision of Random Variables

Revision of Random Variables

Mean of a Random Variable

Covariance of a Random Variable

Independence and Uncorrelation

)()......()(

)(),.....,,(:variablesrandomtindependenFor

21

1

21

N

N

i

iN

xpxpxp

xpxxxpN

Remember: Two Statistically Independent Random Variables

)()()( YEXEXYE

)()()( YVarXVarYXVar

If X and Y are statistically independent, then

LMMSE