Introduction

Contents

1 Introduction to Adaptive Filters 21.1 Motivation for Adaptive Filtering . . . . . . . . . 4

1.2 Overview of Adaptive Filtering Principles . . . . . 7

1.2.1 Advantages and Features of Adaptive Filters 8

1.2.2 Adaptive Filter Structures . . . . . . . . . 11

1.2.3 Adaptation Approaches . . . . . . . . . . 17

1.3 Applications of Adaptive Filters . . . . . . . . . . 19

1.3.1 Modelling or System Identification . . . . 19

1.3.2 Inverse Modelling . . . . . . . . . . . . . 21

1.3.3 Linear Prediction . . . . . . . . . . . . . 22

1.3.4 Interference Cancellation . . . . . . . . . 23

1.4 Concluding Remarks . . . . . . . . . . . . . . . 25

1.5 Learning Objectives of this Module . . . . . . . . 26

1.6 Conventions and Notations . . . . . . . . . . . . 28

1

Chapter 1

Introduction to Adaptive Filters

In this chapter, a brief and introductory overview on the topic

of adaptive filtering is given. In the context of signal processing,

filter is a device that is designed to process the signals at its

input in a specified manner and generate output signals that

meet certain well defined objectives. For example, a filter may

be designed

to extract information about specific quantities of interestfrom noisy data (e.g. enhancing a desired signal buried

in noise, recovering the transmitted data bits from received

noisy signal etc),

to aid in tracking dynamic physical processes (e.g. missileguidance, jamming the radar of an enemy vehicle etc),

to aid in medical diagnosis by analyzing the measured bio-medical signals,

to find efficient representation of signals (e.g. compressionof speech and image data), etc.

2

EE5301 Chap -1: Introduction to Adaptive Filters; S. Puthusserypady 3

In general, adaptive filtering refers to the process by which

a filter tries to adjust its parameters so as to respond, with some

specified objectives, to the changes that are taking place in its

surroundings. The changes and/or state of the surroundings may

or may not be directly measurable. That is why an adaptive

filter is employed to learn these changes. Therefore, the study of

adaptive filters involves several aspects as listed below.

Filter Structure: The type of filters used (e.g. non-recursivefilters, recursive filters etc.)

Task of the Filter: The function to be fulfilled by the filter(e.g. channel equalization, channel identification, interefer-

ence cancellation etc.)

Adaptation Algorithm: The steps specified for doing theadaptation

Performance Measures: The measures that can be used toassess the quality of the adaptation process (e.g. speed of

convergence, accuracy etc.)

Analysis: Theoretical analysis of the adaptation process andperformance so as to aid in the selection as well as design of

the adaptive algorithm.

In this module on adaptive signal processing, we will touch upon

all the above aspects and the application of adaptive filters to

real-world problems.


( )H z( )s n

( )y n( )W z( )x n

Equalizer Channel

( )v n( )s n

Threshold detector

Figure 1.1: A baseband transmission and reception system with fixedequalizer.

1.1 Motivation for Adaptive Filtering

Consider the baseband data transmission and reception system

shown in Figure 1.1. Here, the input s(n) denotes the trans-

mitted data symbols, H(z) denotes the channel transfer func-

tion and v(n) denotes the channel noise. A simple reception

scheme, consisting of an equalizer W (z) and a threshold detec-

tor, is considered. As the transmitted symbols pass through the

channel, they are subjected to several distortions. The received

signal x(n) is a noisy version of the distorted symbols. The role

of the equalizer is to compensate for the distortions introduced

by the channel and noise on the data symbols, in such a way that

the threshold detector should be able to satisfactorily recover the

data symbols.

To design an equalizer to achieve the above objective, we

could resort to a one-time effort by setting up the mean squared

error (MSE) at the equalizer output as a cost function and solving

for the optimum equalizer that minimizes the MSE. However, as

we will see later, the equation for the optimum equalizer involves


certain statistics (e.g. auto-correlation and cross-correlation) of

the underlying signals. To compute these statistics, we need to

know the channel characteristics. If these quantities are not avail-

able, then we will need to use time-averaging on the measured

signals (i.e. channel input, equalizer input, desired signal) to

estimate the required statistics. This can be very costly in terms

of computations, memory and time. Furthermore, if the channel

characteristics are time-varying, then we will need to repeat the

above computations very often so as to respond to the changes

in the channel characteristics. On the other hand, the adaptive

equalization approach offers a very attractive alternative way for

getting the (near) optimum filter settings.

( ) ( )= d n s n

( )e n

( )H z( )s n ( )y n( )W z( )x nEqualizer Channel

( )v n( )s n

Threshold detector

Training sequence

Figure 1.2: A baseband transmission and reception system with adaptiveequalizer.

The adaptive equalization set-up is shown in Figure 1.2. A

training sequence, which is also known to the receiver, is usu-

ally transmitted at the start of transmission. Ideally, the equal-

izer output is expected to be equal to the transmitted symbols.


Because the training sequence is known to the receiver, we can

generate an error signal that indicates the amount of discrepancy

between the actual output of the equalizer and the desired out-

put. This error signal can in turn be used to adjust (or, adapt)

the equalizer parameters to reduce the error in some specified

sense (e.g. MSE). At the end of the training mode, the equal-

izer parameters would have converged close to their optimal val-

ues. At this time, the detected symbols (or, decisions) at the

threshold detector output can be considered very reliable. From

then onwards, the decisions can be used to continue adapting

the equalizer. This is called the decision directed mode. This is

helpful to fine-tune the equalizer and to track slow changes in

the channel characteristics.

In the above process of adapting the equalizer parameters,

what is equivalently happening is that we are indirectly acquiring

some knowledge about the channel characteristics. This is the

main idea behind adaptive filtering, i.e. to learn about a system

by observing the changes taking place in the response of the

system to known (partially or fully) inputs and then to reflect

this knowledge in adjusting the parameters of a filter to achieve

certain desired objectives.


( )e n

Adaptive filter

( )x n ( )y n( )d n

input signal output signal

desired signal

error signal

Figure 1.3: Schematic diagram showing the main components of an adap-tive filtering set-up.

1.2 Overview of Adaptive Filtering Principles

The main principles of adaptive filtering can be described by con-

sidering the schematic shown in Figure 1.3. In a nut-shell, the

purpose of the adaptive filter is to produce an output signal,

based on a given input signal, that very closely matches a spec-

ified desired signal. The match between the desired signal and

filter output is quantified by a performance function (also called,

cost function) defined using the error signal. An adaptation

algorithm, which adjusts the filter coefficients, is developed to

optimize the given performance function in an iterative manner,

e.g. minimize the mean square value of the error. The mean

square error (MSE) is one of the most widely used performance

functions.


1.2.1 Advantages and Features of Adaptive Filters

The example of adaptive equalization discussed in Section 1.1

answers the following questions:

i) Why choose the adaptive approach ?

ii) Why do not we solve for the optimum filter coefficients by a

one-time optimization of the selected cost function ?

In an attempt to give a more complete picture of adaptive filters,

we list below some of their advantages and features.

Advantages of Adaptive Filters:1. Memory and Computational Requirements: Estimation of

the required statistics for direct computation of the optimum

filter parameters requires accumulation of large amounts of

signal samples and doing averaging on these large data se-

quences. Further, this approach also results in large delay in

the filter output.

On the other hand, adaptive approaches do not need to

accumulate signal samples, thereby resulting in large savings

in memory and computational costs. Furthermore, they do

not introduce any significant delay in the filter output.

2. Self-Optimization: Adaptive filters are self-optimizing sys-

tems. That is, they can adapt themselves to an existing

environment to achieve near-optimum performance.

3. Time-Varying Environment: Adaptive filters can adapt their


parameters to provide near-optimum performance even when

the environment parameters are (slowly) time-varying.

4. Graceful Degradation: Adaptive filters can repair themselves

to certain extent. That is, if some parts of an adaptive

filter are damaged, the remaining parts can (to some extent)

automatically compensate for the damaged parts. In other

words, they exhibit graceful degradation in performance.

On ther other hand, non-adaptive systems almost collapse if

some parts are damaged.

5. Ease of Design: Adaptive filters help to relieve the designer

from having to collect accurate knowledge of the environ-

ment (e.g. channel characteristics).

6. Implementation: Adaptive algorithms are usually much sim-

pler to code in software and DSP (digital signal processing)

processors, or to implement in hardware, compared to non-

adaptive approaches.

General Features of Adaptive Filters:1. Adaptive filters require an initial training effort. However,

there are also blind adaptive algorithms that relax this re-

quirement to varying degrees1.1In digital communication systems, the study of blind adaptation

approaches is receiving more attention currently since blind schemesresult in efficient use of the available bandwidth for transmitting theuser data (rather than training+user data).


2. Strictly speaking, adaptive filters are non-linear because the

filter parameters are data-dependent. As a result, an adap-

tive filter does not obey the principle of superposition even

if the filter structure is that of a linear filter.

3. Outputs of adaptive filters are non-stationary in nature since

the filter parameters are changing with time.

Selection of Adaptive Algorithms: In the literature, there ex-ists a wide variety of adaptive algorithms. For any particular

application, we need to wisely choose the adaptive algorithm to

use. In general, examining the following factors help to decide

on the choice of one algorithm above another.

1. Convergence speed: It is the number of iterations required

for the algorithm to converge close enough to the optimum

filter parameters.

2. Mis-adjustment: It is a quantitative measure of the amount

by which the final value of the MSE deviates from the mini-

mum MSE.

3. Tracking: It is the ability of the algorithm to track varia-

tions in the environment. Tracking performance involves the

trade-off between convergence speed and accuracy.

4. Robustness: It refers to the ability of the algorithm to op-

erate satisfactorily in the presence of small disturbances (ex-

ternal or internal).


5. Complexity: It refers to the number of computations and

memory elements required for completing each iteration of

the adaptive algorithm.

6. Structure: This refers to the structure of information flow in

the algorithm. This has great impact on efficient hardware

implementation and achievable speed of adaptive algorithms.

7. Numerical Properties: This refers to the behaviour of the

algorithm under finite-precision implementation (i.e. quan-

tization erors) and ill-conditioned input data. We prefer to

have a numerically robust adaptive algorithm.

1.2.2 Adaptive Filter Structures

In signal processing applications, the most commonly used struc-

tures for implementing digital (discrete-time) adaptive filtering

are transversal filter, linear combiner, lattice predictor, and re-

cursive filter.

Transversal Filter: A transversal filter with N coefficients(taps or weights) is shown in Figure 1.4. The input x(n) passes

through a series of identical delay elements whose delay is matched

to the arrival rate of the input samples. The filter output y(n) is

a linear combination of the outputs of the delay elements, and is


1z

( )x n

( )y n

0w

1w

1z1z

1Nw

2w

( 1)x n ( 1)x n N +( 2)x n

Figure 1.4: A transversal filter (or, tapped-delay line filter) with N taps.

given by2

y(n) =N1i=0

wix(n i).For this reason, it is also known as a tapped-delay line filter.

The transfer function of this filter is given by

H(z) =Y (z)

X(z)=

N1i=0

wizi

where Y (z) and X(z) are the z-transforms of y(n) and

x(n), respectively.

Linear Combiner: In some applications (e.g. beamforming),the tap inputs may not be delayed samples of a single input. In

such cases, the linear combiner structure shown in Figure 1.5 is2In adaptive filters, the filter coefficients should be denoted as

w0(n), w1(n), , wN1(n) to express the fact that the coefficients arechanging with time due to adaptation. For the sake of convenience, wedo not explicitly show this in the filter structures shown in Figures 1.4to 1.7.


( )y n

0( )x n

0w

1w

1Nw

2w

1( )x n 1( )Nx n2( )x n

Figure 1.5: A linear combiner with N taps.

used. Its output y(n) is a linear combination of the different

input signals, and is given by

y(n) =N1i=0

wixi(n).

Note that the linear combiner becomes a transversal filter if we

choose xi(n) = x(n i), i = 0, 1, . . . , N 1. The compu-tational complexities of linear combiner and transversal filter are

equal and is given by N multiplications and N 1 additions.

( )Nf n

( )Nb n 1( )Nb n

1( )Nf n 2( )f n

1( )b n

1( )f n

0( )b n

( )x n

0( )f n 1

2 N

2( )b n

Figure 1.6: A lattice predictor of order N .

Lattice Predictor: The lattice predictor is known for its mod-ular structure. A lattice predictor of order N consists of

N stages as shown in Figure 1.6. Each of the stages has two


inputs and two outputs. The mth stage is described by a pair

of input-output relations given by

fm(n) = fm1(n) mbm1(n 1)bm(n) = bm1(n 1) mfm1(n)

for m = 1, 2, . . . , N , with f0(n) = b0(n) = x(n). Its compu-

tational complexity is 2N multiplications and 2N additions.

( )x n

1z

( 1)x n1z

( 2)x n

1z

( )b

x n N

1b

bN

b

2b

( )y n

1z

1z

( 1)y n1a

1z

( 2)y n2a

aNa

( )ay n N

0b

Figure 1.7: The structure of an IIR filter with Nb zeros and Na poles.

Recursive Filter: The structures in Figures 1.4, 1.5 and 1.6correspond to non-recursive filters, i.e. computation of the filter

output does not involve any feedback or recursive mechanism.

Therefore, they are also called finite impulse response (FIR) fil-

ters since their impulse responses are of finite duration in time.


Further, the number of coefficients in these filters specify the

effective length of the impulse response of these filters. Conse-

quently, it is impossible to use these filters to implement impulse

responses that are very long, due to complexity.

On the other hand, impulse responses of infinite duration

can be implemented using filters with finite number of parame-

ters by incorporating feedback in the computation of the filter

output. The structure of the resulting filter, called infinite im-

pulse response (IIR) filter, is shown in Figure 1.7. Its output is

given by the recursive equation

y(n) =Nbi=0

bix(n i) +Nai=1

aiy(n i)

where a1, a2, , aNa and b0, b1, , bNb are coefficients ofthe feedback and feedforward paths, respectively. The transfer

function of this filter is given by

H(z) =Y (z)

X(z)=

Nbi=0

bizi

1Nai=1

aizi.

Thus, H(z) has Nb zeros (i.e. roots of the numerator) and

Na poles (i.e. roots of the denominator). One of the main

advantages of IIR filters is the computational simplicity in im-

plementing infinite (very long) impulse responses. Its computa-

tional complexity is Na+Nb+1 multiplications and Na+Nb ad-


ditions. The IIR structure becomes a FIR structure if Na = 0.

Unlike FIR filters, stability is an important concern in IIR

filters. This makes adaptation of IIR filters more difficult com-

pared to FIR filters. To gurantee stability, all the poles of the

filter must lie within the unit circle |z| = 1. Another difficultyassociated with adaptive IIR filters is that the MSE performance

function associated with these filters may have local optima due

to their recursive nature.

Systolic Array: The systolic array structure is developed forparallel computing applications, and is suitable for implement-

ing linear algebraic computations such as matrix multiplication,

triangularization, matrix transformation etc. The use of systolic

arrays help to achieve high throughput rates, which are required

in several real-time applications of advanced signal processing.

Non-linear Filters: The filter structures considered above arelinear in nature. In situations where the environment or system to

be modeled behaves non-linearly, we need to use non-linear filter

structures. Volterra filters and neural networks are examples of

non-linear filter structures.

Real and Complex Filters: In some applications, the filterinput and desired signal are complex-valued rather than real-

valued. For example, the baseband signal corresponding to a


QAM (quadrature amplitude modulation) modulated signal is

complex-valued. In processing such signals, we need to use filters

with complex-valued coefficients.

1.2.3 Adaptation Approaches

The various adaptation approaches can be broadly classified to

belong to either stochastic or deterministic frameworks depend-

ing upon if the underlying cost function is a statistical average

or time-average. The most popularly used cost function in the

stochastic framework is mean square error (MSE) and the re-

sulting filters are called Wiener filters. The most popularly used

cost function in the deterministic framework is the weighted sum

of squared errors and the resulting approaches are called least

squares approaches (LS).

Wiener Filter Approach: The optimum coefficients of theWiener filter are obtained by minimizing the MSE cost function.

Clearly, construction and minimization of this cost function re-

quires certain statistics which need to be obtained by ensemble

averaging. The adaptive approach to Wiener filters is to replace

these statistics by easily obtainable estimates. For example, the

well known LMS (least mean square) adaptive algorithm is ob-

tained when the MSE E[e2(n)] (E[] denotes statistical expec-tation operator) is replaced by the instantaneous squared error

e2(n).


Transform domain adaptive filtering approaches help toachieve faster convergence speed. The adaptation is done in

transform domain, which is so selected as to result in faster

convergence of the algorithm in this domain compared to

regular time domain implementation.

Block adaptation approaches help to achieve reduction incomputational requirements when the number of filter coef-

ficients is very large. The adaptation is done only once every

block. By making use of FFT (fast Fourier transform), this

can be done efficiently.

Least Squares Approach: In this case, the optimum filter co-efficients are obtained by minimizing the sum of weighted squared

errors for a given data. This deterministic approach results in al-

gorithms that may, in general, converge faster than MSE-based

algorithms. However, these algorithms are generally more com-

putationally demanding and numerically sensitive compared to

the MSE-based algorithms. There are three classes of recursive

least squares (RLS) adaptive algorithms.

Standard RLS Algorithm: Its complexity is proportional toN 2 where N is the number of filter coefficients.

QR-Decomposition based RLS Algorithm: This version ismore robust to numerical errors. Its complexity is also pro-

portional to N 2. It can be implemented using systolic

arrays which are suitable for parallel computing.


Fast RLS Algorithms: The complexity of this family of al-gorithms is proportional to N .

1.3 Applications of Adaptive Filters

Adaptive filtering has been successfully applied in several diverse

fields such as communications, control, speech processing, bio-

medical engineering, data storage etc. In all these applications,

the role of the adaptive filter is to filter a given input signal to

match a specified desired signal. An essential difference between

the various applications is the manner in which the desired signal

is obtained. The functions of the four basic classes of adap-

tive filtering applications are modelling or system identification,

inverse modelling, linear prediction and interference cancellation.

1.3.1 Modelling or System Identification

In modelling applications, the task of the adaptive filter is to

provide a model that represents the best fit to an unknown plant.

This is illustrated in Figure 1.8. Usually, the plant output is a

noisy signal. The task of the adaptive filter is to learn the transfer

function of the plant.

Typical applications of modelling are self-tuning regulators in

control systems and adaptive channel identification in commu-

nication or storage systems. Figure 1.9 illustrates the self-tuning

regulator. The adaptive filter assumes the role of Plant Model

in this figure. The identified model parameters are used to mod-


( )d n

( )e n

( )y nAdaptive

Filter

( )x nPlant

Figure 1.8: Adaptive system modelling.

( )d n

( )x n

Design

( )u n

Regulator Parameters

Plant Model

Regulator Plant

Model Parameters

Figure 1.9: Applications of adaptive system modelling: Self-tuning regu-lator.

ify the control signal for the plant. Figure 1.10 illustrates the

adaptive channel identification system. The extracted channel

parameters can be used to design the equalizer and detector. As

mentioned earlier, after the initial training mode, the adaptation

can continue by replacing the training sequence with the symbol

decisions from the output of the detector.


( )e n

( )G z( )s n

( )y n( )W z

( )d nChannel

( )v n( )s n

Training sequence

Channel model

Equalizer &

Detector

Channel Parameters

Figure 1.10: Applications of adaptive system modelling: Adaptive channelidentification.

1.3.2 Inverse Modelling

In inverse modelling applications, the task of the adaptive filter

is to provide an inverse model of an unknown plant. This is

illustrated in Figure 1.11.

( )d n ( )e n

( )y nAdaptive Filter

( )x nPlant

Delay

( )s n

Figure 1.11: Adaptive inverse system modelling.

A typical application of inverse modelling is adaptive chan-

nel equalization in communication or storage systems. We dis-

cussed this in Section 1.1 and is illustrated in Figure 1.2. The

adaptive filter models inverse of the channel transfer function

while minimizing the channel noise. Again, after the initial train-


ing mode, the adaptation can continue by replacing the training

sequence with the symbol decisions from the detector output.

( )d n

( )e n

( )y nAdaptive Filter

( )x nDelay

( )u n

Figure 1.12: Adaptive linear prediction.

1.3.3 Linear Prediction

In linear prediction applications, the adaptive filter is meant to

provide the best prediction of the current sample of a random

signal using its past samples. This is illustrated in Figure 1.12.

The adaptive filter output is the prediction estimate of the sig-

nal u(n). Consequently, the error signal e(n) is the unpre-

dictable part of u(n). This, coupled with the concept of au-

toregressive modelling, can be used for finding parametric repre-

sentations of correlated random processes.

Typical applications of linear prediction are in speech process-

ing, signal detection and spectral estimation. For the sake of

illustration, we consider the adaptive line enhancement (ALE)

problem which is an example of adaptive signal detection. The

objective of ALE is to extract a narrowband signal from a wide-

band interference or vice-versa. An example of ALE is to clean up


the measured bio-medical signal from the 50/60 Hz power-line

interference. The required system set-up will look exactly similar

to Figure 1.12 with u(n) being the bio-medical signal mixed with

power-line interference. Because narrowband signals are highly

correlated processes compared to wideband signals, output of the

adaptive filter (or, adaptive predictor) will be a good estimation

of the narrowband component in u(n). Consequently, the error

signal e(n) will represent the cleaned-up bio-medical signal.

Primary input

Reference input

( )d n

( )e n( )y nAdaptive Filter

( )x nInterf- erence

desired signal

Figure 1.13: Adaptive interference cancellation.

1.3.4 Interference Cancellation

In interference cancellation applications, the adaptive filter is to

cancel unknown interferences contained in a primary signal. The

primary signal is a mixture of the desired signal and the interfer-

ences. A reference signal is applied to the adaptive filter input

and this signal is supposed to be very weakly correlated with the

desired signal component in the primary signal and highly cor-

related with the interference component. This is illustrated in

Figure 1.13. Thus, the output of the adaptive filter becomes a

good estimate of the interference. Subtracting this estimated in-


terference y(n) from the primary input, we get the error signal

e(n) as the cleaned-up desired signal.

reflections Microphone

Loudspeaker

Person A

Conference room

( )d n

( )e n( )y nAdaptive

Filter

transmit signal

receive signal

noise

Figure 1.14: Application of adaptive interference cancellation for acousticecho cancellation in teleconferencing systems.

Typical applications of interference cancellation are in echo

cancellation in telephone networks, active noise control and

beamforming in communications. The application of acoutic

echo cancellation in teleconferencing systems is illustrated in Fig-

ure 1.14. The adaptive filter plays the role of creating a replica

of the echo of picked up by the microphone. It does this by us-

ing the speech signal from the far-end speaker as the reference

signal.


1.4 Concluding Remarks

Throughout this module, we will be using non-recursive filterstructures for our study of adaptive algorithms.

Our studies will be mainly focused on real-valued systems.For the sake of generality and completion, we will give ex-

tensions to complex-valued systems.

We will not consider finite-precision effects and details ofhardware implementation aspects in this module.

Our study will be mainly centered around stationary signalscenarios. Non-stationary signal scenarios and tracking is-

sues will not be discussed in detail.


1.5 Learning Objectives of this Module

Formulation, design and development, and analysis ofadaptive approaches for practical problems.

Principles of iterative/adaptive approaches for optimizationof nonlinear functions:

Various algorithms (e.g. steepest descent, LMS, New-

ton, RLS)

Stochastic and deterministic approaches to optimum es-

timation

Simplified approaches to adaptive filtering

Adaptive filtering in the presence of constraints.

Analysis of iterative/adaptive algorithms: convergence behaviour (speed, stability)

accuracy

complexity

Quantification of the goodness of adaptive algorithms.

Influence of signal statistics on the performance of the algo-rithm.

Influence of applications (e.g. modelling, inverse modellingetc.) on the performance of the algorithm.

Approaches to improve convergence of adaptive algorithms(e.g. transform domain adaptive filtering).


Properties of transversal and lattice filter structures. Parametric modelling of random processes. Mathematical tools to help in algorithm developmentand analysis.

Strengthen the fundamentals of signals and systems. Acquire deeper insights on digital signal processing method-ologies, interpretation of signals etc.

Translation of mathematical results into physicalinterpretation in the study of signals and systems.


1.6 Conventions and Notations

Boldface uppercase letters denote matrices and boldface low-ercase letters denote vectors. Non-boldface lowercase letters

denote scalar quantities.

Superscripts H, T and denote Hermitian (conjugate)transposition, ordinary transposition, and complex conjuga-

tion, respectively.

Unless otherwise mentioned, the signals and systems are as-sumed to be real-valued.

E[x] denotes expectation of x and |x| denotes magnitudeof x. The operator z1 denotes one sample delay.

IM denotes the MM identity matrix. Boldface 0 denotesa vector or matrix of zeros, with appropriate dimensions.

Let A be a M M matrix with its (i, j)th elementgiven by Ai,j for i, j = 0, 1, . . . ,M 1. Then i) tr(A)stands for trace of A and is equal to

M1i=0

Ai,i, ii) diag(A)

stands for diagonal of A and it results in a diagonal matrix

whose main diagonal is same as the main diagonal of A, and

iii) det(A) stands for determinant of A. If x is a M 1vector, then diag(x) results in a M M diagonal matrixwhose main diagonal is equal to x.

All the signals and systems considered here are discrete-timein nature. The signal samples are assumed to be at the


normalized sampling period of 1 second, i.e. the sampling

frequency is 1Hz.

The integers k and n are used to denote iteration indicesand time indices, respectively. The time dependence of a

variable x is represented by x(n), meaning the value of

x at nth instant.

Impulse responses of filters are represented using subscriptedvariables. For example, hk denotes the value of the k

th co-

efficient (or weight, tap) in the discrete-time impulse re-

sponse of a filter. These coefficients are also assumed to be

at the spacing of 1 second.

The autocorrelation of a random process {x(n)} is definedas xx(m) = E [x(n)x

(nm)] for m = 0,1,2, . . ..All the random processes are assumed to be zero-mean and

wide-sense stationary, unless otherwise stated explicitly.

Similarly, cross-correlation between random processes {x(n)}and {y(n)} is defined as xy(m) = E [x(n)y(nm)]for m = 0,1,2, . . ..

The random processes {x(n)} and {d(n)} (i.e. the tapinputs and desired signal, respectively) are assumed to be

individually and jointly wide-sense stationary.

The power spectral density (PSD) of {x(n)} is defined asxx

(ej

)=

m=

xx(m)ejm, where is the normalized


frequency variable in radians (i.e. = 2piFT = 2pif and

f = FT where F is the unnormalized frequency in Hz,

T is the sampling period in Seconds, and f is the normal-

ized frequency).

Similarly, the cross-spectral density between {x(n)} and {y(n)}is defined as xy

(ej

)=

m=

xy(m)ejm.

Note: Bandwidth for real and complex signals.

Output of a filter with impulse response hi, i = 0, 1, . . . , N1, and input x(n) is computed as

y(n) = h0x(n) + h1x(n 1) + + hN1x(nN + 1).In vector notation, this can be written as y(n) = hTx(n)

where the N 1 vectors h and x(n) are given byh = [h0, h1, , hN1]T

and x(n) = [x(n), x(n 1), , x(nN + 1)]T ,respectively.

Let a = bHc where a is a complex scalar and b and care complex vectors. Then, a = bTc = cHb and |a|2 =bHc2 = bHc cHb = bH (ccH)b.

Let R denote the N N correlation matrix of the signal{x(n)}. Further, let q0,q1, ,qN1 be the N ortho-normal eigenvectors of R corresponding to the eigenvalues

0 1 N1. Then, its eigen (or, spectral)


decomposition is given by

R = QQT =N1i=0

iqiqTi (1.1)

where Q = [q0,q1, ,qN1] is a unitary matrix and =diag [0, 1, , N1] is a diagonal matrix.

The trace of R is given by the sum of its eigenvalues:tr (R) = Nxx(0) =

N1i=0

i, (1.2)

where xx(0) = E[x2(n)

]is the diagonal element of R.

The determinant of R is given by the product of its eigen-values:

det (R) =N1i=0

i. (1.3)

For any N N matrices A and B, we havedet (AB) = det(A) det(B). (1.4)

For any M N matrix A and N M matrix B, wehave

tr (AB) = tr(BA) (1.5)

(AB)T = BTAT (1.6)

(AB)H = BHAH (1.7)

(AB) = AB. (1.8)

For any N N invertible matrices A and B, we have(AB)1 = B1A1. (1.9)

Introduction

Documents

Transcript of Introduction