M513 Tutorial 4 Sols

download M513 Tutorial 4 Sols

of 20

Transcript of M513 Tutorial 4 Sols

  • 8/3/2019 M513 Tutorial 4 Sols

    1/20

    Department of Electronic and Computer EngineeringM513 Advanced DSP Techniques

    [email protected]

    December 2010

    Tutorial Sheet 4 Optimal and Adaptive Filters

    Question 1.

    Consider a least squares method to desing an IIR filter to approximate some ideal impulse

    response hd(n). Prove that in order to design this filter, a set of equations given below needs to besolved:

    ( ) ( )1

    for 0,1,...,

    0 for 1,...,

    pn

    d k d

    k

    b n qh n a h n k

    n q q p=

    =+ =

    = + +

    where akand bkrepresent IIR filter coefficients to be determined in order to design the filter.

    Solution:

    A general difference equation for the IIR filter can be expressed as:

    ( ) ( ) ( )0 1

    q p

    k k

    k k

    y n b x n k a y n k = =

    =

    The transfer function of this filter can be obtained by applying the z-transform to abovedifference equation:

    ( ) ( ) ( )

    ( ) ( )

    ( )( )( )

    ( )( )

    0 1

    1 0

    0

    1

    1

    1

    q pk k

    k k

    k k

    p qk k

    k k

    k k

    qk

    k

    k

    pk

    kk

    Y z b z X z a z Y z

    Y z a z X z b z

    b zY z B z

    H z X z A z

    a z

    = =

    = =

    =

    =

    =

    + =

    = = =

    +

    where A(z) and B(z) are two polynomials involving all akand bkfilter coefficients.

    Note that ifh(n) represents the impulse response of the IIR filter we also have:

    ( ) ( ) 00

    1

    1

    qk

    kn k

    pkn

    k

    k

    b z

    H z h n z

    a z

    =

    =

    =

    = =

    +

  • 8/3/2019 M513 Tutorial 4 Sols

    2/20

    Department of Electronic and Computer EngineeringM513 Advanced DSP Techniques

    [email protected]

    December 2010

    To get the required set of equations, we can write ( ) ( ) ( )H z B z A z= as:

    ( ) ( ) ( )A z H z B z= Recognising that the multiplication on the left side of the equation corresponds to convolution in

    time domain, we can revert back to time domain as:

    ( ) ( ) ( )1

    p

    n k n

    k

    a h n h n a h n k b=

    = + =

    We have a set ofp+q+1 unknowns to be determined, so we need p+q+1 linear equations to

    achieve this. This can be done by setting h(n)=hd(n) for n=0,1,, p+q+1. This results in thefollowing set of equations:

    ( ) ( )1

    p

    d k d n

    k

    h n a h n k b=

    + =

    Noting that bn is a finite length sequence, i.e. that bn=0 for nq the above can also be

    written as:

    ( ) ( )1

    for 0,1,...,

    0 for 1,...,

    pn

    d k d

    k

    b n qh n a h n k

    n q q p=

    =+ =

    = + +

  • 8/3/2019 M513 Tutorial 4 Sols

    3/20

    Department of Electronic and Computer EngineeringM513 Advanced DSP Techniques

    [email protected]

    December 2010

    Question 2:

    Prony method solves the problem of IIR filter design described in the previous question in two

    steps. In the first step, coefficients akare found by minimizing the square error defined as:

    ( )2

    1n q

    e n

    = +

    =

    where e(n) represents the design error on the impulse response segment where n>q+1, i.e. since:

    ( ) ( )1

    0 for 1p

    d k d

    k

    h n a h n k n q=

    + = > +

    we have: ( ) ( ) ( )1

    p

    d k d

    k

    e n h n a h n k =

    = +

    Derive the set of equations used to obtain coefficients akusing this approach.

    Solution:

    To find the coefficients ak, according to this approach, we differentiate with respect to each ak,and set the derivatives equal to zero as follows:

    ( )( )

    ( )( )

    ( ) ( )

    ( ) ( )

    1

    1

    1

    1

    1

    2

    2

    2

    2

    0

    n qk k

    p

    k d

    k

    n q k

    d

    n q

    d

    n q

    e ne n

    a a

    a h n k

    e na

    e n h n k

    e n h n k

    = +

    =

    = +

    = +

    = +

    =

    =

    =

    =

    Dividing by two and substituting for e(n) we have

    ( ) ( ) ( ) ( ) ( )1 1 1

    0p

    d d k d d

    n q n q l

    e n h n k h n a h n l h n k

    = + = + =

    = + =

    i.e.

  • 8/3/2019 M513 Tutorial 4 Sols

    4/20

    Department of Electronic and Computer EngineeringM513 Advanced DSP Techniques

    [email protected]

    December 2010

    ( ) ( ) ( ) ( )

    ( ) ( ) ( ) ( )

    ( ) ( )

    1 1 1

    1 1 1

    1

    p

    k d d d d

    n q l n q

    p

    k d d d d

    l n q n q

    p

    k h h

    l

    a h n l h n k h n h n k

    a h n l h n k h n h n k

    a r l k r k

    = + = = +

    = = + = +

    =

    =

    =

    =

    where ( ) ( ) ( )1

    h d d

    n q

    r l k h n l h n k

    = +

    = and ( ) ( ) ( )1

    h d d

    n q

    r k h n h n k

    = +

    = are autocorrelations

    ofhd(n).

    Set of equations given with: ( ) ( )1

    p

    k h h

    l

    a r l k r k =

    = for k=1,2,p is also known as Prony

    normal equations. Using the symmetrical property of autocorrelation sequence, matrix form of

    those equations is:

    ( ) ( ) ( )

    ( ) ( )

    ( ) ( ) ( )

    ( )

    ( )

    ( )

    1

    2

    0 1 ... 1 1

    1 0 2

    ...... ...

    1 2 0

    d d d d

    d d d

    pd d d d

    ar r r p r

    ar r r

    ar p r p r r p

    =

    or in the more compact form:

    =dd ddR a r

    After determining coefficients akfrom the above system of equations, coefficients bkcan then be

    determined using:

    ( ) ( )1

    for 0,1,...,p

    d k d n

    k

    h n a h n k b n q=

    + = =

    (Note that the only unknowns in the above q equations are q coefficients bk, and they can

    therefore be easily determined.)

  • 8/3/2019 M513 Tutorial 4 Sols

    5/20

    Department of Electronic and Computer EngineeringM513 Advanced DSP Techniques

    [email protected]

    December 2010

    Question 3.

    We would like to build a predictor of digital waveforms. Such a system would form and estimate

    of a later sample (say n0 samples later) by observingp consecutive data samples. Thus we wouldset:

    ( ) ( ) ( )01

    p

    p

    k

    x n n a k x n k =

    + =

    The predictor coefficients ap(k) are to be chosen to minimise the cost function defined as

    ( ) ( )2

    0 0

    0

    p

    n

    x n n x n n

    =

    = + +

    a) Determine the equations that define the optimum set of coefficients ap(k).b) Ifn0=0, how is your formulation of this problem different from Pronys method?

    Solution:

    a) We want to find the predictor coefficients ap(k) that minimise the linear prediction error( )

    2

    0

    p

    n

    e n

    =

    =

    where ( ) ( ) ( )0 0e n x n n x n n= + +

    To find the coefficients, differentiate p with respect to ap(k), and set the derivatives equalto zero as follows:

    ( ) ( )( )

    ( )0

    0

    2 0p

    np p

    x n n

    e na k a k

    =

    +

    = = Since

    ( ) ( ) ( )01

    p

    p

    k

    x n n a k x n k =

    + =

    then

    ( )

    ( )( )0

    p

    x n nx n k

    a k

    +=

    so we have

    ( ) ( ) ( )02 0p

    np e n x n k a k

    =

    = =

    Dividing by two, and substituting for e(n) we have

    ( ) ( ) ( ) ( )00 1

    0 ; 1,2,...,p

    p

    n k

    x n n a l x n l x n k k p

    = =

    + = = =

    Therefore the normal equations are:

  • 8/3/2019 M513 Tutorial 4 Sols

    6/20

    Department of Electronic and Computer EngineeringM513 Advanced DSP Techniques

    [email protected]

    December 2010

    ( ) ( ) ( )01

    p

    p xx xx

    k

    a l r l k r k n=

    = +

    where

    ( ) ( ) ( )0

    xx

    n

    r l k x n l x n k

    =

    =

    ( ) ( ) ( )0 00

    xx

    n

    r k n x n n x n k

    =

    + = +

    b) With n0=0 these equations are the same as the all-pole normal equations using Pronysmethod (difference in notation, h(n) instead ofx(n) makes no difference to the algorithm),

    except that the right-hand side does not have a minus sign. Therefore, the solution differs

    in sign.

  • 8/3/2019 M513 Tutorial 4 Sols

    7/20

    Department of Electronic and Computer EngineeringM513 Advanced DSP Techniques

    [email protected]

    December 2010

    Questions 4.

    Derive the Wiener-Hopf equation for the Wiener FIR filter working as a noise canceller

    according to a setup given in the figure below.

    Solution:

    In this setup, the task of Wiener filter working as a noise canceller is to estimate signal d(n) froma noise corrupted observation x(n)=d(n)+v1(n) recorded by a primary sensor. The additional

    information about the corrupting noise is obtained from a secondary sensor that is placed

    elsewhere in the noise field but away from the d(n) signal source. Although the noise v2(n)measured by the secondary source is correlated with the noise in the primary sensor signal theprocesses will not be equal. Wiener filter is designed to estimate the noise v1(n) from the signalv2(n) received by the secondary sensor. The estimate v1(n) is then subtracted from the primary

    signalx(n) to form the estimate ofd(n) which is given with:

    ( ) ( ) ( ) ( )1 d n x n v n e n= =

    Wiener filter is designed by minimising the sum of squares of error e(n) defined as:

    ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )2 2

    1 12

    2 1 2

    0 0 0 0 0

    N N

    p

    n n k n k

    e n x n w k v n k d n v n w k v n k

    = = = = =

    = = = +

    whereNrepresents the order of the FIR Wiener filter with coefficients w(0),w(1),,w(N-1).

    To minimize thep it is necessary and sufficient to obtain derivatives of p with respect to filter

    coefficients w(k) and equate them to zero:

    ( )( )

    ( )( )( )

    ( ) ( )20 0

    2 2 0p

    n n

    e ne n e n v n k

    w k w k

    = =

    = = =

    W(z)

    ( )2v n ( )1v n

    ( ) ( ) ( )1x n d n v n= +

    ( )e n -+

    signal source

    noise source

    primary

    sensor

    secondary

    sensor

  • 8/3/2019 M513 Tutorial 4 Sols

    8/20

    Department of Electronic and Computer EngineeringM513 Advanced DSP Techniques

    [email protected]

    December 2010

    Dividing the above equation by 2 and substituting the expression for e(n) we have:

    ( ) ( ) ( ) ( ) ( ) ( ) ( )

    ( ) ( )( ) ( ) ( ) ( )( ) ( )

    ( ) ( ) ( ) ( )2 1 2 2 2

    1

    2 1 2 2

    0 0 0

    1

    1 2 2 2

    0 0 0

    1

    0

    0

    N

    n n l

    N

    n k n

    N

    dv v v v v

    k

    e n v n k d n v n w k v n l v n k

    d n v n v n k w k v n l v n k

    r k r k w k r l k

    = = =

    = = =

    =

    = +

    = +

    = +

    =

    where

    ( ) ( ) ( )2 20dv

    nr k d n v n k

    == represents the cross correlation between the informationcarrying signal d(n) and noise v2(n), in general case those two sequences are uncorrelated

    so ( )2

    0dv

    r k =

    ( ) ( ) ( )1 2 1 2

    0

    v v

    n

    r k v n v n k

    =

    = represents the cross correlation between two versions of the

    noise signal shown in the figure

    ( ) ( ) ( )2 2 2 2

    0

    v v

    n

    r l k v n l v n k

    =

    = represents the autocorrelation of the v2(n) version of

    the noise signal shown in the figure

    Assuming no correlation between the noise and information carrying signals, i.e. ( )2

    0dvr k = the

    set of Wiener-Hopf equations becomes:

    ( ) ( ) ( ) ( ) ( ) ( ) ( )2 1 2 2 2 1 2 2 2

    1 1

    0 0

    0N N

    dv v v v v v v v v

    k k

    r k r k w k r l k r k w k r l k

    = =

    + = =

    i.e.

    ( ) ( ) ( )1 2 2 2

    1

    0

    N

    v v v v

    k

    r k w k r l k

    =

    = for each Wiener filter coefficient, i.e. for k=0,1,,N-1

    In the matrix form we have:1 2 2v v v

    =r R w

    From this equation set of optimal filter coefficients is easily obtained from:

    2 1 2

    1

    opt v v v

    =w R r

  • 8/3/2019 M513 Tutorial 4 Sols

    9/20

    Department of Electronic and Computer EngineeringM513 Advanced DSP Techniques

    [email protected]

    December 2010

    Notice that in the noise cancellation example, analysed in this question, ( )1 2v v

    r k sequence is not

    usually obtainable as the ( )1v n signal is not directly accessible, i.e. it is contained in the signal

    ( )x n . We can however calculate and use the cross correlation sequence ( )2xv

    r k instead of

    ( )1 2v v

    r k since for the uncorrelated signals d(n) and v2(n) we have:

    ( ) ( ) ( )

    ( ) ( )( ) ( )

    ( ) ( ) ( ) ( )

    ( ) ( )

    ( )

    2

    1 2

    2

    1 2

    2 1 2

    1 20

    xv

    v v

    r k E x n v n

    E d n v n v n

    E d n v n E v n v n

    E v n v n

    r k

    =

    = +

    = +

    = +

    =

    Question 5.

    Show how to modify the expression forp obtained in the previous question as:

    ( ) ( ) ( ) ( )2

    1

    1 2

    0 0

    N

    p

    n k

    d n v n w k v n k

    = =

    = +

    to get the cost function:

    ( ) ( ) ( ) ( ) ( )1 2 2 2

    1 1 1

    0 0 0

    . 2 N N N

    p v v v v

    k k l

    J const w k r k w l w k r l k

    = = =

    = = +

    and prove that the minimization of this cost function results in the same set of Wiener-Hopf

    equations.

    Solution:

    Another, slightly different approach to obtaining derivatives( )p

    w k

    and Wiener-Hopf equation

    is to first square the term in the brackets of the above equation and identify the autocorrelation

    and crosscorrelation terms. Assuming no correlation between the d(n) and v1(n) or v2(n) signals

    we have:

  • 8/3/2019 M513 Tutorial 4 Sols

    10/20

    Department of Electronic and Computer EngineeringM513 Advanced DSP Techniques

    [email protected]

    December 2010

    ( ) ( ) ( ) ( )

    ( )( ) ( ) ( ) ( )( )

    ( ) ( )( ) ( ) ( )

    ( ) ( ) ( ) ( )

    ( ) ( ) ( ) ( ) ( )1 1 1 2 2 2

    21

    1 2

    0 0

    2 2

    1 1

    1

    1 2

    0 0

    1 1

    2 2

    0 0

    1 1 1

    0 0 0

    2

    2

    2

    N

    p

    n k

    N

    n k

    N N

    k l

    N N N

    dd v v v v v v

    k k l

    d n v n w k v n k

    d n d n v n v n

    d n v n w k v n k

    w l v n l w k v n k

    r r w k r k w l w k r l k

    = =

    = =

    = =

    = = =

    = +

    + +

    = +

    +

    = + +

    Taking derivative of the above expression with respect to each w(k) and equating to zero, wehave:

    ( )( ) ( ) ( )

    1 2 2 2

    1

    0

    2 2 0N

    p

    v v v v

    l

    r k w l r l k w k

    =

    = + =

    i.e.

    ( ) ( ) ( )1 2 2 2

    1

    0

    N

    v v v v

    l

    r k w l r l k

    =

    = for k=0,1,,N-1

    so in the matrix form we have

    1 2 2v v v=r R w

    which is the same equation as obtained in the previous question.

  • 8/3/2019 M513 Tutorial 4 Sols

    11/20

    Department of Electronic and Computer EngineeringM513 Advanced DSP Techniques

    [email protected]

    December 2010

    Question 6.

    Consider the standard expression for the mean square error of the Wiener filter given in thevector form:

    ( )2 2 T Tdx xJ E d n = + r w w R w

    a) Starting with the Wiener-Hopf equation obtain the expression for the minimum meansquare error (MMSE), i.e.Jmin.

    b) Using the obtained expression, prove that MMSE can be rewritten in the following way:min T xJ J= + v R v

    where v represents a so called misalignment vector defined asopt= v w w .

    Solution:

    a)Jmin is the value of the mean square error for w= w optso it can be obtained by inserting the

    Wiener-Hopf equation (i.e. solution for wopt) into starting MSE equation

    ( )2min 2T T

    dx opt opt x opt J E d n = + r w w R w

    From the Wiener-Hopf equation, for the optimal set of filter weights we have:

    dx x opt =r R w

    Combining the above two equations the expression for the MMSE is obtained as:

    ( )

    ( )

    ( )

    2

    min

    2

    2

    2

    2

    T T

    dx opt opt dx

    T T

    dx opt dx opt

    T

    dx opt

    J E d n

    E d n

    E d n

    = +

    = +

    =

    r w w r

    r w r w

    r w

    b)

    Now, since ( )2 2 T Tdx xJ E d n = + r w w R w we have:

  • 8/3/2019 M513 Tutorial 4 Sols

    12/20

    Department of Electronic and Computer EngineeringM513 Advanced DSP Techniques

    [email protected]

    December 2010

    ( )

    ( ) ( )

    2

    min

    min

    min

    2

    2

    2

    2

    T T

    dx x

    T T T

    dx opt dx x

    T TT

    x opt opt x opt x

    T T T T T

    opt x opt opt x x

    J E d n

    J

    J

    J

    = +

    = + +

    = + +

    = + +

    r w w R w

    r w r w w R w

    R w w R w w w R w

    w R w w R w w R w

    Rx is symmetrical matrix, so RxT=Rx. With some further manipulations of the above expression

    we finally get to required form of the expression forJmin:

    ( )

    min

    min

    min

    min

    min

    2

    2

    T T T T T

    opt x opt opt x x

    T T T

    opt x opt opt x x

    T T T T opt x opt opt x opt x xx

    TT T T

    opt x opt xx opt opt x x

    T T T T

    opt xx opt xx opt opt xx xx

    J J

    J

    J

    J

    J

    J

    = + +

    = + +

    = + +

    = + +

    = + +

    =

    w R w w R w w R w

    w R w w R w w R w

    w R w w R w w R w w R w

    w R w R w w w R w w R w

    w R w w R w w R w w R w

    ( ) ( )

    ( ) ( )

    ( ) ( )

    ( ) ( )

    min

    min

    min

    min

    min

    T T T T

    opt x opt opt x

    T T

    opt x opt

    T T

    opt xx opt

    T

    opt xx opt

    T

    xx

    J

    J R

    J R

    J R

    +

    = +

    = +

    = +

    = +

    w w R w w w R w

    w w R w w

    w w w w

    w w w w

    v v

  • 8/3/2019 M513 Tutorial 4 Sols

    13/20

    Department of Electronic and Computer EngineeringM513 Advanced DSP Techniques

    [email protected]

    December 2010

    Question 7.

    We want to use two-coefficient Wiener filter to filter noisy data ( ) ( ) ( )x n d n v n= + . The noise

    v(n) has a zero mean value, unit variance and is uncorrelated with the desired signal d(n).

    Furthermore, assume ( ) 0.6mdr m = and ( ) ( )vr m m= . Find the following:

    a) Cross-correlation vector rdx.b) Wiener equation for this system and two coefficients of the Wiener filter,

    i.e. w0 and w1.c) Minimum mean square error value, i.e.Jmin.

    Solution:

    a) Since d(n) and v(n) are independent, WSS processes

    ( ) ( ) ( )

    ( ) ( ) ( ) ( )

    ( )

    dx

    d

    m E d n x n m

    E d n d n m E d n v n m

    m

    =

    = +

    =

    r

    r

    so( )

    ( )

    0 1

    1 0.6

    d

    dx

    d

    r

    r

    = =

    r

    b) Also,( ) ( ) ( )

    ( )( ) ( )( )

    ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )

    ( ) ( )

    ( ) ( )

    x

    d v

    m E x n x n m

    E d n v n d n m v n m

    E d n d n m E d n v n m E v n d n m E v n v n m

    m m

    =

    = + +

    = + + +

    = +

    R

    R R

    Combining the above equation x d v

    = +R R R and the Wiener filter solution, dx x opt =r R w we

    have:

    ( )dx d v opt +=r R R w

    For a two coefficients Wiener filter, the autocorrelation matrix for signal dis a 22 Toeplitz

    matrix, i.e.

    ( ) ( )

    ( ) ( )

    0 1 1 0.6

    1 0 0.6 1

    d d

    d

    d d

    r r

    r r

    = =

    R

    Autocorrelation matrix for unit variance noise v is 22 identity matrix, i.e.

  • 8/3/2019 M513 Tutorial 4 Sols

    14/20

    Department of Electronic and Computer EngineeringM513 Advanced DSP Techniques

    [email protected]

    December 2010

    2

    2

    1 00

    0 10

    v

    v

    v

    = =

    R

    The Wiener equation ( )dx d v opt +=r R R w in expanded form is therefore:

    0

    1

    2 0.6 1

    0.6 2 0.6

    w

    w

    =

    The coefficients of the Wiener filter are solutions of the above matrix equation, i.e.

    ( )1

    opt d v dx+

    =w R R r

    0

    1

    0.549 0.165 1 0.451

    0.165 0.549 0.6 0.165

    w

    w

    = =

    c) The minimum mean square errorJmin can be obtained from ( )2minT

    dx opt J E d n = r w (see

    previous question).

    Since the autocorrelation function for the signal dis ( ) ( ) ( )2 , 0 1d dE d n r n n r = = =

    ( )

    ( )

    [ ]

    2

    min

    0

    0.4511 1 0.6

    0.165

    1 0.549

    0.451

    T

    dx opt

    T

    d dx opt

    J E d n

    r

    =

    =

    =

    =

    =

    r w

    r w

  • 8/3/2019 M513 Tutorial 4 Sols

    15/20

    Department of Electronic and Computer EngineeringM513 Advanced DSP Techniques

    [email protected]

    December 2010

    Question 8.

    With reference to figure given below state, discuss and derive three methods used to find the

    MMSE (minimum mean square error) by iterative adjustment of FIR coefficients Newton,

    Steepest Descent and LMS method.

    Solution:

    Newtons method uses the gradient of the MSE surface to find the MMSE in one iteration.

    Starting from the equation for the gradient of the MSE surface:

    ( ) 2 2dx x = +r R w

    The optimum weight vector can be obtained, at the point where ( ) 0 = , i.e.

    2 2 0dx x opt

    + =r R w i.e. 1opt x dx=w R r

    We can multiply both sides of the gradient equation by 11

    2x

    R from the left-hand side to obtain:

    ( )1 11

    2x x dx opt = + = +R R r w w w

    or, after rearranging the terms in the equation:

    ( )11

    2opt x

    = w w R

    This can be easily modified into adaptive algorithm:

    ( )111

    2k k x

    + = w w R

    This equation still requires demanding calculation of 1x

    R matrix, but can cope with nonquadratic

    type MSE surfaces and nonstationarity of the signals.

    Steepest descent technique does not require the knowledge ofxR . This algorithm searches for

    the MMSE point in the direction of the negative gradient of that surface:

    FIR filter

    LMS update

    x(n) y(n)

    e(n) d(n)

    w(n)

    +-

  • 8/3/2019 M513 Tutorial 4 Sols

    16/20

    Department of Electronic and Computer EngineeringM513 Advanced DSP Techniques

    [email protected]

    December 2010

    ( )( )1k k k + = w w w

    where is a constant that regulates the step size. This equation is quite similar to the one for

    Newtons method. The only differences are that1

    2has been replaced by the user-defined

    parameter and non-existence on xR in the equation. However notice that the steepest descent

    method requires the knowledge of the performance function ( )k w for gradient computation.

    The LMS algorithm is similar to steepest descent method but does not require a priori knowledge

    of ( )k w . The gradient in this method is computed by differentiating the instantaneous squared

    error ( )2

    e n instead of mean/expected value of squared error ( )2

    E e n

    . This estimate of the

    gradient of the MSE surface is therefore obtained as:

    ( )( ) ( ) ( ) ( ) ( )( )

    ( )2

    22 2

    T

    k k

    k k

    k k k

    e n d ne n e n e ne n

    = = = =

    x ww x

    w w w

    Replacing this expression for the gradient of ( )k w into steepest descent algorithm expression

    the LMS algorithm is obtained:

    ( )( ) ( )1 2k k k k k e n + = = +w w w w x

    In this case the shape of the error surface and the calculation of the autocorrelation matrix are notrequired.

    The LMS algorithm proceeds according to the following steps:

    1. The weight vector is initialised by setting all filter coefficients to some random values.A common choice is to set all the taps to 0.

    2. A choice of step size is made. Although it is possible to determine theoretical limitsfor this step size where convergence is ensured, is usually picked by trial and error

    method.

    3. The vector of previous and current input signal samples kx is formed and the filteroutput calculated:

    ( )

    T

    k k

    y n = x w

    4. Error is calculated: ( ) ( ) ( )e n d n y n= 5. Weight vector is updated, i.e. 1k+w vector is calculated according to weight update

    equation: ( )1 2k k ke n+ = +w w x

    6. 1k k= + and algorithm jumps to step 3.

  • 8/3/2019 M513 Tutorial 4 Sols

    17/20

    Department of Electronic and Computer EngineeringM513 Advanced DSP Techniques

    [email protected]

    December 2010

    Question 9.

    a) Use diagrams and relevant equations to explain and discuss the application of adaptivefilters in system identification application.

    b) LMS based FIR adaptive filter working in the system identification configuration uses N=4.If the exact transfer function of the system to be identified is:

    ( )1

    1

    1.25 0.35

    1 0.5

    zH z

    z

    +=

    write the equations for the signals d(n) and e(n) and the update equation for each adaptive

    filter coefficient, i.e. w1(n) w1(n).

    Solution:

    a)

    Unknown system to be identified is c(n) and h(n) is a digital filter used to model c(n). The basicconcept is that the adaptive filter (model) adjusts itself, intending to cause its output to match the

    output of the unknown system. When the difference (error signal, e(n)) between the physical

    system response d(n) and adaptive model responsey(n) has been minimised, the adaptive modelreproduces unknown system or provides an approximation to it. Provided that the order of the

    adaptive filter matches that of the unknown system, unknown system is identified in an optimumsense. In actual applications, however, there will normally be additive noise present at theadaptive filter input so the filter structure will not exactly match that of the unknown system.

    Once the good model of the unknown system is obtained, DFT (on the estimated impulse

    response h(n)) can be performed in order to obtain the frequency response of the system.

    When the plant is time varying, the plant output is nonstationary. In such a situation, the adaptive

    filtering algorithm has the task of keeping the modelling error small by continually tracking timevariations of the plant dynamics.

  • 8/3/2019 M513 Tutorial 4 Sols

    18/20

    Department of Electronic and Computer EngineeringM513 Advanced DSP Techniques

    [email protected]

    December 2010

    b)

    ( )( )

    ( )

    1

    1 2

    1.5 0.3

    1 0.5 0.25

    D z zG z

    X z z z

    += =

    +

    ( ) ( ) ( ) ( )1 2 11 0.5 0.25 1.5 0.3 D z z z X z z + = +

    ( ) ( ) ( ) ( ) ( )1 1 21.5 0.3 0.5 0.25D z X z z D z z z = + + Taking the inverse z-transform we have:

    ( ) 1.5 ( ) 0.3 ( 1) 0.5 ( 1) 0.25 ( 2)d n x n x n d n d n= + + +

    The adaptive FIR filter output is:

    ( ) (0) ( ) (1) ( 1) (2) ( 2) (3) ( 3)y n w x n w x n w x n w x n= + + +

    ( ) ( ) ( )e n d n y n=

    The weights update equations are

    ( ) ( ) 2 ( ) ( )w i w i e n x n i= + for 0, ,3i =

    or

    (0) (0) 2 ( ) ( )w w e n x n= +

    (1) (1) 2 ( ) ( 1)w w e n x n= +

    (2) (2) 2 ( ) ( 2)w w e n x n= +

    (3) (3) 2 ( ) ( 3)w w e n x n= +

  • 8/3/2019 M513 Tutorial 4 Sols

    19/20

    Department of Electronic and Computer EngineeringM513 Advanced DSP Techniques

    [email protected]

    December 2010

    Question 10.

    a) Use diagrams and relevant equations to explain and discuss the application of adaptivefilters in inverse system modeling applications.

    b) LMS based FIR adaptive filter working in the inverse system modelling configuration usesN=2. The system to be inverted has an impulse response given with:

    ( ) ( ) ( )1g n n n =

    Discuss the properties of the estimated inverse model with respect to values of parameter inthe above expression.

    Solution:

    a)

    Inverse system identification configuration is very similar to system identification but the

    unknown system is now placed in series with the adaptive filter.

    Adaptive filter becomes the inverse of the unknown system when the error between the delayedreference signal and adaptive filter output e(n) gets very small. So the goal of the inverse

    identification (equalisation) procedure is to obtain the transversal filter that satisfies the equation:

    ( ) ( )C H z z z=

    As shown in the figure the process requires a delay inserted in the desired signal d(n) path to keep

    the data at the summation synchronised. Adding the delay keeps the system causal. Without thedelay element, the adaptive filter algorithm tries to match the output from the adaptive filter(y(n)) to input data (x(n)) that has not yet reached the adaptive elements because it is passing

    through the unknown system. In essence, the filter ends up trying to look ahead in time. As hard

    as it tries, the filter can never adapt: e(n) never reaches a very small value and the adaptive filternever compensates for the unknown system response. The adaptive filter therefore never provides

    a true inverse response to the unknown system. Including a delay equal or similar to the delay

    caused by the unknown system prevents this condition.

  • 8/3/2019 M513 Tutorial 4 Sols

    20/20

    Department of Electronic and Computer EngineeringM513 Advanced DSP Techniques

    [email protected]

    December 2010

    b)

    The system to be inverted has a transfer function:

    ( )

    11G z z = , i.e. zero at z = . If 1 >

    G(z) has a zero outside the unit circle, i.e. G(z) is not a minimum phase filter, i.e. causal and

    stable realisation ofG(z) is not realisable. If 1 < , G(z) is a minimum phase and G-1(z) is easily

    obtainable as:

    ( )1 11

    1G z

    z

    =

    and the inverse filter has an impulse response:

    ( ) ( )1g n u n =