Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf ·...
Transcript of Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf ·...
![Page 1: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/1.jpg)
ESE 531: Digital Signal Processing
Lec 21: April 14, 2020 Adaptive Filters
Penn ESE 531 Spring 2020 – Khanna
![Page 2: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/2.jpg)
Lecture Outline
! Circular convolution as linear convolution with aliasing
! Adaptive Filters
2 Penn ESE 531 Spring 2020 – Khanna
![Page 3: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/3.jpg)
Circular Convolution
Linear Convolution with aliasing!
Penn ESE 531 Spring 2020 - Khanna 3
![Page 4: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/4.jpg)
Circular Convolution
! Circular Convolution:
For two signals of length N
4 Penn ESE 531 Spring 2020 – Khanna Adapted from M. Lustig, EECS Berkeley
![Page 5: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/5.jpg)
Compute Circular Convolution Sum
5 Penn ESE 531 Spring 2020 – Khanna Adapted from M. Lustig, EECS Berkeley
![Page 6: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/6.jpg)
Compute Circular Convolution Sum
6 Penn ESE 531 Spring 2020 – Khanna Adapted from M. Lustig, EECS Berkeley
![Page 7: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/7.jpg)
Compute Circular Convolution Sum
7 Penn ESE 531 Spring 2020 – Khanna Adapted from M. Lustig, EECS Berkeley
![Page 8: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/8.jpg)
Compute Circular Convolution Sum
8
y[0]=2
Penn ESE 531 Spring 2020 – Khanna Adapted from M. Lustig, EECS Berkeley
![Page 9: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/9.jpg)
Compute Circular Convolution Sum
9
y[0]=2 y[1]=2
Penn ESE 531 Spring 2020 – Khanna Adapted from M. Lustig, EECS Berkeley
![Page 10: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/10.jpg)
Compute Circular Convolution Sum
10
y[0]=2 y[1]=2 y[2]=3
Penn ESE 531 Spring 2020 – Khanna Adapted from M. Lustig, EECS Berkeley
![Page 11: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/11.jpg)
Compute Circular Convolution Sum
11
y[0]=2 y[1]=2 y[2]=3 y[3]=4
Penn ESE 531 Spring 2020 – Khanna Adapted from M. Lustig, EECS Berkeley
![Page 12: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/12.jpg)
Result
12
y[0]=2 y[1]=2 y[2]=3 y[3]=4
Penn ESE 531 Spring 2020 – Khanna Adapted from M. Lustig, EECS Berkeley
![Page 13: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/13.jpg)
Linear Convolution
! We start with two non-periodic sequences:
" E.g. x[n] is a signal and h[n] a filter’s impulse response
! We want to compute the linear convolution:
" y[n] is nonzero for 0 ≤ n ≤ L+P-2 with length M=L+P-1
13 Penn ESE 531 Spring 2020 – Khanna Adapted from M. Lustig, EECS Berkeley
Requires LP multiplications
![Page 14: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/14.jpg)
Linear Convolution via Circular Convolution
! Zero-pad x[n] by P-1 zeros
! Zero-pad h[n] by L-1 zeros
! Now, both sequences are length M=L+P-1
14 Penn ESE 531 Spring 2020 – Khanna Adapted from M. Lustig, EECS Berkeley
![Page 15: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/15.jpg)
Circular Conv. via Linear Conv. w/ Aliasing
! If the DTFT X(ejω) of a sequence x[n] is sampled at N frequencies ωk=2πk/N, then the resulting sequence X[k] corresponds to the periodic sequence
15 Penn ESE 531 Spring 2020 - Khanna
![Page 16: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/16.jpg)
Circular Conv. via Linear Conv. w/ Aliasing
! If the DTFT X(ejω) of a sequence x[n] is sampled at N frequencies ωk=2πk/N, then the resulting sequence X[k] corresponds to the periodic sequence
! And is the DFT of one period given as
16 Penn ESE 531 Spring 2020 - Khanna
![Page 17: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/17.jpg)
Circular Conv. via Linear Conv. w/ Aliasing
! If x[n] has length less than or equal to N, then xp[n]=x[n]
17 Penn ESE 531 Spring 2020 - Khanna
![Page 18: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/18.jpg)
Circular Conv. via Linear Conv. w/ Aliasing
! If x[n] has length less than or equal to N, then xp[n]=x[n]
! However if the length of x[n] is greater than N, this might not be true and we get aliasing in time " N-point convolution results in N-point sequence
18 Penn ESE 531 Spring 2020 - Khanna
![Page 19: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/19.jpg)
Circular Conv. via Linear Conv. w/ Aliasing
19 Penn ESE 531 Spring 2020 - Khanna
! Given two N-point sequences (x1[n] and x2[n]) and their N-point DFTs (X1[k] and X2[k])
! The N-point DFT of x3[n]=x1[n]*x2[n] is defined as
X 3[k]= X 3(ej(2πk /N ) )
![Page 20: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/20.jpg)
Circular Conv. via Linear Conv. w/ Aliasing
20 Penn ESE 531 Spring 2020 - Khanna
! Given two N-point sequences (x1[n] and x2[n]) and their N-point DFTs (X1[k] and X2[k])
! The N-point DFT of x3[n]=x1[n]*x2[n] is defined as
! And X3[k]=X1[k]#X2[k], where the inverse DFT of X3[k] is
X 3[k]= X 3(ej(2πk /N ) )
x3p[n]=!x3[n] 0 ≤ n ≤ N −1
0 else
⎧⎨⎪
⎩⎪
![Page 21: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/21.jpg)
Circular Conv. via Linear Conv. w/ Aliasing
21 Penn ESE 531 Spring 2020 - Khanna
! Given two N-point sequences (x1[n] and x2[n]) and their N-point DFTs (X1[k] and X2[k])
! The N-point DFT of x3[n]=x1[n]*x2[n] is defined as
! And X3[k]=X1[k]#X2[k], where the inverse DFT of X3[k] is
X 3[k]= X 3(ej(2πk /N ) )
![Page 22: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/22.jpg)
Circular Conv. as Linear Conv. w/ Aliasing
22 Penn ESE 531 Spring 2020 - Khanna
! Thus x3p[n]=
x1[n− rN ]* x2[n− rN ]r=−∞
∞
∑ 0 ≤ n ≤ N −1
0 else
⎧
⎨⎪
⎩⎪
x3p[n]= x1[n]⊗ x2[n]N
![Page 23: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/23.jpg)
Circular Conv. as Linear Conv. w/ Aliasing
23 Penn ESE 531 Spring 2020 - Khanna
! Thus x3p[n]=
x1[n− rN ]* x2[n− rN ]r=−∞
∞
∑ 0 ≤ n ≤ N −1
0 else
⎧
⎨⎪
⎩⎪
x3p[n]= x1[n]⊗ x2[n]N
![Page 24: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/24.jpg)
Circular Conv. as Linear Conv. w/ Aliasing
24 Penn ESE 531 Spring 2020 - Khanna
! Thus
! The N-point circular convolution is the sum of linear convolutions shifted in time by N
x3p[n]=x1[n− rN ]* x2[n− rN ]
r=−∞
∞
∑ 0 ≤ n ≤ N −1
0 else
⎧
⎨⎪
⎩⎪
x3p[n]= x1[n]⊗ x2[n]N
![Page 25: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/25.jpg)
Example 1:
! Let
! The N=L=6-point circular convolution results in
25 Penn ESE 531 Spring 2020 - Khanna
![Page 26: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/26.jpg)
Example 1:
! Let
! The N=L=6-point circular convolution results in
26 Penn ESE 531 Spring 2020 - Khanna
![Page 27: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/27.jpg)
Example 1:
! Let
! The linear convolution results in
27 Penn ESE 531 Spring 2020 - Khanna
![Page 28: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/28.jpg)
Example 1:
! Let
! The linear convolution results in
28 Penn ESE 531 Spring 2020 - Khanna
![Page 29: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/29.jpg)
Example 1:
! The sum of N-shifted linear convolutions equals the N-point circular convolution
29 Penn ESE 531 Spring 2020 - Khanna
![Page 30: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/30.jpg)
Example 1:
! The sum of N-shifted linear convolutions equals the N-point circular convolution
30 Penn ESE 531 Spring 2020 - Khanna
![Page 31: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/31.jpg)
Example 1:
! The sum of N-shifted linear convolutions equals the N-point circular convolution
31 Penn ESE 531 Spring 2020 - Khanna
![Page 32: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/32.jpg)
Example 1:
! If I want the circular convolution and linear convolution to be the same, what do I do?
32 Penn ESE 531 Spring 2020 - Khanna
![Page 33: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/33.jpg)
Example 1:
! If I want the circular convolution and linear convolution to be the same, what do I do? " Take the N=2L-point circular convolution
33 Penn ESE 531 Spring 2020 - Khanna
![Page 34: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/34.jpg)
Example 1:
! If I want the circular convolution and linear convolution to be the same, what do I do? " Take the N=2L-point circular convolution
34 Penn ESE 531 Spring 2020 - Khanna
![Page 35: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/35.jpg)
Example 2:
! Let
35 Penn ESE 531 Spring 2020 - Khanna
![Page 36: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/36.jpg)
Example 2:
! Let
! What does the L-point circular convolution look like?
36 Penn ESE 531 Spring 2020 - Khanna
Linear convolution
![Page 37: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/37.jpg)
Example 2:
! Let
! What does the L-point circular convolution look like?
37 Penn ESE 531 Spring 2020 - Khanna
Linear convolution
![Page 38: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/38.jpg)
Example 2:
! The L-shifted linear convolutions
38 Penn ESE 531 Spring 2020 - Khanna
![Page 39: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/39.jpg)
Example 2:
! The L-shifted linear convolutions
39 Penn ESE 531 Spring 2020 - Khanna
![Page 40: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/40.jpg)
Adaptive Filters
Penn ESE 531 Spring 2020 - Khanna 40
![Page 41: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/41.jpg)
Adaptive Filters
! An adaptive filter is an adjustable filter that processes in time " It adapts…
41
Adaptive Filter
Update Coefficients
x[n] y[n]
d[n]
e[n]=d[n]-y[n]
+ _
Penn ESE 531 Spring 2020 - Khanna
![Page 42: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/42.jpg)
Adaptive Filter Applications
! System Identification
42 Penn ESE 531 Spring 2020 - Khanna
![Page 43: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/43.jpg)
Adaptive Filter Applications
! Identification of inverse system
43 Penn ESE 531 Spring 2020 - Khanna
![Page 44: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/44.jpg)
Adaptive Filter Applications
! Adaptive Interference Cancellation
44 Penn ESE 531 Spring 2020 - Khanna
![Page 45: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/45.jpg)
Adaptive Filter Applications
! Adaptive Prediction
45 Penn ESE 531 Spring 2020 - Khanna
![Page 46: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/46.jpg)
Stochastic Gradient Approach
! Most commonly used type of Adaptive Filters ! Define cost function as mean-squared error
" Difference between filter output and desired response
! Based on the method of steepest descent " Move towards the minimum on the error surface to get to minimum " Requires the gradient of the error surface to be known
46 Penn ESE 531 Spring 2020 - Khanna
![Page 47: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/47.jpg)
Stochastic Gradient Approach
! Most commonly used type of Adaptive Filters ! Define cost function as mean-squared error
" Difference between filter output and desired response
! Based on the method of steepest descent " Move towards the minimum on the error surface to get to minimum " Requires the gradient of the error surface to be known
47 Penn ESE 531 Spring 2020 - Khanna
![Page 48: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/48.jpg)
Least-Mean-Square (LMS) Algorithm
! The LMS Algorithm consists of two basic processes " Filtering process
" Calculate the output of FIR filter by convolving input and taps " Calculate estimation error by comparing the output to desired signal
" Adaptation process " Adjust tap weights based on the estimation error
48 Penn ESE 531 Spring 2020 - Khanna
![Page 49: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/49.jpg)
Adaptive FIR Filter: LMS
49 Penn ESE 531 Spring 2020 - Khanna
![Page 50: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/50.jpg)
Adaptive FIR Filter: LMS
50 Penn ESE 531 Spring 2020 - Khanna
![Page 51: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/51.jpg)
Adaptive FIR Filter: LMS
51 Penn ESE 531 Spring 2020 - Khanna
aTb = a1 a2 a3 a4⎡⎣⎢
⎤⎦⎥
b1b2b3b4
⎡
⎣
⎢⎢⎢⎢⎢⎢
⎤
⎦
⎥⎥⎥⎥⎥⎥
= a1b1 + a2b2 + a3b3 + a4b4
∂ aTb( )∂a
=∂ aTb( )∂a1
∂ aTb( )∂a2
∂ aTb( )∂a3
∂ aTb( )∂a4
⎡
⎣
⎢⎢⎢
⎤
⎦
⎥⎥⎥
T
=
b1b2b3b4
⎡
⎣
⎢⎢⎢⎢⎢⎢
⎤
⎦
⎥⎥⎥⎥⎥⎥
= b
![Page 52: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/52.jpg)
Adaptive FIR Filter: LMS
52 Penn ESE 531 Spring 2020 - Khanna
![Page 53: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/53.jpg)
Adaptive FIR Filter: LMS
53 Penn ESE 531 Spring 2020 - Khanna
![Page 54: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/54.jpg)
Adaptive Filter Applications
! Adaptive Interference Cancellation
54 Penn ESE 531 Spring 2020 - Khanna
![Page 55: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/55.jpg)
Adaptive Interference Cancellation
55 Penn ESE 531 Spring 2020 - Khanna
![Page 56: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/56.jpg)
Adaptive Interference Cancellation
56 Penn ESE 531 Spring 2020 - Khanna
s[n]= s[n]+w[n]− w[n]= s[n]+w[n]− hn
T !wn
![Page 57: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/57.jpg)
Adaptive Interference Cancellation
57 Penn ESE 531 Spring 2020 - Khanna
s[n]= s[n]+w[n]− w[n]= s[n]+w[n]− hn
T !wnMinimizing (ŝ[n])2 removes noise w[n]
![Page 58: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/58.jpg)
Adaptive Interference Cancellation
58 Penn ESE 531 Spring 2020 - Khanna
s[n]= s[n]+w[n]− w[n]= s[n]+w[n]− hn
T !wnMinimizing (ŝ[n])2 removes noise w[n]
s[n]( )2= s[n]+w[n]− hn
T !wn( )2
∂s2[n]∂hn
= 2 s[n]+w[n]− hnT !wn( )(− !wn )
= 2s[n](− !wn ) = −2s[n] !wn
![Page 59: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/59.jpg)
Adaptive Interference Cancellation
59 Penn ESE 531 Spring 2020 - Khanna
![Page 60: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/60.jpg)
Stability of LMS
! The LMS algorithm is convergent in the mean square if and only if the step-size parameter satisfy
! Here λmax is the largest eigenvalue of the correlation matrix
of the input data ! More practical test for stability is
! Larger values for step size
" Increases adaptation rate (faster adaptation) " Increases residual mean-squared error
60
max
20
λ<µ<
power signal input2
0 <µ<
![Page 61: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/61.jpg)
Big Ideas
61 Penn ESE 531 Spring 2020 – Khanna Adapted from M. Lustig, EECS Berkeley
! Linear vs. Circular Convolution " Use circular convolution (i.e DFT) to perform fast linear
convolution " Overlap-Add, Overlap-Save
" Circular convolution is linear convolution with aliasing
! Adaptive Filters " Use LMS algorithm to update filter coefficients " applications like system ID, channel equalization, and
signal prediction
![Page 62: Lec 21: April 14, 2020 Adaptive Filters - Penn Engineeringese531/spring2020/handouts/lec21.pdf · Lec 21: April 14, 2020 Adaptive Filters Penn ESE 531 Spring 2020 – Khanna . Lecture](https://reader033.fdocuments.in/reader033/viewer/2022051512/602fc48ead356f7d40522ed4/html5/thumbnails/62.jpg)
Admin
! Project " Out now " Work in pairs " Due 4/28
! Final Exam – Tentative 5/7 " Administered in Canvas
" 2hr timed exam in 8hr window
" Open notes " Random set of questions " Covers lec 1-20
62 Penn ESE 531 Spring 2020 – Khanna Adapted from M. Lustig, EECS Berkeley