Digital Audio Signal Processing Lecture-7 Kalman Filters for AEC/AFC/ANC/Noise Reduction Marc Moonen...
-
Upload
eustacia-mckinney -
Category
Documents
-
view
218 -
download
2
Transcript of Digital Audio Signal Processing Lecture-7 Kalman Filters for AEC/AFC/ANC/Noise Reduction Marc Moonen...
![Page 1: Digital Audio Signal Processing Lecture-7 Kalman Filters for AEC/AFC/ANC/Noise Reduction Marc Moonen Dept. E.E./ESAT-STADIUS, KU Leuven marc.moonen@kuleuven.be.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bfe41a28abf838cb58f6/html5/thumbnails/1.jpg)
Digital Audio Signal ProcessingLecture-7
Kalman Filtersfor AEC/AFC/ANC/Noise Reduction
Marc Moonen Dept. E.E./ESAT-STADIUS, KU Leuven
www.esat.kuleuven.be/stadius/
![Page 2: Digital Audio Signal Processing Lecture-7 Kalman Filters for AEC/AFC/ANC/Noise Reduction Marc Moonen Dept. E.E./ESAT-STADIUS, KU Leuven marc.moonen@kuleuven.be.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bfe41a28abf838cb58f6/html5/thumbnails/2.jpg)
Digital Audio Signal Processing Version 2015-2016 Lecture 7: Kalman Filters p. 2 / 30
Overview
Kalman Filters– Introduction – Least Squares Parameter Estimation– Standard Kalman Filter– Square-Root Kalman Filter
DASP Applications– AEC (and AFC/ANC/..)– Noise Reduction (Lecture-3)
![Page 3: Digital Audio Signal Processing Lecture-7 Kalman Filters for AEC/AFC/ANC/Noise Reduction Marc Moonen Dept. E.E./ESAT-STADIUS, KU Leuven marc.moonen@kuleuven.be.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bfe41a28abf838cb58f6/html5/thumbnails/3.jpg)
Digital Audio Signal Processing Version 2015-2016 Lecture 7: Kalman Filters p. 3 / 30
Introduction: Least Squares Parameter Estimation
In DSP-CIS, have introduced ‘Least Squares’ estimation as an alternative
(based on observed data/signal samples) to optimal filter design
(based on statistical information)…
![Page 4: Digital Audio Signal Processing Lecture-7 Kalman Filters for AEC/AFC/ANC/Noise Reduction Marc Moonen Dept. E.E./ESAT-STADIUS, KU Leuven marc.moonen@kuleuven.be.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bfe41a28abf838cb58f6/html5/thumbnails/4.jpg)
Digital Audio Signal Processing Version 2015-2016 Lecture 7: Kalman Filters p. 4 / 30
Introduction: Least Squares Parameter Estimation
Given… L vectors of input variables (=‘regressors’)
L corresponding observations of a dependent variable and assume a
linear regression/observation model where w0 is an unknown parameter vector (=‘regression coefficients’)
and ek is unknown additive noise
Then…
the aim is to estimate wo
Least Squares (LS) estimate is (see previous page for definitions of U and d)
‘Least Squares’ approach is also used in parameter estimation in a linear regression model, where the problem statement is as follows…
![Page 5: Digital Audio Signal Processing Lecture-7 Kalman Filters for AEC/AFC/ANC/Noise Reduction Marc Moonen Dept. E.E./ESAT-STADIUS, KU Leuven marc.moonen@kuleuven.be.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bfe41a28abf838cb58f6/html5/thumbnails/5.jpg)
Digital Audio Signal Processing Version 2015-2016 Lecture 7: Kalman Filters p. 5 / 30
Introduction: Least Squares Parameter Estimation
• If the input variables uk are given/fixed (*) and the additive noise e is a random vector with zero-mean
then the LS estimate is ‘unbiased’ i.e.
• If in addition the noise e has unit covariance matrix then the (estimation) error covariance matrix is
(*) Input variables can also be random variables, possibly correlated with the additive noise, etc... Also regression coefficients can be random variables, etc…etc... All this not considered here.
![Page 6: Digital Audio Signal Processing Lecture-7 Kalman Filters for AEC/AFC/ANC/Noise Reduction Marc Moonen Dept. E.E./ESAT-STADIUS, KU Leuven marc.moonen@kuleuven.be.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bfe41a28abf838cb58f6/html5/thumbnails/6.jpg)
Digital Audio Signal Processing Version 2015-2016 Lecture 7: Kalman Filters p. 6 / 30
Introduction: Least Squares Parameter Estimation
• The Mean Square Error (MSE) of the estimation is
PS: This MSE is different from the one in Wiener Filtering, check formulas
• Under the given assumptions, it is shown that amongst all linear estimators, i.e. estimators of the form
(=linear function of d)
the LS estimator (with Z=(UTU)-1UT and z=0) mimimizes the MSE i.e. it is the Linear Minimum MSE (MMSE) estimator
• Under the given assumptions, if furthermore e is a Gaussian distributed random vector, it is shown that the LS estimator is also the (‘general’, i.e. not restricted to ‘linear’) MMSE estimator.
Optional reading: https://en.wikipedia.org/wiki/Minimum_mean_square_error
![Page 7: Digital Audio Signal Processing Lecture-7 Kalman Filters for AEC/AFC/ANC/Noise Reduction Marc Moonen Dept. E.E./ESAT-STADIUS, KU Leuven marc.moonen@kuleuven.be.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bfe41a28abf838cb58f6/html5/thumbnails/7.jpg)
Digital Audio Signal Processing Version 2015-2016 Lecture 7: Kalman Filters p. 7 / 30
Introduction: Least Squares Parameter Estimation
• PS-1: If noise e is zero-mean with non-unit covariance matrix
where V1/2 is the lower triangular Cholesky factor (‘square root’), the Linear MMSE estimator & error covariance matrix are
which corresponds to the LS estimator for the so-called pre-whitened observation model
where the additive noise is indeed white..
Example: If V=σ2.I then w^ =(UTU)-1UT d with error covariance matrix σ2.(UTU)-1
![Page 8: Digital Audio Signal Processing Lecture-7 Kalman Filters for AEC/AFC/ANC/Noise Reduction Marc Moonen Dept. E.E./ESAT-STADIUS, KU Leuven marc.moonen@kuleuven.be.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bfe41a28abf838cb58f6/html5/thumbnails/8.jpg)
Digital Audio Signal Processing Version 2015-2016 Lecture 7: Kalman Filters p. 8 / 30
Introduction: Least Squares Parameter Estimation
• PS-2: If an initial estimate is available (e.g. from previous observations) with error covariance matrix
where P1/2 is the lower triangular Cholesky factor (‘square root’),
the Linear MMSE estimator & error covariance matrix are
which corresponds to the LS estimator for the model
Example: P-1=0 corresponds to ∞ variance of the initial estimate, i.e. back to p.7
![Page 9: Digital Audio Signal Processing Lecture-7 Kalman Filters for AEC/AFC/ANC/Noise Reduction Marc Moonen Dept. E.E./ESAT-STADIUS, KU Leuven marc.moonen@kuleuven.be.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bfe41a28abf838cb58f6/html5/thumbnails/9.jpg)
Digital Audio Signal Processing Version 2015-2016 Lecture 7: Kalman Filters p. 9 / 30
Rud
olf
Em
il K
álm
án (
1930
- )
Introduction: Least Squares Parameter Estimation
A Kalman Filter also solves a parameter estimation problem, but now the parameter vector is dynamic instead of static, i.e. changes over time
The time-evolution of the parameter vector is described by the ‘state equation’ in a state-space model, and the linear regression model of p.4 then corresponds to the ‘output equation’ of the state-space model (details in next slides..)
• In the next slides, the general formulation of the (Standard) Kalman Filter is given• In p.16 it is seen how this relates to Least Squares estimation
Kalman Filters are used everywhere! (aerospace, economics, manufacturing, instrumentation, weather forecasting, navigation, …)
![Page 10: Digital Audio Signal Processing Lecture-7 Kalman Filters for AEC/AFC/ANC/Noise Reduction Marc Moonen Dept. E.E./ESAT-STADIUS, KU Leuven marc.moonen@kuleuven.be.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bfe41a28abf838cb58f6/html5/thumbnails/10.jpg)
Digital Audio Signal Processing Version 2015-2016 Lecture 7: Kalman Filters p. 10 / 30
State space model of a time-varying discrete-time system
(PS: can also have multiple outputs)
where v[k] and w[k] are mutually uncorrelated, zero mean, white noises
Standard Kalman Filter
process noise
measurement noise
input signal vectorstate vector
output signal
state equation
output equation
![Page 11: Digital Audio Signal Processing Lecture-7 Kalman Filters for AEC/AFC/ANC/Noise Reduction Marc Moonen Dept. E.E./ESAT-STADIUS, KU Leuven marc.moonen@kuleuven.be.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bfe41a28abf838cb58f6/html5/thumbnails/11.jpg)
Digital Audio Signal Processing Version 2015-2016 Lecture 7: Kalman Filters p. 11 / 30
• Example: IIR filter
State space model is
(no process/measurement noise here)
Standard Kalman Filter
xbo
xb4
xb3
xb2
xb1
+ +++
y[k]
u[k] + +++
x-a4
x-a3
x-a2
x-a1
x1[k] x2[k] x3[k] x4[k]
![Page 12: Digital Audio Signal Processing Lecture-7 Kalman Filters for AEC/AFC/ANC/Noise Reduction Marc Moonen Dept. E.E./ESAT-STADIUS, KU Leuven marc.moonen@kuleuven.be.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bfe41a28abf838cb58f6/html5/thumbnails/12.jpg)
Digital Audio Signal Processing Version 2015-2016 Lecture 7: Kalman Filters p. 12 / 30
State estimation problem
Given… A[k], B[k], C[k], D[k], V[k], W[k], k=0,1,2,... and input/output observations u[k],y[k], k=0,1,2,...
Then… estimate the internal states x[k], k=0,1,2,...
Standard Kalman Filter
state vector
![Page 13: Digital Audio Signal Processing Lecture-7 Kalman Filters for AEC/AFC/ANC/Noise Reduction Marc Moonen Dept. E.E./ESAT-STADIUS, KU Leuven marc.moonen@kuleuven.be.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bfe41a28abf838cb58f6/html5/thumbnails/13.jpg)
Digital Audio Signal Processing Version 2015-2016 Lecture 7: Kalman Filters p. 13 / 30
Standard Kalman Filter
Definition: = Linear MMSE-estimate of xk using all available data up until time l
• `FILTERING’ = estimate
• `PREDICTION’ = estimate
• `SMOOTHING’ = estimate
PS: will use shorthand notation xk, yk ,.. instead of x[k], y[k],.. from now on
![Page 14: Digital Audio Signal Processing Lecture-7 Kalman Filters for AEC/AFC/ANC/Noise Reduction Marc Moonen Dept. E.E./ESAT-STADIUS, KU Leuven marc.moonen@kuleuven.be.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bfe41a28abf838cb58f6/html5/thumbnails/14.jpg)
Digital Audio Signal Processing Version 2015-2016 Lecture 7: Kalman Filters p. 14 / 30
The ‘Standard Kalman Filter’ (or ‘Conventional Kalman Filter’)
operation @ time k (k=0,1,2,..) is as follows:
Given a prediction of the state vector @ time k based on previous observations (up to time k-1)with corresponding error covariance matrix
Step-1: Measurement Update =Compute an improved (filtered) estimate
based on ‘output equation’ @ time k (=observation y[k])
Step-2: Time Update
=Compute a prediction of the next state vector
based on ‘state equation’
Standard Kalman Filter
![Page 15: Digital Audio Signal Processing Lecture-7 Kalman Filters for AEC/AFC/ANC/Noise Reduction Marc Moonen Dept. E.E./ESAT-STADIUS, KU Leuven marc.moonen@kuleuven.be.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bfe41a28abf838cb58f6/html5/thumbnails/15.jpg)
Digital Audio Signal Processing Version 2015-2016 Lecture 7: Kalman Filters p. 15 / 30
The ‘Standard Kalman Filter’ formulas are as follows (without proof)
Initalization:
For k=0,1,2,…Step-1: Measurement Update
Step-2: Time Update
Standard Kalman Filter
compare to standard RLS!
Try to derive this from state equation
![Page 16: Digital Audio Signal Processing Lecture-7 Kalman Filters for AEC/AFC/ANC/Noise Reduction Marc Moonen Dept. E.E./ESAT-STADIUS, KU Leuven marc.moonen@kuleuven.be.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bfe41a28abf838cb58f6/html5/thumbnails/16.jpg)
Digital Audio Signal Processing Version 2015-2016 Lecture 7: Kalman Filters p. 16 / 30
PS: ‘Standard RLS’ is a special case of ‘Standard KF’
Standard Kalman Filter
Internal state vector is FIR filter
coefficients vector, which is
assumed to be time-invariant
![Page 17: Digital Audio Signal Processing Lecture-7 Kalman Filters for AEC/AFC/ANC/Noise Reduction Marc Moonen Dept. E.E./ESAT-STADIUS, KU Leuven marc.moonen@kuleuven.be.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bfe41a28abf838cb58f6/html5/thumbnails/17.jpg)
Digital Audio Signal Processing Version 2015-2016 Lecture 7: Kalman Filters p. 17 / 30
PS: ‘Standard RLS’ is a special case of ‘Standard KF’
Standard RLS is not numerically stable (see DSP-CIS),
hence (similarly) the Standard KF is not numerically stable(i.e. finite precision implementation diverges from infinite precision implementation)
Will therefore again derive an alternative
Square-Root Algorithm which can be shown to be numerically stable
(i.e. distance between finite precision implementation and infinite precision implementation is bounded)
Standard Kalman Filter
![Page 18: Digital Audio Signal Processing Lecture-7 Kalman Filters for AEC/AFC/ANC/Noise Reduction Marc Moonen Dept. E.E./ESAT-STADIUS, KU Leuven marc.moonen@kuleuven.be.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bfe41a28abf838cb58f6/html5/thumbnails/18.jpg)
Digital Audio Signal Processing Version 2015-2016 Lecture 7: Kalman Filters p. 18 / 30
The state estimation/prediction @ time k corresponds to a parameter estimation problem in a linear regression model (p.4), where the parameter vector contains all previous state vectors…
Square-Root Kalman Filter
![Page 19: Digital Audio Signal Processing Lecture-7 Kalman Filters for AEC/AFC/ANC/Noise Reduction Marc Moonen Dept. E.E./ESAT-STADIUS, KU Leuven marc.moonen@kuleuven.be.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bfe41a28abf838cb58f6/html5/thumbnails/19.jpg)
Digital Audio Signal Processing Version 2015-2016 Lecture 7: Kalman Filters p. 19 / 30
Square-Root Kalman Filter
compare to p.7
The state estimation/prediction @ time k corresponds to a parameter estimation problem in a linear regression model (p.4), where the parameter vector contains all previous state vectors…
![Page 20: Digital Audio Signal Processing Lecture-7 Kalman Filters for AEC/AFC/ANC/Noise Reduction Marc Moonen Dept. E.E./ESAT-STADIUS, KU Leuven marc.moonen@kuleuven.be.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bfe41a28abf838cb58f6/html5/thumbnails/20.jpg)
Digital Audio Signal Processing Version 2015-2016 Lecture 7: Kalman Filters p. 20 / 30
Square-Root Kalman Filter
Linear MMSE
explain subscript
![Page 21: Digital Audio Signal Processing Lecture-7 Kalman Filters for AEC/AFC/ANC/Noise Reduction Marc Moonen Dept. E.E./ESAT-STADIUS, KU Leuven marc.moonen@kuleuven.be.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bfe41a28abf838cb58f6/html5/thumbnails/21.jpg)
Digital Audio Signal Processing Version 2015-2016 Lecture 7: Kalman Filters p. 21 / 30
Square-Root Kalman Filter
Triangular factor & right-hand side propagated from time k-1 to time k
..hence requires only lower-right/lower part ! (explain)
is then developed as follows
![Page 22: Digital Audio Signal Processing Lecture-7 Kalman Filters for AEC/AFC/ANC/Noise Reduction Marc Moonen Dept. E.E./ESAT-STADIUS, KU Leuven marc.moonen@kuleuven.be.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bfe41a28abf838cb58f6/html5/thumbnails/22.jpg)
Digital Audio Signal Processing Version 2015-2016 Lecture 7: Kalman Filters p. 22 / 30
Square-Root Kalman Filter
Propagated from
time k-1 to time k
Propagated from
time k to time k+1
backsubstitution
A recursive algorithm based on QR-updating can then be derived,
which is given as follows
PS: QRD-RLS can be shown to be a special case of this
![Page 23: Digital Audio Signal Processing Lecture-7 Kalman Filters for AEC/AFC/ANC/Noise Reduction Marc Moonen Dept. E.E./ESAT-STADIUS, KU Leuven marc.moonen@kuleuven.be.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bfe41a28abf838cb58f6/html5/thumbnails/23.jpg)
Digital Audio Signal Processing Version 2015-2016 Lecture 7: Kalman Filters p. 23 / 30
Overview
Kalman Filters– Introduction – Least Squares Parameter Estimation– Standard Kalman Filter– Square-Root Kalman Filter
DASP Applications– AEC (and AFC/ANC/..)– Noise Reduction (Lecture-3)
![Page 24: Digital Audio Signal Processing Lecture-7 Kalman Filters for AEC/AFC/ANC/Noise Reduction Marc Moonen Dept. E.E./ESAT-STADIUS, KU Leuven marc.moonen@kuleuven.be.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bfe41a28abf838cb58f6/html5/thumbnails/24.jpg)
Digital Audio Signal Processing Version 2015-2016 Lecture 7: Kalman Filters p. 24 / 30
Kalman Filter for AEC (and AFC/ANC/…)
• Time-invariant echo path model
Echo path is assumed to be wk (=regression/state vector)
xk takes the place of C[k] (!)
e[k] is near-end speech, noise, modeling error,..
Kalman Filter then reduces to (standard/QRD) RLS
![Page 25: Digital Audio Signal Processing Lecture-7 Kalman Filters for AEC/AFC/ANC/Noise Reduction Marc Moonen Dept. E.E./ESAT-STADIUS, KU Leuven marc.moonen@kuleuven.be.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bfe41a28abf838cb58f6/html5/thumbnails/25.jpg)
Digital Audio Signal Processing Version 2015-2016 Lecture 7: Kalman Filters p. 25 / 30
Kalman Filter for AEC (and AFC/ANC/…)
• Random walk model
• ‘Leaky’ random Walk Model
• Frequency domain version
•
![Page 26: Digital Audio Signal Processing Lecture-7 Kalman Filters for AEC/AFC/ANC/Noise Reduction Marc Moonen Dept. E.E./ESAT-STADIUS, KU Leuven marc.moonen@kuleuven.be.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bfe41a28abf838cb58f6/html5/thumbnails/26.jpg)
Digital Audio Signal Processing Version 2015-2016 Lecture 7: Kalman Filters p. 26 / 30
Kalman Filter for AEC (and AFC/ANC/…)
• Brain teaser:
For which state space model does the Kalman
Filter reduce to exponentially weighted RLS ?
![Page 27: Digital Audio Signal Processing Lecture-7 Kalman Filters for AEC/AFC/ANC/Noise Reduction Marc Moonen Dept. E.E./ESAT-STADIUS, KU Leuven marc.moonen@kuleuven.be.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bfe41a28abf838cb58f6/html5/thumbnails/27.jpg)
Digital Audio Signal Processing Version 2015-2016 Lecture 7: Kalman Filters p. 27 / 30
Kalman Filter for Noise Reduction
• Assume AR model of speech and noise
• Equivalent state-space model is…
y=microphone signal
u[k], w[k] = zero mean, unit
variance,white noise
][][
][][]1[
kky
kkkTxc
vAxx
![Page 28: Digital Audio Signal Processing Lecture-7 Kalman Filters for AEC/AFC/ANC/Noise Reduction Marc Moonen Dept. E.E./ESAT-STADIUS, KU Leuven marc.moonen@kuleuven.be.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bfe41a28abf838cb58f6/html5/thumbnails/28.jpg)
Digital Audio Signal Processing Version 2015-2016 Lecture 7: Kalman Filters p. 28 / 30
Kalman Filter for Noise Reduction
with:
n
s
NM
T
n
s
g0
0gGC
A0
0AA ;100100;
Tkwkuk ][][.][ Gv
;00;00 nTns
Ts gg gg
TGGQ .
s[k]
an
d n
[k] a
re in
clud
ed
in s
tate
ve
cto
r,
hen
ce c
an b
e e
stim
ate
d b
y K
alm
an
Filt
er
![Page 29: Digital Audio Signal Processing Lecture-7 Kalman Filters for AEC/AFC/ANC/Noise Reduction Marc Moonen Dept. E.E./ESAT-STADIUS, KU Leuven marc.moonen@kuleuven.be.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bfe41a28abf838cb58f6/html5/thumbnails/29.jpg)
Digital Audio Signal Processing Version 2015-2016 Lecture 7: Kalman Filters p. 29 / 30
Kalman Filter for Noise Reduction
Disadvantages iterative approach:• complexity• delay
split signal
in frames
estimate
parameters
Kalman Filter
reconstruct
signal
iterations
y[k]
][̂ks
][ˆ min
Iterative algorithm (details omitted)
![Page 30: Digital Audio Signal Processing Lecture-7 Kalman Filters for AEC/AFC/ANC/Noise Reduction Marc Moonen Dept. E.E./ESAT-STADIUS, KU Leuven marc.moonen@kuleuven.be.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bfe41a28abf838cb58f6/html5/thumbnails/30.jpg)
Digital Audio Signal Processing Version 2015-2016 Lecture 7: Kalman Filters p. 30 / 30
Kalman Filter for Noise Reduction
iteration index time index (no iterations)
State Estimator
(Kalman Filter)
Parameters Estimator
(Kalman Filter)
D
D
]1|1[ˆ kkn
][ky
]|[ˆ],|[̂ kknkks
]1|1[ˆ kkgn],|[ˆ kkgs ]|[ˆ kkgn
ns ˆ,ˆ
ns gg ˆ,ˆ
Sequential algorithm (details omitted)