Recursive Bayesian Estimation Using Gaussian Sums
Transcript of Recursive Bayesian Estimation Using Gaussian Sums
-
8/11/2019 Recursive Bayesian Estimation Using Gaussian Sums
1/15
-
8/11/2019 Recursive Bayesian Estimation Using Gaussian Sums
2/15
-
8/11/2019 Recursive Bayesian Estimation Using Gaussian Sums
3/15
R e c u r s i v e B a y e s i a n e s t i m a t i o n u s i n g g a u s s i a n s u m s 4 6 7
s u g g e s t e d b r i e f l y b y A O K t [ 1 1 ] . M o r e r e c e n t l y ,
CAMERON [12] and Lo [13] have as sum ed t ha t th e
a p r i o r i
d e n s i t y f u n c t i o n f o r t h e i n i t i a l s t a t e o f
l i n e a r s y s t e m s w i t h g a u s s i a n n o i s e s e q u e n c e s h a s
t h e f o r m o f a g a u s s ia n s u m .
2 .1 . T h e o r e t ic a l f o u n d a t i o n s o f t h e a p p r o x i m a t D n
C o n s i d e r a p r o b a b i l i t y d e n s i t y f u n c t i o n p w h i c h
i s a s s u m e d t o h a v e t h e f o l l o w i n g p r o p e r ti e s .
1 . p i s de f ined and cont inuous a t a l l bu t a f in i t e
n u m b e r o f l o ca t io n s
2. Ioop x ) d x = 1
3. p x)>_ 0 for a l l x
I t is c o n v e n i e n t a l t h o u g h n o t n e c e s s a r y t o c o n s i d e r
o n l y sc a l a r r a n d o m v a r i a b l es . T h e g e n e r a l i z a t i o n
t o t h e v e c t o r c a s e i s n o t d i f f i c u lt b u t c o m p l i c a t e s th e
p r e s e n t a t i o n a n d , i t i s f e l t , u n n e c e s s a r i l y d e t r a c t s
f r o m t h e b a s i c i d e a s.
T h e p r o b l e m o f a p p r o x i m a t i n g p c a n b e c o n -
v e n i e n t l y c o n s i d e r e d w i t h i n t h e c o n t e x t o f d e l t a
fami l i e s of pos i t ive type [14] . Bas ica l ly , t hese a re
f a m i l ie s o f f u n c t i o n s w h i c h c o n v e r g e t o a d e l t a , o r
i m p u l s e , f u n c t i o n a s a p a r a m e t e r c h a r a c t e r i z i n g t h e
f a m i l y c o n v e r g e s t o a l i m i t v a l u e . M o r e p r e c is e ly ,
l e t { 6 4} b e a f a m i l y o f f u n c t i o n s o n ( - o % o o) w h i c h
are in t egrab le over every in te rva l . Th i s is ca l l ed a
d e l t a f a m i l y o f p o s i ti v e t y p e i f t h e f o l l o w i n g c o n -
di t ions a re s a t i s f i ed .
(i)
a a
6 ~ x ) d x t e n d s t o o n e a s 2 t e n d s t o s o m e
l imi t va lue ; to for some a .
( i i ) F o r e v e r y c o n s t a n t y > _ 0 , n o m a t t e r h o w
s m a l l, 6 x t e n d s t o z e r o u n i f o r m l y f o r
7 < [ x [ < ~ a s 2 t e n ds t o 2 0.
(iii) 6 z x ) > 0 for a l l x and 2 .
U s i n g t h e d e l t a f a m i l ie s , th e f o l l o w i n g r e s u l t c a n
b e u s e d f o r t h e a p p r o x i m a t i o n o f a d e n s i ty f u n c t io n
p .
T h e o r e m 2.1 . Th e sequence p x x ) w h i c h i s f o r m e d
b y t h e c o n v o l u t i o n o f 6 z a n d p
p a x ) = f ~ oo c S a x - u ) p u ) d u
(7)
c o n v e rg e s u n i f o r m l y t o p x ) o n e v e r y i n t e r i o r s u b -
i n t e r v a l o f ( - ~ , o o) .
Fo r a p ro of of th i s re su l t , s ee KOREVAAR [14] .
W h e n p h a s a f i n i t e n u m b e r o f d i s c o n t i n u i t i e s , t h e
t h e o r e m i s s t i l l v a l i d e x c e p t a t t h e p o i n t s o f d i s -
c o n t i n u i t y . I t s h o u l d b e n o t e d t h a t e s s e n t ia l l y t h e
sam e resu l t i s g iven by T heo rem 2 .1 in FELLER[15].
I f { 6a } i s r e q u i r e d t o s a t i sf y t h e c o n d i t i o n t h a t
i t fo l l o w s f r o m e q u a t i o n ( 7 ) t h a t P a is a p r o b a b i l i t y
d e n s i t y f u n c t i o n f o r a l l 2.
I t i s b a s i c a ll y t h e p r e s e n c e o f t h e g a u s s i a n w e i g h t -
i n g f u n c t i o n t h a t h a s m a d e t h e E d g e w o r t h e x p a n -
s i o n a t t r a c t i v e f o r u s e i n t h e B a y e s i a n r e c u r s i o n
r e l a ti o n s . T h e o p e r a t i o n s d e f i n e d b y e q u a t i o n s
( 3 - 6 ) a r e s i m p l if i e d w h e n t h e
a p r i o r i
dens i t i e s a re
g a u s s i a n o r c l o s e ly r e l a t e d t o t h e g a u s s ia n . B e a r i n g
t h i s in m i n d , t h e f o l l o w i n g d e l ta f a m i l y i s a n a t u r a l
c h o i c e f o r d e n s i ty a p p r o x i m a t i o n s . L e t
,L x) A_U~ x)
= (2rc22) x p [ - x 2 / i2 ] .
( 8 )
I t i s s h o w n w i t h o u t d i f f ic u l ty t h a t N a x ) f o r m s a
d e l t a f a m i l y o f p o s it i v e ty p e a s ; t ~ 0 . T h a t i s , a s t h e
v a r i a n c e t e n d s t o z e r o , t h e g a u s s i a n d e n s i t y t e n d s
t o t h e d e l t a f u n c t i o n .
U s i n g e q u a t i o n s ( 7 ) a n d ( 8) , t h e d e n s i t y a p p r o x i -
m a t i o n p ~ is w r i t t e n a s
oo
P z x ) = p u ) N z x - u ) d u .
(9)
00
I t i s t h i s f o r m t h a t p r o v i d e s t h e b a s i s f o r t h e
g a u s s ia n s u m a p p r o x i m a t i o n t h a t i s t h e s u b j e ct o f
th i s d i scuss ion .
W h i l e e q u a t i o n ( 9 ) i s a n i n t e r e s ti n g r e s u lt , i t d o e s
n o t i m m e d i a t e l y p r o v i d e t h e a p p r o x i m a t i o n t h a t
c a n b e u s e d f o r s p e c if ic a p p l i c a t i o n . H o w e v e r , i t is
c l e a r t h a t p u ) N x x - u ) i s i n t egrable on ( - o% oo)
a n d i s a t l e a s t p i e c ew i s e c o n t i n u o u s . T h u s , ( 9) c a n
i t s e lf b e a p p r o x i m a t e d o n a n y f i n i t e i n t e r v a l b y a
R i e m a n n s u m . I n p a r t ic u l a r , co n s i d e r a n a p p r o x i -
m a t i o n o f P z o v e r s o m e b o u n d e d i n t er v a l ( a, b )
g i v e n b y
/1
p .. /x ) = ~ ~__Ep(x~)N~(x- x,)[ ~,- 4 ,- i ] (i0)
w h e r e t h e i n t e r v a l ( a , b ) i s d i v i d e d i n t o n s u b -
in te rva l s by se lec t ing poin t s ~ , such th a t
a = ~ o < ~ l < . . . ~ , = b .
I n e a c h s u b i n t e r v a l, c h o o s e t h e p o i n t x ~ s u c h t h a t
f
~
P X , ) [ ~ i - 4 , - 1 ] = p x ) d x
~ I
w h i c h i s p o s s i b le b y t h e m e a n - v a l u e th e o r e m . T h e
c o n s t a n t k i s a n o r m a l i z i n g c o n s t a n t e q u a l t o
f
~ o 6 z ( x ) d x = l
k > x , d x
-
8/11/2019 Recursive Bayesian Estimation Using Gaussian Sums
4/15
4 6 8 H .W . SOREN SO N an d D . L . A LSPA CH
a n d i n s u r e s t h a t p . , x i s a d e n s i t y f u n c t i o n . C l e a r l y ,
f o r ( b - a ) s u f fi c i e n tl y l a r g e , k c a n b e m a d e a r b i -
t r a r il y d o s e t o 1. N o t e t h a t i t f o l lo w s t h a t
T h u s , o n e c a n a t t e m p t t o c h o o s e
~ i , # i a i ( i= 1 , 2 . . . . , n )
n
k , '= p ( x i ) [ { , - 4 , - a] = 1
(1 1 )
s o t h a t p . , a e s s e n t i a l ly i s a c o n v e x c o m b i n a t i o n o f
g a u s s i a n d e n s i t y f t m c t i o n s N a . I t i s b a s i c a l l y t h i s
f o r m t h a t w i l l b e u s e d i n a l l f u t u r e d i s c u s s i o n a n d
w h i c h i s r e f e r r e d t o h e r e a f t e r a s t h e gauss ian sum
approximation.
I t i s i m p o r t a n t t o r e c o g n i z e t h e
p . , a t h a t a r e f o r m e d i n t hi s m a n n e r a r e v a li d p r o b a -
b i l i t y d e n s i t y f u n c t i o n s f o r a l l n , 2 .
2 .2 . Implem enta t ion o f the gaussian sum approxima-
tion
T h e p r e c e d i n g d i s c u s s i o n h a s i n d i c a t e d t h a t a
p r o b a b i l i t y d e n s i t y f u n c t i o n p t h a t h a s a f in i t e
n u m b e r o f d i s c o n ti n u i ti e s c a n b e a p p r o x i m a t e d
a r b i t r a r i l y c l o s e l y o u t s i d e o f a r e g i o n o f a r b i t r a r i l y
s m a l l m e a s u r e a r o u n d e a c h p o i n t o f d i s c o n t in u i t y
b y t h e g a u s s i a n s u m p , . a a s d e f i n e d i n e q u a t i o n ( 1 0 ) .
T h e s e a s y m p t o t i c p r o p e r t i e s a r e c e r t a i n l y n e c e s s a ry
f o r t h e c o n t i n u e d i n v e s t i g a t i o n o f t h e g a u s s i a n s u m .
H o w e v e r , f o r p r a c t i c a l p u r p o s e s , i t i s d e s i r a b l e , i n
f a c t i m p e r a ti v e , t h a t p c a n b e a p p r o x i m a t e d t o
w i t h i n a n a c c e p t a b l e a c c u r a c y b y a r e l a t i v e l y s m a l l
n u m b e r o f t e r m s o f th e s e ri es . T h i s r e q u i r e m e n t
f u r n i s h e s a n a d d i t i o n a l f a c e t t o t h e p r o b l e m t h a t i s
c o n s i d e r e d i n t h i s s e c t i o n .
F o r t h e s u b s e q u e n t d i s c u s s i o n , i t i s c o n v e n i e n t t o
w r i t e t h e g a u s si a n s u m a p p r o x i m a t i o n a s
w h e r e
p . ( x ) = ~ ~ l N . , ( x - p t ) ( 12 )
i =1
~ = 1 ; ~ > 0 f o r a l l i .
i =1
T h e r e l a t i o n o f e q u a t i o n ( 1 2) t o e q u a t i o n ( 1 0) i s
o b v i o u s b y i n sp e c t i on . U n l i k e e q u a t i o n ( 1 0 ) i n
w h i c h t h e v a r i a n c e 2 2 i s c o m m o n t o e v e r y t e r m o f
t h e g a u s s i a n s u m , i t h a s b e e n a s s u m e d t h a t t h e
v a r i a n c e a i 2 c a n v a r y f r o m o n e t e r m t o a n o t h e r .
T h i s h a s b e e n d o n e t o o b t a i n g r e a t e r f l e x i bi l i ty f o r
a p p r o x i m a t i o n s u s i n g a f i n i te n u m b e r o f t er m s .
C e r t a i n l y , a s t h e n u m b e r o f t e r m s i n c r e a s e , i t i s
n e c e s s a r y t o r e q u i r e t h a t t h e a i t e n d t o b e c o m e
e q u a l a n d v a n i s h .
T h e p r o b l e m o f c h o o s i n g t h e p a r a m e t e r s ~ , / ~ i , a~
t o o b t a i n th e " b e s t " a p p r o x i m a t i o n p , t o s o m e
d e n s i t y f u n c t i o n p c a n b e c o n s i d e r e d . T o d e f i n e t h i s
m o r e p r e c i s e l y , c o n s i d e r t h e L k n o r m . T h e d i s ta n c e
b e t w e e n p a n d p , c a n b e d e f i n e d a s
oO
p _ p , k = p ( x ) -
s o t h a t t h e d i st a n c e I I P - P ,[ [ i s m i n i m i z e d . A s t h e
n u m b e r o f t e r m s n i n c re a s e s a n d a s t h e v a r i a n c e
d e c r e a s e s t o z e r o , t h e d i s t a n c e m u s t v a n i s h . H o w -
e v e r , f o r f in i t e n a n d n o n z e r o v a r i a n c e , i t i s re a s o n -
a b l e t o a t t e m p t t o m i n i m i z e t h e d is t a n ce i n a m a n n e r
s u c h a s th i s . I n d o i n g t h i s , t h e s t o c h a s t i c e s t i m a t i o n
p r o b l e m h a s b e e n r e c a s t a t t h i s p o i n t a s a d e t e r -
m i n i s t i c c u r v e f i t t i n g p r o b l e m .
T h e r e a r e o t h e r n o r m s a n d p r o b l e m f o r m u l a t i o n s
t h a t c o u l d b e c on s i d e re d . I n m a n y p r o b l e m s , i t
m a y b e d e s i r a b l e t o c a u s e t h e a p p r o x i m a t i o n t o
m a t c h s o m e o f t h e m o m e n t s , f o r e x a m p l e , t h e m e a n
a n d v a r i a n c e , o f t h e t r u e d e n s i t y e x a c t ly . I f t h i s
w e r e a r e q u i r e m e n t , t h e n o n e c o u l d c o n s i d e r t h e
m o m e n t s a s c o n s t r a i n t s o n t h e m i n i m i z a t i o n
p r o b l e m a n d p r o c e e d a p p r o p ri a t e l y . F o r e x a m p le ,
i f t h e m e a n a s s o c i a t e d w i t h p i s p , t h e n t h e c o n -
s t r a in t t h a t p , h a v e m e a n v a l u e p w o u l d b e
U ~ n ~ i al -- i
= ~ ai l t l. (1 4 )
i =1
T h u s , e q u a t i o n ( 14 ) w o u l d b e c o n s i d e r e d i n a d d i t io n
t o t h e c o n s t r a i n t s o n t h e o q s t a t e d a f t e r e q u a t i o n
(12).
S t u d ie s r e l a te d t o t h e p r o b l e m o f a p p r o x i m a t i n g
p w i t h a s m a l l n u m b e r o f te r m s h a v e b e e n c o n d u c t e d
f o r a la r g e n u m b e r o f d e n s i t y f u n c t i o n s . T h e s e i n -
v e s t i g a t i o n s h a v e i n d i c a t e d , n o t s u r p r i s i n g l y , t h a t
d e n s i t i e s w h i c h h a v e d i s c o n t i n u i t i e s g e n e r a l l y a r e
m o r e d i ff ic u l t t o a p p r o x i m a t e t h a n a r e c o n t i n u o u s
f u n c t i o n s . T h e r e s u l t s f o r t w o d e n s i t y f u n c t i o n s , t h e
u n i f o r m a n d t h e g a m m a , a r e d is c u s se d b e l o w . T h e
u n i f o r m d e n s i t y is d i s c o n t i n u o u s a n d i s o f i n t e r e s t
f r o m t h a t p o i n t o f v ie w . T h e g a m m a f u n c t i o n i s
n o n z e r o o n l y f o r p o s i t i v e v a l u e s o f x s o i s a n
e x a m p l e o f a n o n s y m m e t r i c d e n s i t y t h a t e x t e n d s
o v er a semi- in f in i te r an g e .
C o n s i d e r t h e f o l l o w i n g u n i f o r m d e n s it y f u n c t i o n
J' f o r - 2 < x _ 2
p ( x ) = ~ 0 e l s e w h e r e
1 5 )
T h i s d i s t ri b u t i o n h a s a m e a n v a l u e o f z e r o a n d
v a r i a n c e o f 1 .3 3 3 .
T w o d i f fe r e n t m e t h o d s o f f i tt i ng e q u a t i o n ( 1 5)
h a v e b e e n c o n s i d e r e d . F i r s t , c o n s i d e r a n a p p r o x i -
m a t i o n t h a t i s s u g g e s te d d i r e c t l y b y ( 1 0 ) a n d r e f e r r e d
t o s u b s e q u e n t l y a s a Theorem Fi t . T h e p a r a m e t e rs
o f t h e a p p r o x i m a t i o n a r e c h o s e n i n t h e f o l lo w i n g
g e n e r a l m a n n e r .
-
8/11/2019 Recursive Bayesian Estimation Using Gaussian Sums
5/15
R e c u r s i v e B a y e s i a n e s t i m a t i o n u s i n g g a u s s i a n s u m s 4 6 9
( 1) S e l e c t t h e m e a n v a l u e p~ o f e a c h g a u s s i a n
s o t h a t t h e d e n s i t i e s a r e e q u a l l y s p a c e d o n
( - 2 , 2 ) . B y a n a p p r o p r i a t e lo c a t i o n o f t h e
d e n s i t i e s t h e m e a n v a l u e c o n s t r a i n t ( 1 4 ) c a n
b e s a t is f ie d i m m e d i a t e l y .
(2 ) The we ight ing fac tors 0q a re s e t equa l to
i / n s o
~ t = l
t = l
( 3) T h e v a r i a n c e 12 o f e a c h g a u s s i a n i s th e s a m e
a n d i s s el e c te d s o t h a t t h e L 1 d i s t a n c e b e t w e e n
p a n d p , i s m i n i m i z e d .
T h i s a p p r o x i m a t i o n p r o c e d u r e r e q u i re s o n l y a o n e -
d i m e n s i o n a l s e a r c h t o d e t e r m i n e 2 .
T o i n v e s t i g a t e t h e a c c u r a c y a n d t h e c o n v e r g e n c e
o f t h e a p p r o x i m a t i o n , t h e n u m b e r o f t e r m s i n th e
s u m w a s v a r i e d . F i g u r e l ( a ) s h o w s t h e a p p r o x i m a -
t i o n w h e n 6 , 1 0 , 2 0 a n d 4 9 t e r m s w e r e i n c l u d e d . I t
i s i n t e r e s t i n g t o o b s e r v e t h a t t h e a p p r o x i m a t i o n
r e t ai n s t h e g e n e r a l c h a r a c te r o f t h e u n i f o r m d e n s i t y
e v e n f o r t h e s i x -t e r m ca s e . A s s h o u l d b e e x p e c t e d ,
t h e l a r g e s t e r r o r s a p p e a r i n t h e v i c i n i t y o f t h e d i s -
c o n t i n u i t i e s a t + 2 . T h e s e a p p r o x i m a t i o n s e x h i b i t
a n a p p a r e n t o s c i l l a t i o n a b o u t t h e t r u e v a l u e t h a t i s
n o t v i s u a l l y s a t is f y in g . T h i s o s c i l l a t io n c a n b e
e l i m i n a t e d b y u s i n g a s l i g h t l y l a r g e r v a l u e f o r t h e
v a r i a n c e o f t h e g a u s s i a n t e r m s a s i s d e p i c t e d i n
F ig . l (b ) .
T h e s e c o n d a n d f o u r t h m o m e n t s a n d t h e L 1 e r r o r
a r e l i s te d i n T a b l e 1 f o r t h e s e t w o s e ts o f a p p r o x i m a -
t i o n . T h e i n d i v i d u a l t e r m s w e r e l o c a t e d s y m -
m e t r i ca l ly a b o u t z e r o s o t h a t t h e m e a n v a l u e o f t h e
g a u s s i an s u m a g r ee s w i t h t h a t o f t h e u n i f o r m d e n -
s i ty i n a ll c as e s. N o t e t h a t f o r t h e b e s t f it t h e e r r o r
i n t h e v a r i a n c e i s o n l y 1 . 25 p e r c e n t w h e n 2 0 t e r m s
a r e u s e d . A s s h o u l d b e e x p e c t ed , h i g h e r o r d e r
[a.
t~
13.
O . S -
o )
0 . 4
0 . 3
0 . 2
o I /
3 2 . 1
0 . 5 -
(c)
0 . 4
0 . 3
0.2
'-,.b ' 6
X
0 . 5 -
b )
0 . 4
0 . 3
0 . 2 -
' I -~ 0 2 0 3 0 - 3 - 0 - 2 - 0
i , , i i f i t i ,
- I ' 0 0 I ' 0 2 ' 0 3 ' 0
0.1 { '
r , i i , i , 1 , i
- 3 ' 0 - 2 0 - I 0 0 | ' 0
X
Fzo . 1 . Gau ss ian
[ ~ s e o r c h f i t
. . . . . I 0 t e r m b e s t t h e o r e m f i t
. .. .. .. .. I 0 t e r m s m o o t h e d t h e o r e m f i t
k
\
2 . 0 3 . 0
s u m a p p r o x i m a t i o n s o f u n i f or m
d en s i ty fu n c t io n .
( a ) B e s t t h e o r e m f i t - - 6 , 1 0, 2 0 a n d 4 9 t e r m a p p r o x i -
m a t i o n s .
( b ) S m o o t h e d t h e o r e m f i t - - 6 , 1 0 , 2 0 a n d 4 9 t e r m
a p p r o x i m a t i o n s .
(C) L 2 Search fit com parison,
-
8/11/2019 Recursive Bayesian Estimation Using Gaussian Sums
6/15
4 7 0 H . W . S OR EN SO N a n d D . L . A L SP A CH
m o m e n t s c o n v e r g e m o r e s l ow l y si n ce e r ro r s f a r t h e s t
a w a y f r o m th e m e a n a s su m e m o r e i m p o r t an c e . F o r
e x a m p l e , t h e f o u r t h c e n t r a l m o m e n t h a s a n e r r o r o f
5 p e r c en t f o r t h e 2 0 -t e r m a p p r o x i m a t i o n . T h e
e r r o r s i n t h e m o m e n t s o f t h e s m o o t h e d f it a r e
a g g r a v a t e d o n l y s l i gh t l y a l t h o u g h t h e L I e r r o r
i n c r e a s e s i n a n o n t r i v i a l m a n n e r .
TABLE 1. UNIFORM DENSITY APPROXIMATION
Fourth
central L 1
Variance mome nt error
rue values 1 333 3.200 - -
o f t h e n u m b e r o f t e r m s i n v o l v e d , o b t a i n i n g a s e ar c h
f i t i s s i g n i f i c a n t l y m o r e d i f f i c u l t t h a n o b t a i n i n g a
t h e o r e m f it f o r th e s a m e n u m b e r o f t e rm s . T h u s , t h e
t h e o r e m f it m a y b e m o r e d e s i r a b l e f r o m a p r a c t i c a l
s t a n d p o in t . T h e m o m e n t s a n d L 1 e r r o r f o r t h e
s e a r c h f i t i s a l s o i n c l u d e d i n T a b l e 1 . T h e s e v a l u e s
a r e s l i g h tl y b e t t e r t h a n t h e t h e o r e m f it in v o l v i n g t e n
t e r m s . I t i s i n t e r e s t i n g i n F i g . l ( c ) t o n o t e t h e
" s p i k e s " t h a t h a v e a p p e a r e d a t t h e p o i n t s o f d is -
c o n t i n u i t ie s . T h i s a p p e a r s t o b e a n a l o g o u s t o th e
G i b b s p h e n o m e n o n o f F o u r i e r s er ie s.
T h e s e c o n d e x a m p l e t h a t i s d i s c u s s e d h e r e i s t h e
g a m m a d e n s i t y f u n c t i o n . I t is d e f i n e d a s
B e s t t h e o re m f i t
6 term s 1.581 5-320 0.2199
10 terms 1.417 3.884 0.1271
20 term s 1.354 3.363 0.0623
49 terms 1.336 3.226 0.0272
f
0 f o r x < 0
p ( x ) =
x 3 e X f o r x > 0 .
6
( 1 6 )
S m o o t h e d t h e o re m f i t
6 term s 1 "690 6.387 0.2444
10 terms 1.456 4.224 0.1426
20 terms 1.363 3"442 0.0701
49 terms 1.338 3.238 0.0280
L 2 search fi t
6 terms 1.419 3.626 0.0968
A s a n a l t e r n a t i v e a p p r o a c h , t h e p a r a m e t e r s ~ i
# t, tr2 w e r e c h o s e n t o m i n i m i z e t h e L 2 d i s ta n c e
w h i c h i s h e r e a f t e r r e f e r r e d t o a s a n L 2 s e a r c h f i t.
T h e s e r e s u l ts a r e s u m m a r i z e d i n F i g . 1 c ) f o r n = 6 .
I n c l u d e d f o r c o m p a r i s o n i n t h i s fi g u r e a r e t h e 1 0 -
t e r m t h e o r e m f it s f r o m F i g s. l ( a ) a n d l ( b ) . B e c a u s e
T h e d i s t r i b u t i o n h a s a m e a n v a l u e o f 4 a n d s e c o n d ,
t h i rd , a n d f o u r t h c e n t r al m o m e n t s o f 4 , 8 a n d 7 2
r e s p e c t i v e l y .
F i r s t , c o n s i d e r a t h e o r e m f it o f t h i s d e n s i t y in
w h i c h t h e m e a n v a l ue s a r e d i st r i b u te d u n i f o r m l y o n
( 0 , 10 ). F o r t h e u n i f o r m d e n s i t y , t h e u n i f o r m p l a c e -
m e n t w a s n a t u r a l ; f o r t he g a m m a d e n s i t y i t i s n o t
a s a p p r o p r i a t e . F o r e x a m p l e , f o r n = 6 o r 1 0 , i t i s
s e e n i n F i g . 2 (a ) t h a t t h e a p p r o x i m a t i n g d e n s i t y i s
n o t a s g o o d , a t l e a s t v i s u a l l y , a s o n e m i g h t h o p e .
T h e f i r s t f o u r c e n t r a l m o m e n t s a r e l is t ed i n T a b l e 2 .
C l e a r l y , t h e h i g h e r o r d e r m o m e n t s c o n t a i n l a r g e
e r r o r s a n d e v e n t h e m e a n v a l u e i s i n c o r r e c t i n
c o n t r a s t w i t h t h e u n i f o r m d e n s i t y .
TABLE 2. GAMM A DENSITY
APPROXIMATION
T h i rd F o u r th
c e n t r a l
central L1
Mean Variance mom ent mom ent Error
True values
4 4 8 72
T h e o r em f i t
6 terms in (0, 10) 3"94 4"345 5"496 60.78 0"119
10 terms in (0, 10) 3.94 3.861 4-941 47.84 0.053
20 terms in (0, 10) 3.93 3.611 4-653 41.23 0.023
20 terms in (0, 12) 4.00 4.206 7-477 71.41 0.042
L2 search f i t
1 terms in (0, 10) 3'51 3"510 0 10.53 0"203
2 terms in (0, 10) 3"82 3"427 3.04 34"96 0"078
3 terms in (0, 10) 3.91 3.632 4"711 43-39 0"036
4 term s in (0, 10) 3"95 3-744 5"682 49-57 0"018
-
8/11/2019 Recursive Bayesian Estimation Using Gaussian Sums
7/15
R e c u r s i v e B a y e s i a , e s t i m a t i o n u s i n g g a u s s ia n s u m s 4 7 1
Lt.
(L
O - Z
0 ' 2
0 .1
0 .1
0 , 0
0 . 0
- 2 . 0
/J
0 0
a)
1 0 - t e r m
. . . . . . . . . . . . 0 . Z
6 - t e r m . . . . . . . .
o . 2
0 . 1
LL
Q
0 -
0 1
;X
~ *k O 0
2 0
4 0 6 0 8 ' 0
I 0 0 1 2 ' 0
x
: - ' 0 .0 ' - : ~
1 4 " 0 - 2 0
0 0
( b )
i n t e r v a l s
0 ,10 ) . . . . . . .
( 0 , 1 2 ) . . . . . . . . . .
2 . 0 4 . 0 6 . 0 8 . 0 I 0 . 0 1 2 - 0 1 4 . 0
X
0 . 2
0 ' 2
o.,
rt
0 ' 1
I 0 ' 0 t
0 0 t x
- 2 - 0 0 . 0 2 - 0 4 . 0 6 . 0
X
F I O 2 G a u s s i a n s u m a p p r o x i m a t io n s o f g a m m a d e n s i ty
function.
(a) Best theorem fit- -6 an d 10 term approximations.
(b) Tw enty term approximations over different inter-
vals.
(c) L2 search fit: 3 and 4 term approximations.
c )
3 - t e r m . . . . . .
4 - t e r m . . . . . . . . . . .
8 ' 0 1 0 ' 0 1 2 " 0 1 4 " 0
T w o d i f f e r e n t 2 0 - t e r m a p p r o x i m a t i o n s a r e d e -
p i c t e d i n F i g. 2 ( b ) . I n o n e t h e m e a n v a l u e s o f t h e
g a u s s i a n t e r m s a r e s e l e c t e d i n t h e i n t e r v a l ( 0 , 1 0 ) ,
w h e r e a s t h e s e c o n d a p p r o x i m a t i o n i s d i st r ib u t e d i n
( 0 , 1 2 ) . N o t e i n t h e fi r st c a s e t h e g a u s s i a n s u m t e n d s
t o z e r o m u c h m o r e r a p i d ly t h a n t h e g a m m a f u n c t i o n
f o r x > 1 0. T h u s , t o i m p r o v e t h e a p p r o x i m a t i o n a n d
t h e m o m e n t s i t i s n e c e s s a r y t o i n c r e a s e t h e i n t e r v a l
o v e r w h i c h t e r m s a r e p l a c e d a n d t h e s e c o n d c u r v e
i n d i c a t e s t h e i n fl u e n c e o f t h i s c h a n g e . T h e r e s u l t s
o f t h e s e t w o c a s e s a r e i n c l u d e d i n F i g . 2 a n d i n d i c a t e
t h a t i n c r e a s i n g t h e i n t e r v a l o v e r w h i c h t h e a p p r o x i -
m a t i o n i s v a l i d h a s s i g n i f i c a n t l y i m p r o v e d t h e
m o m e n t s .
T h e t h e o r e m f i t p r o v i d e s a v e r y s im p l e m e t h o d
f o r ob t a i n i n g a n a p p r o x i m a t i o n . H o w e v e r , i t i s
c l e a r th a t b e t t e r r e s u l t s c o u l d b e o b t a i n e d b y c h o o s -
i n g a t le a s t t h e m e a n v a l u e s o f t h e i n d i v i d u a l
g a u s s ia n t e r m s m o r e c a r e fu l ly . C o n s i d e r n o w s o m e
L 2 se a r c h f i t s . In F ig . 2 ( c ) , th e s e a r c h f i t s f o r th r e e
-
8/11/2019 Recursive Bayesian Estimation Using Gaussian Sums
8/15
472 H.W. SORENSON and D. L. ALSPACH
and four terms are depicted and the moments are
listed in Table 2. Clearly, values of the mean and
variance appear to be converging to the true values
of 4. Note also that the 4-term approximation is
considerably better than the 10-term theorem fit.
Thus, the search technique, while more difficult to
obtain, points out the desirability of judicious
placement of the gauss,an terms in order to obtain
the most suitable approximation.
3. LINEAR SYSTEMS WITH NONGAUSSIAN
NOISE
It is envisioned that the gauss,an sum approxima-
tion will be very useful in dealing with non-linear
stochastic systems. However, many of the
properties and concomitant difficulties of the
approximation are exhibited by considering linear
systems which are influenced by nongaussian noise.
As is well-known, the a p o s t e r i o r , density p ( X k / Z k )
is gauss,an for all k when the system is linear and
the initial state and plant and measurement noise
sequences are gauss,an. The mean and variance of
the conditional density are described by the Kalman
filter equations. When nongaussian distributions
are ascribed to the initial state and/or noise se-
quences,
p ( X k / Z k )
is no longer gauss,an and it is
generally impossible to determine p ( X k / Z k ) in a
closed form. Furthermore, in the linear, gauss,an
problem, the conditional mean, i.e. the minimum
variance estimate, is a linear function of the
measurement data and the conditional variance is
independent of the measurement data. These
characteristics are generally not true for a system
which is either non-linear or nongaussian.
For the following discussion, consider a scalar
system whose state evolves according to
X k = ~ k , k - l X k - 1 + W k - 1
(17)
and whose behavior is observed through measure-
ment data z k described by
z k = H k x ~ + V K. (18)
Suppose that the density function describing the
initial state has the form
lo
p(xo)= ~ ~ , o N t r ' i ( X o - , ; o ) . (19)
i = l
Assume that the plant and measurement noise
sequences (i.e.
{ W k }
and {Vk} are mutually inde-
pendent, white noise sequences with density func-
tions represented by
~k
p(w~)= Y t~,~Nqk,(w~--o),~) (20)
i = 1
m
P @ k ) : E ] ' i k N r , k ( O k - - V i k ) ( 2 1 )
i = 1
There are a variety of ways in which the gauss,an
sum approximation could be introduced. For
example, it is natural to proceed in the manner tha t
is to be discussed here in which the a p r i o r i distribu-
tions are represented by gauss,an sums as in equa-
tions (19), (20) and (21). This approach has the
advantage that the approximation can be deter-
mined off-line and then used directly in the Bayesian
recurs,on relations. An alternative approach
would be to perform the approximation in more of
an on-line procedure. Instead of approximating the
a p r i o r i
densities, one could deal with
p ( X J Z k )
in
equation (3) and the integrand in (4) and derive
approximations at each stage. This would be more
direct but has the disadvantage that considerable
computation may be required during the processing
of data. Discussion of the implementat ion of this
approach will not be attempted in this paper.
3 . 1. D e t e r m i n a t i o n o f t h e a posterior, d i s t r i b u t i o n s
Suppose that the
a p r i o r i
density functions are
given by equations (19, 20 and 21). In using these
representations in the Bayesian recurs,on relations,
it is useful to note the following properties of
gauss,an density functions
S e h o l i u m 3.1: For a4:0,
N . ( x - a y ) = 1 N . l . ( y - x / a ) .
(22)
a
S c h o l i u m 3.2:
where
N ~ , ( x - , i ) N , j ( x - , j )
2
- - 2 2 1 ~
- N t . , , + ~ j ( . , - . s ) N ~ , s ( x - . , j )
. , 4 . J 4
2 2
_ 2 _ _ i f , 7 j
O s
a ? + a ~ .
(23)
The proofs of (22) and (23) are omitted.
Armed with these two results it is a simple matter
to prove that the following descriptions of the
filtering and prediction densities are true.
T h e o r e m
3.1. Suppose that
P ( X k / Z k - O
is
described by
It ,
p ( X k / Z k - 1 ) = E a * k N , ; k ( X k -- ; k ) (24)
i = 1
Then,
p ( X k / Z k )
is given by
l k m k
p(~dz~)= E Z , j N . , ~ x ~ - ~ , j ) (25)
i = 1 j = I
-
8/11/2019 Recursive Bayesian Estimation Using Gaussian Sums
9/15
R e c u r s i v e B a y e s i a n e s t i m a t i o n u s i n g g a u s s i a n s u m s 4 73
w h e r e
e~ , , .~ N ~(z~ - P~ ,. )
C i j = O t l k T j k N x ( z k P i j m ~ = 1
y ' 2 _ _ 1 t . /' 2 _ . '2 - - _ 2
- - ~ k O t k 1- r j k
, a l ~ H ~ r z
eq = I~ik4 ~ ,2 ~-- 7-Y 7, 2 L k - - V~k-- Hk tk]
~ t k - r l k t r j k
2 , '2 , '4 2 , '2 2 2
t ru = ~ i~ - t rt k H k / (a i~ H ~ + r ~ ) .
I t i s o b v i o u s t h a t t h e c u > 0 a n d t h a t
l k rak
2 Z c,~=~.
T h u s , e q u a t i o n ( 2 5 ) i s a g a u s s i a n s u m a n d f o r c o n -
v e n i e n e e o n e c a n r e w r i t e i t a s
I lk
p(x k /Z k ) = ~ . a t kN ~ , ~ (X k- - I~i k) (26)
t = 1
w h e r e n k = ( l k ) ( m k ) a n d t h e a ik , P i k a n d / q k a r e
f o r m e d i n a n o b v i o u s f a s h i o n f r o m t h e c u , tr u a n d
e U
T h e p r o o f o f t h e t h e o r e m i s s t r a i g h t f o r w a r d .
F r o m t h e d e f i n i ti o n o f th e m e a s u r e m e n t r e l a t i o n
( 18 ) a n d t h e m e a s u r e m e n t n o i s e d e n s i t y ( 21 ), o n e
sees tha t
m k
P(Zk/Xk) = ~ . , ~ k N , , k ( Z k - - H k x k - - V tk ) .
U s i n g t h i s a n d ( 2 4) i n (3 ) a n d a p p l y i n g t h e t w o
s c h o l i u m s , o n e o b t a i n s ( 25 ).
T h e p r e d i c t i o n d e n s i t y i s d e t e r m i n e d f r o m ( 4) a n d
l e a d s t o t h e f o l l o w i n g r e su l t .
T h e o r e m
3 .2 . A s s u m e t h a t
p ( x k / Z k )
i s g iven by
( 26 ). T h e n f o r t h e l i n e a r s y s te m (1 7 ) a n d p l a n t
n o i s e ( 20 ) t h e p r e d i c t i o n d e n s i t y
p ( X k + 1 / Z k )
is
n k $ k
p(x~+l/z~)= Y, 2 a ~j J /x~ + l
i = j= l
- Ck + 1, kl~ik - 09jk) (27)
w h e r e
2 2 2 2
2 u = ~ + l , k P lk + q yk
I t i s c o n v e n i e n t t o r e d e f i n e te r m s s o t h a t ( 27 ) c a n
b e w r i t t e n a s
l k + l
p x ~ +
~ l Z ~ ) = ~ ~ ( ~ + ~ ) N ~ ( ~ + , ) ( x k + ~ - ~ i ( ~ + ~)).
2 8 )
C l e a r l y , t h e d e f i n i t i o n o f p ( X o ) h a s t h e f o r m ( 2 8 )
a s d o e s t h e p ( x k / Z k - 1 ) a s s u m e d i n T h e o r e m 3 . 1 .
T h u s , i t f o l l o w s t h a t t h e g a u s s i a n s u m s r e p e a t
t h e m s e l v e s f r o m o n e s t a g e t o t h e n e x t a n d t h a t ( 2 6 )
a n d ( 2 8 ) c a n b e r e g a r d e d a s t h e g e n e r a l f o r m s t o r
a n a r b i t r a r y s t ag e . T h u s , t h e g a u s s i a n s u m c a n
a l m o s t b e r e g a r d e d a s a r e p r o d u c i n g d e n s i t y [ 1 6 ] .
I t i s i m p o r t a n t , h o w e v e r , t o n o t e t h a t t h e n u m b e r
o f t e r m s i n t h e g a u s s i a n s u m i n c r e as e s a t e a c h s t a g e
s o t h a t t h e d e n s i t y is n o t d e s c r i b e d b y a f i x e d
n u m b e r o f p a r am e t e r s . T h e d e n s i t y w o u l d b e t r u ly
r e p r o d u c i n g i f o n l y t h e i n i t i a l s t a t e w e r e n o n -
gauss ian , for exam ple , s ee Ref . [12]. I t i s c l ea r in
t h i s c a s e t h a t t h e n u m b e r o f te r m s i n t h e g a u s s i a n
s u m r e m a i n s e q u a l t o t h e n u m b e r u s e d t o d e f i n e
p ( x o ) so t h a t p ( X k / Z k ) i s d e s c r i b e d b y a f i x e d n u m b e r
o f p a r a m e t e r s .
T h e r e a r e s e v e r a l a s p e c t s t h a t r e q u i r e c o m m e n t
a t t h i s p o i n t . F i r s t , i f t h e g a u s s i a n s u m s f o r t h e a
p r i o r i
d e n s i t y a ll c o n t a i n o n l y o n e t e r m , t h a t i s , t h e y
a r e g a u s s i a n , t h e K a l m a n f i l t e r e q u a t i o n s a n d t h e
g a u s s i a n
a p o s t e r i o r i
d e n s i t y a r e o b t a i n e d . I n f a c t
t h e e u a n d a . . 2 i n (2 5) a n d t h e m e a n s a n d v a r i a n c e s
~J
21k2 in ( 2 7) e a c h r e p r e s e n t t h e K a l m a n f i l t e r e q u a -
t i o n s f o r t h e i j t h d e n s i t y c o m b i n a t i o n . T h u s , t h e
g a u s s i a n s u m i n a m a n n e r o 1 s p e a k i n g d e s c r i b e s a
c o m b i n a t i o n o f K a l m a n f i lt er s o p e r a ti n g i n c o n c e r t .
T o e x a m i n e t h i s t u r t h e r , c o n s i d e r t h e f i r s t a n d
s e c o n d m o m e n t s a s s o c i a t ed w i t h t h e p r e d i c t i o n a n d
f i l t e r ing d ens i t i e s .
T h e o r e m 3.3
l k m k
1 = 1 j = l
e E ( x ~ - ~ / ~ ) ~ l z d A _ p ~ / ~
l k m k
Y __ ., Y . , c i j [ o , j 2 + ~ k / , , - ~ t j) 2 ] 3 0 )
i = 1 j = l
E [ x k + ~ l Z d A e ~ + x / ~ = O k + ~ , k e ~ /k + E [ w d (31)
E [ x ~ + 1 - ~ + ~ ~ ) 2[
Z d = p k
+ ~ / k
2
= ~ k+ l , k2pk / k2 + E { [w k- -E (o gk ) ] 2} (32)
w h e r e
~k
t-----1
~k
E { E W k - - ~ ( ( D k ) ] 2 } = ~ ~ i k q i k 2 d l- ( .Oik 2) - - E 2 W k ) .
i - - - I
I n e q u a t i o n ( 2 9 ) , t h e m e a n v a l u e ~ k / k i s f o r m e d
a s t h e c o n v e x c o m b i n a t i o n o f th e m e a n v a l u e s 8 q
o f t h e i n d i v i d u a l t e r m s , o r K a l m a n f i l t e r s , o f t h e
g a u s s i a n su m . I t is
i m p o r t a n t t o r e c o g n i z e
t h a t t h e
c q , a s i s a p p a r e n t f r o m e q u a t i o n ( 25 ), d e p e n d u p o n
t h e m e a s u r e m e n t d a t a . T h u s , t h e c o n d i t i o n a l m e a n
is a n o n - l i n e a r f u n c t i o n o f t h e c u r r e n t m e a s u r e m e n t
d a t a .
-
8/11/2019 Recursive Bayesian Estimation Using Gaussian Sums
10/15
4 7 4 H .W . SOREN SO N an d D . L . A LSPA CH
T h e c o n d i t i o n a l variance p k / k 2 d e s c r i b e d b y ( 3 0 )
i s m o r e t h a n a c o n v e x c o m b i n a t i o n o f t he v a r i a n c es
o f t h e i n d iv i d u a l t e r m s b e c a u s e o f t h e p r e s e n c e o f
t h e t e r m ( ~ k / k - - e i j) z - T i f fs s h o w s t h a t t h e v a r i a n c e
i s i n c r e a se d b y t h e p r e s e n c e o f t e rm s w h o s e m e a n
v a l u e s d i f f e r s i g n i fi c a n t l y f r o m t h e c o n d i t i o n a l m e a n
~ k / k. T h e i n f l u e n c e o f th e s e t e r m s i s t e m p e r e d b y
t h e w e i g h t i n g f a c t o r c ~ j . N o t e a l s o t h a t t h e c o n -
d i t i o n al v a r i a n c e ( i n c o n t r a s t t o t h e l i n e a r K a l m a n
f i lt e r ) i s a f u n c t i o n o f t h e m e a s u r e m e n t d a t a b e c a u s e
o f th e c i j an d th e (~k /k - - e i j ) .
T h e m e a n a n d v a r i a n c e o f t h e p r e d i c t io n d e n -
s i t y a r e d e s c r i b e d in a n o b v i o u s m a n n e r . I f t h e
g a u s s ia n s u m i s a n a p p r o x i m a t i o n t o t h e t r u e
n o i s e d e n s i t y , t h e s e r e l a t i o n s s u g g e s t t h e d e s i r a b i l i t y
o f m a t c h i n g t h e fi rs t t w o m o m e n t s e x a c t ly i n o r d e r
t o o b t a i n a n a c c u r a t e d e s c r i p t io n o f t h e c o n d i t i o n a l
m e a n ~ k+ l /k a n d v a r i a n c e P k + l /~ .
A s d i s c u s s e d e a r l i e r, i t is c o n v e n i e n t t o a s s i g n t h e
s a m e v a r i a n c e t o a l l t e r m s o f t h e g a u s s i a n s u m .
T h u s , i f t h e i n i ti a l s t a t e a n d t h e m e a s u r e m e n t a n d
n o i s e s e q u e n c e s a r e i d e n t i c a l l y d i s t r i b u t e d , i t i s
r e a s o n a b l e t o c o n s i d e r t h e v a r i a n c e s f o r a l l t e r m s t o
b e i d e n t i c a l a n d t o d e t e r m i n e t h e c o n s e q u e n c e s o f
t h is a s s um p t i o n . N o t e f r o m S c h o l i u m 3 . 2 t h a t i f
t h e n
0"i2 = O ' j 2 m o - 2
rU 2 = a 2 /2 f o r a l l i , j .
T h u s , t h e v a r i a n c e r e m a i n s t h e s a m e f o r a l l t e r m s
i n t h e g a u s s i a n s u m
w h e n p x k / Z k )
i s f o r m e d a n d t h e
c o n c e n t r a t i o n s a s d e s c r i b e d b y t h e v a r i a n c e b e c o m e s
g r e a t er . F u r t h e r m o r e , f r o m S c h o l i u m 3 .2 it f o l lo w s
t h a t t h e m e a n v a l u e i s gi v e n b y
i j - - ~ i 3 7 / ~ j
2
s o t h e n e w m e a n v a l u e i s th e a v e r a g e o f t h e p r e v i o u s
m e a n s . T h i s s u g g es t s t h e p o s s i b i l i t y t h a t t h e m e a n
v a l u e s o f s o m e t e r m s , s i n c e t h e y a r e t h e a v e r a g e o f
t w o o t h e r t e r m s , m a y b e c o m e e q u a l , o r a l m o s t
e q u a l. I f t w o t e r m s o f t he s u m h a v e e q u a l m e a n s
a n d v a r i a n c e s , t h e y c o u l d b e c o m b i n e d b y a d d i n g
t h e i r r e s p e c t iv e w e i g h t i n g f a c t o r s . T h i s w o u l d
r e d u c e t h e t o t a l n u m b e r o f te r m s i n t h e g a u s s ia n
s u m .
T h e
~j
i n ( 2 5 ) a r e e s s e n t i a l l y d e t e r m i n e d b y a
g a u s s i a n d e n s it y . T h u s , i f
Zk - -p ~ j
b e c o m e s v e r y
l a r g e , t h e n t h e c i j m a y b e c o m e s u f f i c i e n t l y s m a l l
t h a t t h e e n t i r e t e r m i s n e g l ig i b l e. I f t e r m s c o u l d b e
n e g l e ct e d , th e n t h e t o t a l n u m b e r o f t e r m s i n t h e
g a u s s i a n s u m c o u l d b e r e d u c e d .
T h e p r e d i c t i o n a n d f i l t e r i n g d e n s i t i e s a r e r e p r e -
s e n t e d a t e a c h s t a g e b y a g a u s s i a n s um . H o w e v e r ,
i t h a s b e e n s e e n t h a t t h e s u m s h a v e t h e c h a r a c t e r i s t i c
t h a t t h e n u m b e r o f t e r m s i n c r e a s e s a t e a c h s t a g e a s
t h e p r o d u c t o f th e n u m b e r o f te r m s i n t he t w o c o n -
s t i t u e n t s u m s f r o m w h i c h t h e d e n s i t i e s a r e f o r m e d .
T h i s f a c t c o u l d s e r i o u s l y r e d u c e t h e p r a c t i c a l i t y o
t h i s a p p r o x i m a t i o n i f t h e r e w e r e n o a l l e v i a t i n g
c i r c u m s t a n c e s . T h e d i s c u s si o n a b o v e r e g a r d i n g t h e
d i m i n i s h i n g o f t h e w e i g h t i n g f a c t o r s a n d t h e c o m -
b i n i ng o f t e r m s w i t h n e a r l y e q u a l m o m e n t s h a s
i n t r o d u c e d t h e m e c h a n i s m s w h i c h s i g n i f i c a n t l y
r e d u c e t h e a p p a r e n t i ll e f fe c t s c a u s e d b y t h e i n c r e a se
i n t h e n u m b e r o f t e r m s i n th e s u m . I t is a n o b s e r v e d
f a c t t h a t t h e m e c h a n i s m s w h e r e b y t e r m s c a n b e
n e g l e ct e d o r c o m b i n e d a r e i n d e e d o p e r a t i v e a n d i n
f a c t c a n s o m e t i m e s p e r m i t t h e n u m b e r o f t e r m s i n
t h e s e r ie s t o b e r e d u c e d b y a s u b s t a n t i a l a m o u n t .
S i n c e w e i g h t i n g f a c t o r s f o r i n d i v i d u a l t e r m s d o
n o t v a n i s h i d e n t i ca l l y n o r d o t h e m e a n a n d v a r i a n c e
o f m o r e g a u s s i a n d e n s i t i e s b e c o m e i d e n t i c a l, i t is
n e c e s s a r y t o e s t a b l i s h c r i t e r i a b y w h i c h o n e c a n
d e t e r m i n e w h e n t e r m s a r e n e g l ig i b l e o r a r e a p p r o x i -
m a t e l y t h e sa m e . T h i s is a c c o m p l i s h e d b y d e fi n i n g
n u m e r i c a l t h r e s h o l d s w h i c h a r e p r e s c r i b e d t o m a i n -
r a i n t h e n u m e r i c a l e r r o r le s s t h a n a c c e p t a b l e l i m i t s.
C o n s i d e r t h e e f f e c t s o n t h e L ~ e r r o r o f n e g l e c t i n g
t e r m s w i t h s m a l l w e i g h t i n g f a c t o r s . S u p p o s e t h a t
t h e d e n s i t y i s
p ( x ) = ~ o q N a x - a i ) (3 3 )
i = 1
a n d s u c h t h a t a l , a 2 . . . . , ~ , , - l ( m < n ) a r e l es s t h a n
s o m e p o s i ti v e n u m b e r 6 ~ . N o t e t h a t t h e v a r ia n c e
h a s b e e n a s s u m e d t o b e t h e s a m e f o r e a c h t e r m .
C o n s i d e r r e p l a c i n g p b y P a w h e r e
pA(X) = 1 ~
o~iN, , x - -a i ) .
(3 4 )
~ , O ~i i = m
i = m
T h e t o l l o w i n g
d i f f icu l ty .
T h e o r e m
3.4.
b o u n d i s
d e t e r m i n e d w i t h o u t
f
~ m - - 1
Ip(x)-p (x)ldx 2 E ( 3 5 )
1 = 1
< 2(m - 1 )61 . (36)
T h e L ~ e r r o r c a u s e d b y n e g l ec t i ng ( m - 1) t e r m s
each o f w h ich a r e l e s s th an f i~ i s s een in (3 5 ) to b e
l e s s t h a n t w i c e t h e s u m o f t h e n e g l e c t e d t e r m s .
T h u s , t h e t h r e s h o l d fi~ c a n b e s e l e c t e d b y u s i n g ( 3 6 )
o r ( 3 5 ) t o k e e p t h e i n c r e a s e d L 1 e r r o r w i t h i n
a c c e p t a b l e l i m i ts .
C o n s i d e r t h e s i t u a t i o n i n w h i c h t h e a b s o l u t e
v a l u e o f t h e d i f f e r e n c e o f t h e m e a n v a l u e s o f t w o
t e r m s i s sm a l l . I n p a r t i c u l a r , s u p p o s e t h a t a 1 a n d a 2
a r e a p p r o x i m a t e l y t h e s a m e a n d c o n s i d e r t h e L 1
e r r o r t h a t r e s u l t s i f t h e
p x )
g iv en in (3 3 ) i s r ep lac ed
b y
p a ( x ) = ~
o q N , x - a i ) + o q + o t z ) N ~ x - ~ )
(3 7 )
i = 3
-
8/11/2019 Recursive Bayesian Estimation Using Gaussian Sums
11/15
R e c u r s i v e B a y e s i a n e s t i m a t i o n u s i n g g a u s s i a n s u m s 4 7 5
w h e r e
U s i n g ( 3 7 ) , o n e c a n p r o v e t h e f o l l o w i n g b o u n d .
F o r a d e t a i l e d p r o o f o f th i s a n d o t h e r r e s u l t s , se e
Ref. [17].
Theorem 3.5.
f ~ _ o l P x ) - p A X ) l d x < _ 4 ~ 2 M l a 2 - a l I+ ~
(3 8 )
T h u s t e r m s c a n b e c o m b i n e d i f t h e r i g h t - h a n d s i d e
o f ( 3 8 ) i s l e s s t h a n s o m e p o s i t i v e n u m b e r 6 2 w h i c h
r e p r e s e n t s t h e a l l o w a b l e L ~ e r r o r . T h e M i n ( 3 8) i s
t h e m a x i m u m v a l u e o f N , a n d i s gi v e n b y
1
M =
O b s e r v e t h a t a s t h e v a r i a n c e t r d e c r e a s e s , t h e
d i s t a n c e b e t w e e n t w o t e r m s t o b e c o m b i n e d m u s t
a l s o d e c r e a s e i n o r d e r t o r e t a i n t h e s a m e e r r o r
b o u n d .
3.2. A numerical example
I n t h i s s e c t i o n t h e r e s u l t s p r e s e n t e d i n s e c t i o n 3 .1
a r e a p p l i e d t o a s p e c i fi c e x a m p l e . T o m a k e ( 1 7 )
a n d ( 1 8 ) m o r e s p e c i f ic , s u p p o s e t h a t t h e s y s t e m i s
X k ~ - - X k - I ~ Wk- 1
(3 9 )
Zk = Xk + vk
(4 0 )
w h e r e t h e xo, Wk, Vk k= O , 1 . . . ) a r e a s s u m e d t o
b e u n i f o r m l y d is t r i b u te d o n ( - 2 , 2 ) as d e fi n e d b y
(15).
T h e p r o b l e m t h a t i s c o n s i d e r e d re p r e s e nt s s o m e -
t h i n g o f a w o r s t c a s e f o r t h e a p p r o x i m a t i o n b e c a u s e
t h e i n i ti a l s t a t e a n d t h e n o i s e s e q u e n c e s a r e a s s u m e d
t o b e u n i f o r m l y d i s t r i b u t e d w i t h t h e d e n s i t y d i s -
c u s s e d i n s e c t i o n 2 .2 . A s d i s c u s s e d t h e r e , t h e d i s -
c o n t i n u i t i e s a t + 2 m a k e i t d i i f ic u l t t o f i t t h i s d e n s i t y
a n d n e c e s si t at e s th e u s e o f m a n y t e r m s i n t h e
g a u s s i a n s u m . T h e s p e c if i c a p p r o x i m a t i o n u s e d
h e r e c o n t a i n s 1 0 t e r m s a n d i s s h o w n i n F i g . l ( b ) .
I t i s a p p a r e n t t h a t t h i s a p p r o x i m a t i o n h a s n o n -
t r i v i a l e r r o r s i n t h e n e i g h b o r h o o d o f t h e d is -
c o n t i n u i t i e s b u t n o n e t h e l e s s r e t a i n s t h e b a s i c
c h a r a c t e r o f t h e u n i f o r m d i s t ri b u t io n .
T h e p e r f o r m a n c e o f th e g a u s si a n s u m a p p r o x i m a -
t i o n f o r t h i s e x a m p l e i s d e s c r i b e d b e l o w b y c o m -
p a r i n g t h e c o n d i t io n a l m e a n a n d v a r i a n c e p r o v i d e d
b y t h e a p p r o x i m a t i o n w i t h t h a t p r e d i c t e d b y t h e
K a l m a n f i l t e r a n d w i t h t h e s t a ti s t ic s o b t a i n e d b y
c o n s i d e r in g t h e t r u e u n i f o r m d i s t ri b u t i on . T h e
l a t t e r h a v e b e e n d e t e r m i n e d f o r t h is e x a m p l e a f t e r a
n o n t r i v ia l a m o u n t o f n u m e r i c a l c o m p u t a t i o n . I n
a d d i t i o n t h e t r u e a posteriori d e n s i t y p Xk/Zk) h a s
b e e n c o m p u t e d a n d i s c o m p a r e d w i t h t h a t o b t a i n e d
u s i n g t h e a p p r o x i m a t i o n .
T h e K a l m a n e r r o r v a r i a n c e is i n d e p e n d e n t o f t h e
m e a s u r e m e n t s e q u e n c e a n d c a n c a u s e m i s l e a d i n g
f i l te r r e s p o n s e . F o r e x a m p l e , f o r s o m e m e a s u r e -
m e n t s e q u e n c e s p e r f e c t k n o w l e d g e o f t h e s t a t e i s
p o s s i b l e , t h a t i s t h e v a r i a n c e i s z e r o , b u t t h e K a l m a n
v a r i a n c e s t il l p r e d i c t s a l a r g e u n c e r t a i n t y . F o r
e x a m p l e , s u p p o s e t h a t t h e m e a s u r e m e n t s a t e a c h
s t a g e a r e e q u a l t o
Z k = 2 k + 4 , k = 0 , 1 . . .
T h e n , t h e m i n i m u m m e a n - s q u a r e e s t i m a t e f o r t h e
s t a t e b a s e d o n t h e u n i f o r m d i s t r i b u t i o n i s
. k / k = 2 k + 2 , k = O , 1 . . .
a n d t h e v a r i a n c e o f t h i s e s t i m a t e i s
Pk/k2=O f o r a l l k .
T h u s , f o r t h i s m e a s u r e m e n t r e a l i z a t i o n t h e m i n i -
m u m m e a n - s q u a r e e s t i m a t e is e r r o r - f r e e . S i n c e t h e
K a l m a n v a r i a n c e is in d e p e n d e n t o f t h e m e a s u r e -
m e n t s , i t is n e c e s s a ri l y a p o o r a p p r o x i m a t i o n o f t h e
a c t u a l c o n d i t i o n a l v a r i an c e . T h e s q u a r e r o o t o f t h e
K a l m a n v a r i a n c e 7KAL a n d t h e e r r o r i n t h e b e s t
l i n e a r e s ti m a t e t o g e t h e r w i t h t h e s q u a r e r o o t o f th e
v a r i a n c e a~s a n d e r r o r i n t h e e s t i m a t e o f t h e s t a te
f o r t h e t e n t e r m g a u s s ia n s u m f o r t hi s m e a s u r e m e n t
r e a l i z a t i o n is s h o w n i n F i g . 3 (a ) . T h i s s h o w s t h a t a
c o n s i d e r a b l e i m p r o v e m e n t i n b o t h t h e m e a n a n d
v a r i a n c e i s p r o v i d e d b y t h e g a u s s i a n s u m w h e n
c o m p a r e d w i t h t h e r e su l ts p r o v i d e d b y th e K a l m a n
f i l ter .
(a)
o ' i o
0 8
0-7
0.6
0 5
0 4
0-5
0.2
0 1
%
I 0
0-9
0.8
0.7
0.6
0.5
0-4-
0 .3
0.2
0.1
0 0
+ f . t r - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + C T K A
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ~ K A L
. . . . '
f
/
( .
: 1 - ~ . . . . . . . . . X C S
- / .
v I 1 I I I I I I 1 I I I I 1 I I I I I
I 2 3 4 5 6 7 8 9 10 II 12 13 14 15 16 17 18 19 20
K
b )
I / /
I , , ' / V , x o o ,
~ . a r R u z
- 5 ', "~'.-'.,:'-~c.-~'
-
8/11/2019 Recursive Bayesian Estimation Using Gaussian Sums
12/15
476 H .W . SORENSON and D. L. ALSPACH
The measurement seq uence d escr ibed above i s
highly imp robab le. A more representat ive case is
d ep ic t ed in F ig . 3 (b ) in w h ich t he p lant and measure-
ment noise s eq uences were chosen us ing a rand om
number generat or for t he un i form d is t r ibut ion
prescr ibed above . Th is f igure again show s t hat t he
gauss ian sum approx imat ion t o t he t rue var iance i s
cons id erab ly improved over t he Kalman es t imat e
and, in fact , agrees very closely with the true
s t and ard d eviat ion a T RU ~ of t he
p o s t e r i o r i
density.
However , t he Kalman var iance i s represent at ive
enough t hat t he error in t he es t imat e of t he mean i s
not part icularly d if ferent than that provided by the
gaussian sum.
T h e p o s t e r i o r i density funct ion for the third and
fourteenth stages is shown in Figs . 4 (a) and 4(b) .
In these f igures , the actual density, the gaussian
approx imat ion provid ed by t he Kalman f i l t er and
t he gauss ian sum approx imat ion are a l l inc lud ed .
At stage 3 , the true variance is smal ler than the
Kalman var iance and t he Kalman mean has a
s ignif icant error whereas at s tage 14 the true
variance is larger than that predicted by the Ka lman
fi lter . No te that the error in the
p r i o r i
density
approximat ion at the d iscont inuit ies is s t i l l evident
in the
p o s t e r i o r i
approx imat ion but t hat t he
general character of the density is reproduced by
the gaussian sum.
b .
( : 3
13.
0 , 7 5
0 . 5 0
0 2 5
a )
T r u e p r o b a b i l i t y d e n s i t y
. . . . . . . K a l m a n a p p r o x i m a t io n
. . . . G a u s s i a n s u m a p p r o x i m a t io n
/ \
l /
+ . . \
. 7 .
. : . ' '. + . \
. f'/ / .
. . . i + . . ' \
\
' '#
+ ~ . X
, , , / / 't..
\
i i
0 5 I ' 0 1 ' 5 2 0 2 ' 5 S ' O
x
0 . 5
0 . 4
~ 0 . 3
a
( 3 -
0 2
O i
I 0
b )
. . . . ,
, . . \
. . L
' ,
j .:.:,',,,
/ ," ". \
/ ' .. \
. : . .. \ \
/ . : ' . \
/ . . ' ' ,
/ . . ' ' .
1 , 0 2 ' 0 3 - 0 . 4 - 0
x
~ 3
5 0
0 ' 5 0
r ~
Q .
0 ' 2 5
( c )
o .7 5 T r u e p r o b a b i l i t y d e n s i t y
. . . . . 8 ~ = 0 . 0 0 1 = 8
. , . ' , ~
; ' ~
~ I I I
0 - 0 0 . 5
0 . 4 -
0 3
Q
n O . 2
%
I I 0
I ' 0 I - 5 2 0 2 ' 5 S ' O 0
x
d )
. .. .. .. .. .. 8 , = 0 . 0 O I , 8 = 0 . 0 0 . 5
. . . . . 8 1= 0 0 0 5 , 8 2 = 0 . 0 O I
. ~ . ~ :+ -h +
Y
;W
~ ..
/ . ' - I : \
~ - 1 , , , , , , ,
1 . 0 2 0 3 0 4 . 0
x
FIO. 4. Influence of 61, &z on p o s t e r i o r l density.
(a) Third stal~l---82------0 001.
Co) Fourteenth stag~---Sl =8 2 =0 .00 1.
( c) T hir d s t a l l = 0 0 0 1 , 6 2 = 0 . 0 0 5 . a n d 5 t = 0 0 0 5 , 6 1 = 0 0 0 1 .
(d) Fourteenth staile---alffi04)05, &z=0 001. and 51 =0 .00 1, 8 z= 0.0 05 .
\
5 ' 0
-
8/11/2019 Recursive Bayesian Estimation Using Gaussian Sums
13/15
Recursive Bayesian estimation using gaussian sums 477
In Figs. 4(a) and 4(b) the approximations at each
stage are based on terms being eliminated when
their weighting functions are less than St=0.001
and combined when the difference in mean values
cause the L ~ error to be less than 52=0.001. The
effect of changing these parameters, that is, 5~ and
52, can be seen in Figs. 4(c ) and 4(d). In these
figures, the effect of changing 5t and c52 is shown.
In one case tS~ is increased from 0.001 to 0.005 while
keeping 52 equal to 0.001. Alternatively, 52 is in-
creased to 0.005 while the value of 5~ is maintained
equal to 0.001. It is apparent in this example tha t
52 has a larger effect on the density approximation.
The increase in this parameter can be seen to
introduce a ripple into the density function and
indicates that the individual terms have become too
widely separated to provide a smooth approxima-
tion. It is interesting tha t the effect appears to be
cumulative as the ripple is not apparent after three
stages but is quite marked a t the 14th stage.
The effect of improving the accuracy of the
density approximation by including more terms in
the
a p r i o r i
representations and by retaining more
terms for the
a p o s t e r i o r i
representation can be seen
in Fig. 5. Twenty terms are included in the
a p r i o r i
densities and 5~ and 52 are reduced to 0.0001. The
figure presents the actual
a p o s t e r i o r i
density, the
gaussian sum approximation, and the gaussian
approximation provided by the Kalman filter
equations. Comparison of the results for stages 3
and 14 with Fig. 4 indicates the improvements in
the approximation that have occurred.
- - T r u e pr o b a bi l it y d e n si ty
. . . . . Kalma n approximation
G a u s s i a n s u m ap p r ox i ma t i on
4 . C O N C L U S I O N S
The approximation of density functions by a
sum of gaussian densities has been discussed as a
reasonable framework within which estimation
policies for non-linear and/or nongaussian stochastic
systems can be established. It has been shown that
a probability density function can be approximated
arbitrarily closely except at discontinuities by such
a gaussian sum. In contrast with the Edgeworth or
Gram-Charlier expansions that have been in-
vestigated earlier, this approximation has the
advantage of converging to a broader class of
density functions. Furthermore, any finite sum of
these terms is itself a valid density function.
The gaussian sum approximation is a departure
from more classical approximation techniques
because the sum is restricted to be positive for all
possible values of the independent variable. As a
result, the series is not orthogonalizable so tha t the
manner in which parameters appearing in the sum
are chosen is not obvious. Two numerical pro-
cedures are discussed in which certain parameters
are chosen to satisfy constraints or are somewhat
arbitrarily selected and others are chosen to mini-
mize the
k
error.
It is anticipated that the gaussian sum approxima-
tion will find its greatest application in developing
estimation policies for non-linear stochastic systems.
However, many of the characteristics exhibited by
non-linear systems and some of the difficulties in
using the gaussian sum in these cases are exhibited
by treating linear systems with nongaussian noise
0 5 0 F . . 2 O F 0 . 7 5 I - 0 ' 5 0 I -
L l b . . l . . . . .
. . . . u _ - - - ' '
02 5 ~ . - - '~ - "m-- -~- I o I0 c -, . . a .- 0 25
/ . . . . . . . . . . . . . . . . . .
_
o , : . , o l . . . . . ; i o o - i o [ . ' ' ,
-~
- 2 - 5 0 0 2 5 0 2 0 1 .5 0 5 ' 0 0 0 2 . 5 0 5 . 0 0
x X X X
0 . 5 0 F . . . . I 0 - 0 - 5 O f - 0 5 0 [ - . ' . .
. . . . . t - b - , , t -
0 .~ :5 . .' ~ _ , ~ l 1 . ~ % 2 5 t - . .7 . t ~ c ~ t - I ,' . . ' ' - '- ' ~ , , ,
o , . . . , J , ' l _ , o l . . . . . . . . . : / , , I % o 1 . ; ' 1 , ,
- I 0 0 i 5 0 - 2 5 0 ' , 0 0 5 0 0 0 2 . 5 0 5 0 0 1 .5 0 4 - 0 0 6 - 5 0
X X X X
0 . 5 0[ 0 5 0 [ 0 5 0 l I 0 [
1 ' 50 4 ' 0 0 6 ' 5 0 0 2 5 0 0 0 1 .5 0 4 0 0 2 0 4 0 6 0
X X X X
5 r l - . . --~-'~m-q/ k 5 r 5 1 - I f ~
o L , ~ f 1 I I I f ' . , o l , . d " ~ I
r
r - . , , o L. ,L " ~ I I I t z ' ~ oi ~ i i t i" L'~---.t
0 2 5 0 5 0 0 0 2 5 0 5 . 0 0 C ,'~ O 2 . 0 0 4 - 5 0 i O 4 0 5 . 0. 6 0
X X X X
Fie 5 A p o s te r io r i d e n s i t y f o r s i x t e e n s t a g e s .
-
8/11/2019 Recursive Bayesian Estimation Using Gaussian Sums
14/15
478 H . W. SORENSON an d D. L. ALSPACH
sources. Of course, whe n the noise is entirely
gaussian the problem degenerates and the familiar
Kalman filter equations are obtained as the exact
solut ion of the problem.
The l inear, nongauss ian est imation problem is
discussed and it is shown that, if the
a priori
density
functions are represented as gaussian sums, then
the number of terms required to describe the a
posteriori
density is equal to the product of the
number of terms of the
a priori
densities used to
form it. The appar ent disa dvanta ge is seen to cause
little difficulty however, b ecause the m omen ts of
many individual terms converge to com mon values
which allows them to be combi ned. Furth er, the
weighting factors associated with man y other terms
become very small and permit those terms to be
neglected without introducing significant error to
the approximation.
Numerical results for a specific system are
presented which provide a demonstrat ion of some
of the effects discussed in the text. These results
confirm dramatical ly that the gaussian sum approxi-
mation can provide considerable insight into
problems that hi therto have been intractable to
analysis.
REFERENCES
[1] A. H. JAZWINSKI: Stochastic Processes and Filtering
Theory.
Academic Press, New York (1970).
[2] R. E. KALMAN: A new approach to l inear filtering and
prediction problems.
J. bas. Engng
82D, 35--45 (1960).
[3] H. W. SORENSON
Advances in Control Systems
Vol. 3,
Ch. 5. Academic Press, New York (1966).
[4] R. COSAERT and E. GOTTZEIN: A decoupled shifting
memory filter method for radio tracking of space
vehicles. 18th International Astronautical Congress,
Belgrade, Yugoslavia (1967).
[5] A. JAZWINSKI: Adaptive filtering. Automatica 5 475-
485 (1969).
[6] Y. C. Ho and R. C. K. LEE: A Bayesian approach to
problems in stochastic estimation and control.
IEEE
Trans. Aut. Control
9, 333-339 (1964).
[71 H. J. KUSHNER: On the differential equations satisfied
by conditional probability densities of Markov pro-
cesses.
SIAMJ. Control
2, 106-119 (1964).
[8] J. R. FmrlER and E. B. STEAR: Optimal non- linear
filtering for independent increment processes, Parts I
and II.
IEEE Trans. Inform. Theory
3, 558-578 (1967).
[9] M. AOKI:
Optimization of Stochastic Systems Topics
in Discrete-Time Systems.
Academic Press, New York
(1967).
[10] H. W. SORENSON and A. R. STOBBERUD: Nonlinear
filtering by approximation of the
a posteriori
density.
Int. J. Control 18, 33-51 (1968).
[11] M. AOKI: Optimal Bayesian and rain-max control of a
class of stochastic and adaptive dynamic systems.
Pro-
ceedings IFA C Symposium on Systems Engineering for
Control System Design
Tokyo, pp. 77-84 (1965).
[12] A. V. CAMERON: Control and estimation of linear
systems with nongaussian
a priori
distributions.
Pro-
ceedings of the Third Annual Conference on Circuit and
System Science
(1969).
[13] J. T. Lo: Finite dimensional sensor orbits and optimal
nonl inear filtering. University of Southern California.
Report USCAE 114, August 1969.
[14] J. KOREYAAR: Mathematical Methods Vol. 1, pp. 330-
333. Academic Press, New York (1968).
[l 5] W. FELLER: An Introduction to Probability Theory and
Its Applications
Vol. lI, p. 249. John Wiley, New York
(1966).
[16] J. D. SI'RAOINS: Reproducing distributions lbr machine
learning. Standard Electronics Laboratories, Technical
Report No. 6103-7 (November 1963).
[17] D. L. ALST'ACH: A Bayesian approximation technique
for estimation and control of time-discrete stochastic
systems. Ph.D. Dissertation, University of Cahfornia,
San Diego (1970).
R6sum~--Les relations recurrentes bayesiennes qui decrivent
le comportement de la fonction de densit6 de probabilit6
a posteriori de l'6 tat d'un syst6me al6atoire, discret dans le
temps, en se basant sur les donn6es des mesures disponibles,
ne peuvent ~tre g6n6ralement resolues sous une forme fermee
lorsque le syst6me est soit non-lin6aire, soit non-gaussien.
ke present article introduit et propose une approximation de
densit6, mettant en jeu des combinaisons convexes de fonc-
tions de densit6 gaussiennes, ~t titre de m6thode efficace pour
6viter les difficultes rencontr6es dans l'6valuat ion de ces
relations et dans l'utilisation des densit6s en r6sultant pour
determiner des strat6gies d'6valuation particuli6res. II est
montr6 que, lorsque le nombre de termes de la somme
gaussienne augmente sans limites, l'approximation converge
uniformement vers une fonction de densit6 quelconque dans
une cat6gorie 6tendue. De plus, toute somme finie est elle-
m6me une fonetion de densit6 valable, d'une mani6re
diff6rente des nombreuses autres approximations 6tudi6es
dans le pass6.
Le probl6me de la determination des estimations de la
densit6 a posteriori et de la variance minimale pour des
syst~mes lin~aires avee bruit non-gaussien est trait6 en
utilisant l'approximation des sommes gaussiennes. Ce
probl6me est 6tudi6 parce qu 'il peut ~tre trait6 d'une mani~re
relativement simple en utilisant l'approximation mais
cont ient encore la plupart des difficult~.s rencontr6es en
consid6rant des syst+mes non-lin6aires, puisque la densit6
a posteriori est non-gaussienne. Apr6s la discussion du
probl6me g6n6ral du po int de vue de l'applicationdes sommes
gaussiennes, l'article pr6sente un exemple num6rique dans
lequel les statistiques r6611es de la densit6
a posteriori
sont
compar6es avec les valeurs pr6dites par les approximations
des sommes gaussiennes et du filtre de Kalman.
Zusammenfassung--Die Bayes'schen Rekursionsbezie-
hungen, die das Verhalten der a priori-Wahrscheinlichkeits-
dichtefunktion des Zustandes eines zeitdiskreten stochasti-
schen Systems beschrieben, k6nnen nicht allgemein in
geschlossener Form gel6st werden, wenn das System ent-
weder nichtlinear oder nicht-gaussisch ist. In dieser Arbeit
wird eine Dichteapproximation, die konvexe Kombina-
tionen von Gauss'schen Dichtefunktionen enth~ilt, eingefiihrt
und als bedeutungsvoUer Weg zur Umgehung der Schwierig-
keiten vorgeschlagen, die sich bei der Auswertung dieser
Beziehungen und bei der Benutzung der resultierenden
Dichten ergeben, um die spezifische Schiitzstrategie zu
bestimmen. Ersichtlich konvergiert, da die Zahl der Terme
in der Gauss'schen Summe unbegrenzt w~ichst, die Approxi-
mation gleichm~il~igzu einer Dichtefunktion in einer um-
fassenden Klasse. Weiter ist eine endliche Summe selbst eine
gfiltige Dichtefunktion, anders als viele andere schon
untersuchte Approximationen.
Das Problem der Bestimmung der Sch~ttzungen der a
posteriori-Dichte und des Varianzminimums ftir lineare
Systeme mit nichtgauss'schem Ger~iusch wird unter Behand-
lung der Gauss'schen Summenapproximation behandelt.
Dieses Problem wird betrachtet, well es in einer relativ
geraden Art unter Benutzung der Approximation behandelt
werden kann, abet immer noch die meisten der Schwierig-
keiten enth~ilt, denen man bei der Betrachtung nichtlinearer
Systeme begegnet, da die a posteriori-Dichte nichtgaussisch
ist. Nach der Diskussion des allgemeinen Problems vom
Gesichtspunkt der Anwendung Gauss'scher Summen, wird
ein Zahlenbeispiel gebracht, in dem die aktuellen Statistiken
der a posteriori-Dichte mit den Werten verglichen werde,
die durch die Gauss'sche Summe und durch die Kalman-
Filter-Approximationenvorhergesagt wurden.
-
8/11/2019 Recursive Bayesian Estimation Using Gaussian Sums
15/15
Recursive Bayesian estimation using gaussian sums 479
Pe~IOMe---PeKypcHBHMe
6a~eco m : r ~e
BMpa~eHH~ OnHCM-
aalottlne nOBelIeHHe anocTepHopHoi~ ~ymmH nnOTHOCTH
BepO~ITHOCTH COCTOIIHH~Icay~aI~o~ CHCTeMbI, ]IHCI{peTHO~
n o
SpeMetlrl, OCHOSl,magcl, Ha ~ocTynm~Ix pe3ynbTaTax
H3MepeHgl~, He M O F y T 6MTI, O61,DIHO peii ieHl , B 3aMKHyTOI~
dpOpMe Kor ea CHCTeMa HH60 HCHHHe]~Ha, dIH6 0-~e He
ssnaerc~ raycco~croit HaCTOSi~a~
CTaTb~ BBO~HT H
npemmraer
npH6HH~KeHHe ILrlOTHOCTH,
npa6craromcc
K
COqeTallI~lM
ayccoBCKHX dpyHKl1Hlt HHOTHOCTH, B Ka~ICCTBe
3Op~eKTHBHOFO MeTo~Ia ~n a H3~KaHHfl 3aTpy~HeHHl~
BCTpeqaeMblX B OI~eHKe 3THX Bblpa~KeHH~ H B HCHO.rlb3OBaHHH
B],~TeKaIOILmX H3 HHX HJIOTHOCTeI'~ ~II~1 onpe~eHH~ ~aHHblX
CTaTerH.fi OReHKH. I'IoKa3MBaeTC~ qTO, KOFRa ~IHCHO
~IeHOB raycco~croit CyMMbI 6ecnpe~eH~HO Bo3pacTaeT,
rlpH6nH~KeHHe paBHOMepHo CTpeMrlTC~I K n/o6 o~ t (I~yHKI~HH
ILrIOTHOCTH B IIIHpOKOM KJlaCC~. CB ep x ~TOFO, BCaKa~I
KOHeqHall CyMMa caM a ~IBJl~leTCfl rlpHeMneMOl'~ ~yHKHHeI~
IIHOTHOCTH~ B OTJIPIqI4H OT MHOFO~4~HCHeHH]bIX ~Ip yr tl x
npH6Jm~cermll H3y~aBmHXC~ S np omn oM.
I-[po6J1eMa oHpo~eJ1erm~ OHOHOK anOCTOpHOpHOfl rmO-
THOCTII H MHHHMaYlbHO~ BapHaHTHOCTH ~flH HHHCflHbIX
CHCTeM C IIo-rayccoBCIOIM IIIyMOM pemaeTcf l c n oMo tI~r O
npH6nrmcon~ rayccoBcrdax CyMM. ~Ta npo6fleMa n3y-
tlaeTCfl IIOTOMy qTO OHa MO)KeT 6bITb peI iie Ha cpaBHll Te-
JlbHO npOCTO Hcno.rtl,3y~ n pH fm , xcerme HO co~lep~crlT e me
60JIBIIIHHCTBO 3aTpy~IHeHni~ BCTpeqaeMI, IX pacCMaTpHBa~
HeaaHeianbie CHCTeMbI, H60 ario crepa opria a nn or ao cr b
aa ~e wc s He-raycc oBczoit Hoc ne o6cy:~cjXeHaa o6IUeR
npo6H eMb l C TOqKH 3peHHll HpHMeHeHH~I rayCCOBCKHX
CyMM~ CTaTl~fl ~laeT qHCJIOBOI~ rlpltMep B KOTOpOM ~el~CTBHTe-
IlbHble CTaTHCTHKH anO cTepr lopHo lt ILrlOTHOCTH cpaBHHBalO-
TCfl C 3HaLIeHH~VIH Hpe~cKa3at-IHblMrl HpH6HH)KeHri~Mri
rayc coac Kax CyMM H ~HHbTpa KanMana.