Post on 05-Aug-2020
Summer Institute on Probablility in AI 1994
Inference 2 1
Alg
orit
hms
Par
t II
: C
urre
nt D
irec
tion
s
Bru
ce D
’Am
bros
io a
nd R
ober
t F
ung
Summer Institute on Probablility in AI 1994
Inference 2 2
Intr
oduc
tion
lC
ompu
tati
onal
bre
akth
roug
h of
Bay
es N
ets
is e
xplo
itat
ion
of
cond
itio
nal i
ndep
ende
nce
capt
ured
gra
phic
ally
.
lR
esea
rch
dire
ctio
ns•
expl
oit
addi
tion
al s
truc
ture
•pu
sh e
xpre
ssiv
ity
lSt
ruct
ure
expl
oita
tion
•
Noi
sy-O
rs•
Asy
mm
etri
es (
e.g.
, Sim
ilari
ty N
etw
orks
)
•E
ntro
py
lE
xpre
ssiv
ity
•C
onti
nuou
s va
riab
les
Summer Institute on Probablility in AI 1994
Inference 2 3
Out
line
lR
epre
sent
atio
n•
Noi
sy O
r
•A
sym
met
ries
•C
onti
nuou
s V
ars
lA
ppro
xim
atio
n•
Sim
ulat
ion
•Se
arch
Summer Institute on Probablility in AI 1994
Inference 2 4
Noi
sy O
r
lD
istr
ibut
ion
size
is e
xpon
enti
al in
num
ber
of p
aren
ts•
Dif
ficu
lt t
o ac
quir
e•
Exp
ensi
ve t
o co
mpu
te
lN
oisy
Or
inte
ract
ion
mod
el•
Fin
ding
abs
ent
if a
ll pa
rent
s ab
sent
•pa
rent
con
trib
utio
ns in
depe
nden
t
•w
idel
y us
ed f
or m
ulti
-par
ent
inte
ract
ions
.
lR
epre
sent
atio
ns•
ladd
er m
odel
•lo
cal e
xpre
ssio
nsF2
D3
D2
D1
F1
F4
F3
F5 D4
Summer Institute on Probablility in AI 1994
Inference 2 5
Noi
sy O
r II
lM
odel
impl
ies:
•P
(~f3
|D1,
D2,
D3)
= C
(~f3
|D1)
*C(~
f3|D
2)*C
(~f3
|D3)
•w
here
C(~
f1|D
1) =
P(~
f1|D
1,~d
2,~d
3)•
=
1, D
1=~d
1; (
1-P
(f1|
d1,~
d2,~
d3))
, D1=
d1
•P
(f3|
D1,
D2,
D3)
= 1
-P(~
f3|D
1,D
2,D
3) F2
D3
D2
D1
F1
F4
F3
F5 D4
Summer Institute on Probablility in AI 1994
Inference 2 6
Qui
cksc
ore
lB
asic
exp
ress
ion
is e
xpon
enti
al in
cau
ses
lR
earr
ange
pos
teri
or e
xpre
ssio
n fo
r ef
fici
ent
eval
uati
on
(Hec
kerm
an 8
9):
•L
inea
r in
cau
ses
•L
inea
r in
neg
ativ
e fi
ndin
gs•
Exp
onen
tial
in p
osit
ive
find
ings
lD
oesn
’t t
ake
adva
ntag
e of
top
olog
ical
str
uctu
re
Summer Institute on Probablility in AI 1994
Inference 2 7
Qui
cksc
ore
- N
egat
ive
Fin
ding
s
lP
(D1)
= Σ
D2 P
(F1|
D1,
D2)
*P(F
2|D
1,D
2)*P
(D1)
*P(D
2)l
but
F1
nega
tive
, so:
P(F
1|D
1,D
2) =
C(~
f1|D
1)*C
(~f1
|D2)
lth
eref
ore
•P
(D1)
= Σ
D2C
(~f1
|D1)
*C(~
f1|D
2)*C
(~f2
|D1)
*C(~
f2|D
2)*P
(D1)
*P(D
2)
lre
arra
ngin
g:
•P
(D1)
= (
C(~
f1|D
1)*C
(~f2
|D1)
*P(D
1))
* (Σ
D2C
(~f1
|D2)
*C(~
f2|D
2)*P
(D2)
lL
inea
r in
neg
ativ
e fi
ndin
gs a
nd d
isea
ses
F2
D3
D2
D1
F1
F4
F3
F5 D4
Summer Institute on Probablility in AI 1994
Inference 2 8
Qui
cksc
ore
- P
osit
ive
Fin
ding
s
lP
(D1)
= Σ
D2 P
(F1|
D1,
D2)
*P(F
2|D
1,D
2)*P
(D1)
*P(D
2)l
But
F1
Pos
itiv
e, s
o: P
(F1|
D1,
D2)
= 1
- C
(~f1
|D1)
*C(~
f1|D
2)l
Subs
titu
ting
:
•P
(D1)
= Σ
D2(
1 -
C(~
f1|D
1)*C
(~f1
|D2)
) *
(1 -
C(~
f2|D
1)*C
(~f2
|D2)
) *
P(D
1) *
P(D
2)
ldi
stri
buti
ng o
ver
F1,
F2:
• Σ
D2
(1*1
*P(D
1)*P
(D2)
-C
(~f1
|D1)
*C(~
f1|D
2)*1
*P(D
1)*P
(D2)
•
-1*C
(~f2
|D1)
*C(~
f2|D
2)*P
(D1)
*P(D
2)•
+C
(~f1
|D1)
*C(~
f1|D
2))
* C
(~f2
|D1)
*C(~
f2|D
2) *
P(D
1) *
P(D
2))
lre
arra
ngin
g:
•P
(D1)
*ΣD
2P(D
2) -
C(~
f1|D
1)*P
(D1)
*ΣD
2(C
(~f1
|D2)
*P(D
2))
•
- C
(~f2
|D1)
*P(D
1)*Σ
D2(
C(~
f2|D
2)P
(D2)
)
•
+ C
(~f1
|D1)
C(~
f2|D
1)P
(D1)
*ΣD
2(C
(~f1
|D2)
*C(~
f2|D
2)P
(D2)
)
Summer Institute on Probablility in AI 1994
Inference 2 9
Lad
der
Mod
el
lH
ecke
rman
93
lP
(F12 )
= P
(F1|
D1
only
)
lP
(F11
| F12
, D2)
:
• 1
- F
12 pr
esen
t
•P
(F1|
D2
only
) -
F12
abse
nt
F2
D3
D2
D1
F1
F4
F3
F5 D4
F1
F12
F11
F2
F22
F21
D1
D2
Summer Institute on Probablility in AI 1994
Inference 2 10
Lad
der
mod
el, n
egat
ive
evid
ence
lN
egat
ive
evid
ence
is a
sser
ted
at e
ach
Fx n
ode!
•no
te t
his
mea
ns r
e-w
riti
ng c
hild
dis
trib
utio
ns
•P
’(F
11* |
D2)
= P
(F11
* | D
2, F
12* )
lC
lear
ly s
how
s ne
gati
ve f
indi
ng d
ecou
ples
cau
ses.
F1
F12
F11
F2
F22
F21
D1
D2
D2
F1
2 P
(F11
| D
2, F
12 )
A
bs
Pre
s
Ab
s A
bs
1
.0 0
.0P
res
Ab
s
.4
.6
A
bs
Pre
s
0.0
1.0
Pre
s P
res
0
.0 1
.0
D2
P
’(F1
1 * |
D2)
Ab
sA
bs
1.0
Pre
s
.
6
Summer Institute on Probablility in AI 1994
Inference 2 11
Lad
der
mod
el, P
osit
ive
Evi
denc
e
lP
ost
posi
tive
evi
denc
e on
ly a
t te
rmin
al n
odes
lC
lear
ly li
near
in d
isea
ses
F1
F12
F11
F2
F22
F21
D1
D2
F12/F22
D1
D2
F11/F21
F1/F2
Summer Institute on Probablility in AI 1994
Inference 2 12
Loc
al E
xpre
ssio
ns
lSy
mbo
lic a
lgeb
ra c
an f
acto
r dy
nam
ical
lyl
Ord
er in
whi
ch w
e di
stri
bute
ove
r ev
iden
ce m
atte
rs
F2
D3
D2
D1
F1
F4
F3
F5
D4
Cas
e P
os E
v S
avin
g
1
2
9
1
1
224
5
320
1
423
4
523
5
622
4
722
3
824
7
923
3
1019
0
Summer Institute on Probablility in AI 1994
Inference 2 13
Res
earc
h in
Noi
sy O
r
lG
ener
aliz
ed n
oisy
or
(Sri
niva
s 93
)l
Mul
ti-v
alue
lM
ulti
-lev
el n
etw
orks
(C
PC
S, e
xten
ded
set
fact
orin
g)l
Em
bedd
ing
nois
y or
in g
ener
al n
etw
orks
Summer Institute on Probablility in AI 1994
Inference 2 14
Asy
mm
etri
es
lR
epre
sent
atio
n:•
Val
ue d
epen
dent
inde
pend
ence
•P
(Z|x
,Y)
= .3
•C
onti
ngen
t ex
iste
nce
lIn
fere
nce:
•E
ffic
ient
exp
loit
atio
n of
asy
mm
etry
lD
’Am
bros
io 9
1, H
ecke
rman
and
Gei
ger
93, P
oole
94,
She
noy
??, F
ung
and
Shac
hter
, Bar
low
, DA
rel
ated
lIn
fere
ntia
l cos
t/be
nefi
t un
clea
r
Bet
?
Ret
urn
Val
ue
Summer Institute on Probablility in AI 1994
Inference 2 15
Con
tinu
ous
Var
iabl
es
lSi
gnal
to
sym
bol g
ap:
a m
ajor
em
barr
asm
ent
for
AI
lSo
me
succ
ess
in m
ixed
dis
cret
e/co
ntin
uous
bel
ief
nets
lG
ener
al m
odel
s/si
mul
atio
n-ba
sed
(pre
dict
ive)
• D
emos
(H
enri
on)
lE
xact
, lin
ear
wit
h di
scre
te p
rede
cess
ors
(dia
gnos
tic)
•H
ugin
(Je
nsen
, Ole
sen)
•SP
I (C
hang
and
Fun
g)
lC
onti
nuou
s In
flue
nce
Dia
gram
s (K
enle
y, P
olan
d)
Bea
rin
gC
omm
and
Bac
kp
ress
ure
RP
M
Summer Institute on Probablility in AI 1994
Inference 2 16
App
roxi
mat
ion
lE
xact
Inf
eren
ce is
Har
dl
App
roxi
mat
ion
tech
niqu
es:
•A
ppro
xim
ate
repr
esen
tati
on•
Qua
litat
ive
(Gol
dsch
mid
t, P
earl
, Wel
lman
, ...)
•R
educ
ed m
odel
(B
rees
e, H
ecke
rman
, Hor
vitz
)
•Ig
nori
ng w
eak
depe
nden
cies
(Je
nsen
)
•A
ppro
xim
ate
infe
renc
e•
Sim
ulat
ion
(Pea
rl, F
ung,
Peo
t, ..
.)
•Se
arch
(D
’Am
bros
io, H
enri
on, P
oole
)•
Loc
al e
valu
atio
n (D
rape
r)
Summer Institute on Probablility in AI 1994
Inference 2 17
Bac
kwar
d Si
mul
atio
n
oA
new
alg
orit
hm f
amily
for
pro
babi
listi
c in
fere
nce
in B
ayes
ian
Net
wor
ks.
oA
ddre
sses
tw
o lim
itat
ions
of
curr
ent
sim
ulat
ion
met
hods
(for
war
d, m
arko
v bl
anke
t)�
– s
low
con
verg
ence
whe
n fa
ced
wit
h lo
w-l
ikel
ihoo
d ev
iden
ce�
– s
low
con
verg
ence
wit
h de
term
inis
tic
rela
tion
ship
s�
oN
o m
agic
wit
h re
spec
t to
com
plex
ity
resu
lts.
Summer Institute on Probablility in AI 1994
Inference 2 18
For
war
d Sa
mpl
ing
Exa
mpl
e
D1
E1
D2
D3
E2
D4
lP
ossi
ble
Ord
er•
D1,
D2,
D3,
D4
lW
eigh
t•
P(E
1|d1
)*P
(E2|
d3)
Summer Institute on Probablility in AI 1994
Inference 2 19
Bac
kwar
d Si
mul
atio
n: S
impl
e E
xam
ple
P(X
=x0)
/P(X
=x1)
= 1
0-n
P(E
|X=x
0)/P
(E|X
=x1)
= 1
0m
m >
> n
P(X
=x0|
E)/
P(X
=x1|
E)
= 10
m-n
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.91
X E
tria
ls
pos
teri
orP
(x=
0)
Summer Institute on Probablility in AI 1994
Inference 2 20
Bac
kwar
d Si
mul
atio
n: A
lgor
ithm
lO
rder
ing
•A
nod
e m
ust
be in
stan
tiat
ed t
o be
sam
pled
•T
he u
nion
of
the
pred
eces
sors
of
the
node
s in
the
ord
er m
ust
cove
r al
l the
nod
es in
the
net
wor
k
lSa
mpl
ing
•T
akes
pla
ce f
rom
nor
mal
ized
like
lihoo
ds a
nd s
ets
the
pred
eces
sor
valu
es•
For
war
d sa
mpl
ing
is u
sed
for
node
s w
ith
no d
owns
trea
m
evid
ence
lW
eigh
ting
•Im
port
ance
sam
plin
g•
the
rati
o of
the
pri
or p
roba
bilit
y an
d th
e sa
mpl
ing
prob
abili
ty
Summer Institute on Probablility in AI 1994
Inference 2 21
Bac
kwar
d Si
mul
atio
n:
Wei
ghti
ng E
quat
ion
Z(x)
=P(
x i|xPa
(i))
i∈N∏
P(x j|x
Pa(j
))
P(x j|x
Pa(j
))x Pa
(j)∈
XPa
(j)
∑j∈
Nbo∏
P(x)
=P(
x i|xPa
(i))
i∈N∏
P s(x)=
P(x j|x Pa
(j))
P(x j|x Pa
(j))
x Pa(j
)∈X P
a(j)
∑j∈
N bo∏
Z(x)
=P(
x)P s(x
)
Summer Institute on Probablility in AI 1994
Inference 2 22
Bac
kwar
d Si
mul
atio
n: E
xam
ple
lSi
mul
atio
n ou
t fr
om t
he e
vide
nce
lSa
mpl
ing
sets
a n
ode'
s pr
edec
esso
rs
D1
E1
D2
D3
E2
D4
• P
ossi
ble
Ord
ers
° E
2, E
1, D
2',
D4'
° E
1, D
2',
E2,
D4'
Summer Institute on Probablility in AI 1994
Inference 2 23
Dis
cuss
ion
lIm
prov
ed c
onve
rgen
ce in
low
-lik
elih
ood
situ
atio
nsl
Com
puta
tion
al c
osts
of
norm
aliz
atio
n•
cost
s ca
n be
red
uced
thr
ough
: pr
ecom
puta
tion
and
/or
cach
ing
lIn
tegr
atio
n of
bac
kwar
ds a
nd f
orw
ards
sam
plin
gl
Gro
upin
g of
nod
es f
or s
ampl
ing
lD
ynam
ic n
ode
orde
ring
lH
andl
ing
cont
inuo
us d
eter
min
isti
c re
lati
ons
thro
ugh
func
tion
in
vers
ion.
Summer Institute on Probablility in AI 1994
Inference 2 24
Bac
kwar
d Si
mul
atio
n: S
tatu
s
lH
ave
“van
illa”
impl
emen
tati
on w
orki
ng in
ID
EA
Ll
Hav
e w
orke
d ou
t Q
MR
(e.
g., n
oisy
-or)
for
mul
atio
nl
Cur
rent
ly w
orki
ng o
n Q
MR
impl
emen
tati
on
Summer Institute on Probablility in AI 1994
Inference 2 25
Inst
anti
atio
n C
ontr
ol
lW
hy s
tric
tly
forw
ard
or b
ackw
ard?
lP
arti
al in
stan
tiat
ion
mak
es d
istr
ibut
ion
“inf
orm
ativ
e”l
Ent
ropy
and
oth
er in
form
atio
n-th
eore
tic
mea
sure
s l
Stat
ic v
s D
ynam
ic s
elec
tion
Summer Institute on Probablility in AI 1994
Inference 2 26
Sear
ch
lG
oal:
est
imat
e qu
ery
by c
ompu
ting
mas
s of
larg
e in
stan
tiat
ions
:•
find
inst
anti
atio
n w
ith
max
imum
a-p
oste
rior
i pro
babi
lity.
•re
peat
unt
ill s
uffi
cien
t m
ass
com
pute
d.
lM
etho
d: I
ncre
men
tally
ext
end
an in
stan
tiat
ion
usin
g lo
cal
sear
ch.
lD
’Am
bros
io:
incr
emen
tal p
roba
bilis
tic
infe
renc
e
lH
enri
on:
sear
ch in
larg
e B
N2O
net
sl
Poo
le:
use
of c
onfl
ict
sets
in s
earc
h
Summer Institute on Probablility in AI 1994
Inference 2 27
Sear
ch -
Sim
ple
Exa
mpl
e
A
B C
D
P(C|A)
A t f
t .6 .4
f .5 .5
P(B|A)
A t f
t .8 .2
f .7 .3
P(D|B,
B C t
t t .58
t f .56
f t .54
f f .52
*
P(D|B,C)
P(C|A)
P(B|A)
P(A)
*
*
Σ
Σ
A
B,C
P(D) = Σ P(D|B,C)*
Σ P(C|A)*P(B|A)*P(A)
= .58*.6*.8*.9 (.25056 -> D=t
.56*.8*.6*.1 (.02688 -> D=
B,C
P(A)
t f
.9 .1
A
Summer Institute on Probablility in AI 1994
Inference 2 28
Sear
ch
lL
ike
sim
ulat
ion,
inst
anti
ate
the
vari
able
s in
one
dis
trib
utio
n at
a t
ime
lU
se h
euri
stic
sea
rch
tech
niqu
es t
o gu
ide
inst
anti
atio
n pr
oces
s
lW
hy?
•D
irec
t so
luti
on o
f M
LC
H (
adm
issa
ble
sear
ch r
equi
red!
)•
Few
larg
e te
rms
mig
ht b
e in
form
ativ
e•
skew
ed -
> n+
1 sc
enar
ios
cont
ain
2/e
mas
s!
lH
ow?
•f(
i) =
g(i
) *
h(i)
•g(
i) -
like
lihoo
d of
par
tial
sce
nari
o•
h(i)
- h
euri
stic
est
imat
e of
max
pro
b of
rem
anin
g va
rs g
iven
in
stan
tiat
ion
so f
ar.
lB
ut:
perf
orm
ance
can
be
poor
•L
arge
sea
rch
spac
e•
exte
nsiv
e ba
cktr
acki
ng
Summer Institute on Probablility in AI 1994
Inference 2 29
Sear
ch -
fir
st e
xam
ple
lM
odel
: A
, C, A
’, C
’, A
’’, C
’’ lo
w e
ntro
pyl
Supp
ose
we
know
I -
I’’
all
zero
, O, O
’ ze
ro, O
’’ 1
lSe
arch
sta
rts
wit
h A
, C, A
’, C
’, A
’’, C
’’ a
ll in
hig
h pr
obab
ility
sta
te (
ok)
l0
mas
s w
hen
we
incl
ude
Ol
Now
wha
t?
•P
oor
sear
ch a
rchi
tect
ure
can
yiel
d ex
pone
ntia
l tim
e
I0I1
CAO
I0’
I1’
C’
A’
O’
I0’’
I1’’
C’’
A’’
O’’
Summer Institute on Probablility in AI 1994
Inference 2 30
Poo
le -
Con
flic
ts in
Sea
rch
lC
an d
eriv
e a
conf
lict
invo
lvin
g A
’’ f
rom
fir
st s
earc
h.l
P(V
|O)
:= M
axim
um P
roba
bilit
y of
any
ass
ignm
ent
to
a co
nflic
t co
ntai
ning
var
s V
.
lh(
i) =
Π P
(V|O
) o
ver
all s
ubse
quen
t in
depe
nden
t co
nflic
ts.
lQ
uick
ly r
evea
ls A
, A’
not
wor
th w
orki
ng o
n.l
Poo
le -
UA
I 92
, UA
I-93
I0I1
CAO
I0’
I1’
C’
A’
O’
I0’’
I1’’
C’’
A’’
O’’
Summer Institute on Probablility in AI 1994
Inference 2 31
Con
flic
ts in
Sea
rch
lC
onfl
ict
set:
{C
’, A
’’}
lh
for
any
part
ial t
erm
not
incl
udin
g C
’ or
A’’
is m
ax P
(c’,
a’’
) co
nsis
tent
wit
h ob
s.
lh
for
any
part
ial t
erm
incl
udin
g C
’ or
A’’
is 1
.0
I0I1
CAO
I0’
I1’
C’
A’
O’
I0’’
I1’’
C’’
A’’
O’’
Summer Institute on Probablility in AI 1994
Inference 2 32
D’A
mbr
osio
- f
acto
ring
and
cac
hing
lB
uild
Eva
luat
ion
tree
.l
Top
dow
n, le
ft t
o ri
ght
sear
ch, b
ut c
ache
res
ults
lA
fter
fir
st f
ail,
AN
Y in
stan
tiat
ion
of le
ft s
ubtr
ee w
ith
sam
e C
’ ha
s sa
me
h.
lU
nlik
e P
oole
, fee
ds b
ack
all i
nfo,
not
jus
t co
nflic
ts (
0s)
lB
ut d
oesn
’t g
ener
aliz
e ov
er c
onfl
ict
set
lD
’Am
bros
io:
DX
-92,
UA
I-93
AC
*
*
O
A’
C’
*
*
O’
A’’
C’’
*
*
O’’
*
*
Summer Institute on Probablility in AI 1994
Inference 2 33
Wha
t is
sea
rch
good
for
?
lM
LC
Hl
Pos
teri
or e
stim
atio
nl
Pol
icy
esti
mat
ion
lA
ssum
ptio
ns:
•lo
okin
g fo
r a
few
goo
d in
stan
tiat
ions
•ea
sy t
o fi
nd
•m
ore
is b
ette
r
Summer Institute on Probablility in AI 1994
Inference 2 34
Exp
erim
ents
0.8
90.9
0.9
10.9
20.9
30.9
40.9
50.9
60.9
70.9
8
010
20
30
40
Terms Computed
(MLCH, System OK)
Summer Institute on Probablility in AI 1994
Inference 2 35
Exp
erim
ents
III
0
0.00
2
0.00
4
0.00
6
0.00
8
0.01
0.01
2
05
1015
2025
30
Terms
(MLCH, Single Fault)
Summer Institute on Probablility in AI 1994
Inference 2 36
Cos
t P
er F
ailu
re
12
17
22
27
32
37
42
11
01
00
10
00
Summer Institute on Probablility in AI 1994
Inference 2 37
Sear
ch in
Noi
sy O
R
lT
opN
(H
enri
on, 8
9)l
Aga
in t
he c
ruci
al a
ssum
ptio
n:•
“...o
nly
a ti
ny f
ract
ion
of t
hem
acc
ount
for
mos
t of
the
pr
obab
ility
mas
s.”
lR
(h,F
) =
P(h
|F)/
P(h
0|F)
lM
EP
(d,H
) =
R(H
ud, F
)/R
(H,F
)•
if M
EP
(d,H
) <
1, d
on’t
add
d y
et
lM
EP
(d,H
) >
ME
P(d
,H+)
•if
it is
n’t
wor
th it
to
add
d no
w, i
t ne
ver
will
be
lA
lgor
ithm
:•
star
t w
ith
null
hypo
thes
is•
cons
ider
all
d w
ith
ME
P >
1
•pi
ck b
est
ME
P, a
dd, e
xpan
d re
sult
ing
hypo
thes
is•
loop
Summer Institute on Probablility in AI 1994
Inference 2 38
Sear
ch -
Ope
n is
sues
lP
roba
bilit
y co
mm
unit
y ah
ead
in e
xplo
itin
g ne
twor
k st
ruct
ure
lA
I au
tom
ated
rea
soni
ng c
omm
unit
y ah
ead
in s
uppo
rtin
g dy
nam
ic r
estr
uctu
ring
.
lM
ixed
dis
cret
e/co
ntin
uous
net
wor
ks?
lIn
stan
tiat
ion
orde
r?
Summer Institute on Probablility in AI 1994
Inference 2 39
Sum
mar
y
lR
epre
sent
atio
n•
Noi
sy O
r
•A
sym
met
ries
•C
onti
nuou
s
lA
ppro
xim
atio
n•
Sear
ch•
Sim
ulat
ion
lA
re w
e do
ne?
•br
oad
outl
ine
seem
s un
ders
tood
•lo
ts o
f cl
eanu
p•
hybr
id a
lgor
ithm
s/ar
chit
ectu
res?
•en
gine
erin
g•
expe
rim
enta
tion
•m
inin
g re
al a
pplic
atio
ns (
eg, Q
MR
)