Nonparametric Bayes and human cognition Tom Griffiths Department of Psychology Program in Cognitive...
-
date post
20-Dec-2015 -
Category
Documents
-
view
214 -
download
0
Transcript of Nonparametric Bayes and human cognition Tom Griffiths Department of Psychology Program in Cognitive...
Nonparametric Bayes and human cognition
Tom GriffithsDepartment of Psychology
Program in Cognitive Science
University of California, Berkeley
Analyzing psychological data• Dirichlet process mixture models for capturing
individual differences(Navarro, Griffiths, Steyvers, & Lee, 2006)
• Infinite latent feature models…– …for features influencing similarity
(Navarro & Griffiths, 2007; 2008)– …for features influencing decisions
()
Categorization
How do people represent categories?
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
catdog ?
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
cat
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
cat
Prototypes
(Posner & Keele, 1968; Reed, 1972)
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Prototype
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
cat
cat
cat
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
cat
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
cat
Exemplars
(Medin & Schaffer, 1978; Nosofsky, 1986)
Store everyinstance
(exemplar)in memory
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
cat
cat
cat
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
cat
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
cat
Something in between
(Love et al., 2004; Vanpaemel et al., 2005)
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
cat
cat
cat
A computational problem• Categorization is a classic inductive problem
– data: stimulus x– hypotheses: category c
• We can apply Bayes’ rule:
and choose c such that P(c|x) is maximized
€
P(c | x) =p(x | c)P(c)
p(x | c)P(c)c
∑
Density estimation• We need to estimate some probability
distributions– what is P(c)?– what is p(x|c)?
• Two approaches:– parametric– nonparametric
• These approaches correspond to prototype and exemplar models respectively
(Ashby & Alfonso-Reese, 1995)
Parametric density estimation
Assume that p(x|c) has a simple form, characterized by parameters (indicating the
prototype)
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
Pro
babi
lity
den
sity
x
Nonparametric density estimation
Approximate a probability distribution as a sum of many “kernels” (one per data point)
estimated functionindividual kernelstrue function
n = 10
Pro
babi
lity
den
sity
x
Something in between
mixture distributionmixture components
Pro
babi
lity
x
Use a “mixture” distribution, with more than one component per data point
(Rosseel, 2002)
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
Anderson’s rational model(Anderson, 1990, 1991)
• Treat category labels like any other feature• Define a joint distribution p(x,c) on features using
a mixture model, breaking objects into clusters• Allow the number of clusters to vary…
€
P(cluster j)∝n j j is old
α j is new
⎧ ⎨ ⎩
a Dirichlet process mixture model(Neal, 1998; Sanborn et al., 2006)
A unifying rational model• Density estimation is a unifying framework
– a way of viewing models of categorization
• We can go beyond this to define a unifying model– one model, of which all others are special cases
• Learners can adopt different representations by adaptively selecting between these cases
• Basic tool: two interacting levels of clusters– results from the hierarchical Dirichlet process
(Teh, Jordan, Beal, & Blei, 2004)
The hierarchical Dirichlet process
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
A unifying rational modelclusterexemplarcategory
€
γ∈ (0,∞)
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
€
γ→ ∞
€
α → ∞€
α ∈ (0,∞)€
α → 0
exemplar
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
prototype
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
Anderson
HDP+, and Smith & Minda (1998)
• HDP+, will automatically infer a representation using exemplars, prototypes, or something in between (with α being learned from the data)
• Test on Smith & Minda (1998, Experiment 2)111111011111101111110111111011111110000100
000000100000010000001000000010000001111101
Category A: Category B:
exceptions
HDP+, and Smith & Minda (1998)
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
Pro
babi
lity
of
A prototype
exemplar HDP
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
Log
-lik
elih
ood
exemplar
prototype
HDP
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.The promise of HDP+,+
• In HDP+,+, clusters are shared between categories
– a property of hierarchical Bayesian models
• Learning one category has a direct effect on the prior on probability densities for the next category
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Learning the features of objects
• Most models of human cognition assume objects are represented in terms of abstract features
• What are the features of this object?
• What determines what features we identify?
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
(Austerweil & Griffiths, submitted)
The nonparametric approach
Use the Indian buffet process as a prior on Z
Assume that the total number of features is unbounded, but only a finite number will be
expressed in any finite dataset
(Griffiths & Ghahramani, 2006)
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
(Austerweil & Griffiths, submitted)
An experiment…
(Austerweil & Griffiths, submitted)
Training Testing
Cor
rela
ted
Fac
tori
al
See
nU
nsee
nS
huff
led
Conclusions
• Approaching cognitive problems as computational problems allows cognitive science and machine learning to be mutually informative
• Machine
Credits
CategorizationAdam SanbornKevin CaniniDan Navarro
Learning featuresJoe Austerweil
MCMC with peopleAdam Sanborn
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
Computational Cognitive Science Labhttp://cocosci.berkeley.edu/
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.