Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics
-
Upload
andres-lopez-sepulcre -
Category
Education
-
view
53 -
download
7
description
Transcript of Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics
![Page 1: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/1.jpg)
![Page 2: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/2.jpg)
What is a Probability?
• Physical Probability– Frequentist: long-run outcome– Propensity: property of the system
• Evidential Probability (Bayesian)– Measure of statement (un)certainty
![Page 3: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/3.jpg)
Statistical Falsificationism
• Data is a consequence of the “true” model
• That consequence is probabilistic
Likelihood = P (Data | Model)
• Evidence against the model if the data at hand would be very unlikely under that model
R.A. Fisher
P (Evidence | Hypothesis)
![Page 4: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/4.jpg)
Comparing Multiple Hypothesis
• Falsificationism just rejects hypotheses– Cannot provide support for a hypothesis– Relies on the inability to reject it with time– Only one hypothesis at a time
• In reality, there are often multiple candidate hypotheses
• We can simultaneously calculate the likelihood of our evidence given each of them
• Measure the relative support for hypotheses
Thomas C. ChamberlinImre Lakatos
![Page 5: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/5.jpg)
Likelihood Function
MAXIMUM LIKELIHOOD
ESTIMATE
MAXIMUM LIKELIHOOD
φ = 0.833
![Page 6: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/6.jpg)
8. Bayesian Analysis
![Page 7: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/7.jpg)
Frequentist vs. Bayesian
Frequentist Bayesian
Probability is a long-run average Probability is a degree of belief
There is a true Model, the Data is a random realization
The Data is true/fixed, Models have probabilities
Probability of the data given a hypothesis (Likelihood)
Probability of a hypothesis given the data
Each repeated experiment/observation starts
from ignorance
Can incorporate prior knowledge: probabilities can be updated
Harold JeffereysJerzy Neyman
![Page 8: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/8.jpg)
Bayes Theorem
Thomas Bayes
![Page 9: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/9.jpg)
Bayes Theorem
Prior Knowledge Likelihood
ConstantPosteriorDistribution
Really hard to calculate most times
![Page 10: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/10.jpg)
Estimating Survival
• What is the survival probability?– Check caterpillars every week– Experiment: 5 caterpillars. Ends after 4 weeks– Data: survived 2, 4, 4, 2 and 3 weeks
Clay model caterpillar
1 1 1 0 0
1 1 1 1 1
1 1 1 1 1
1 1 1 0 0
1 1 1 1 0
Caterpillars set Experiment endstime (weeks)
Indi
vidu
als
1 = alive (intact)2 = dead (attacked)
![Page 11: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/11.jpg)
Estimating Survival
• Define a model (assumptions):– Constant probability of survival through time– All individuals are equal– Just one parameter: probability of survival, φ
• Probability of the data given the model: L (D | M)
1 1 1 0 0
1 1 1 1 1
1 1 1 1 1
1 1 1 0 0
1 1 1 1 0
time (weeks)
Indi
vidu
als
1 = alive (intact)2 = dead (attacked)
Clay model caterpillar
![Page 12: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/12.jpg)
Likelihood Approach
• Likelihood of the data for any value of φ L(data |φ)
1 1 1 0 0
1 1 1 1 1
1 1 1 1 1
1 1 1 0 0
1 1 1 1 0
Caterpillars set Experiment ends
time (weeks)
Indi
vidu
als
φ2 ✕ (1-φ)
φ4
φ4
φ2 ✕ (1-φ)
φ3 ✕ (1-φ)
Probability of all data all occurring
independently together
φ15 ✕ (1-φ)3
![Page 13: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/13.jpg)
Likelihood Function
MAXIMUM LIKELIHOOD
ESTIMATE
MAXIMUM LIKELIHOOD
φ = 0.833
![Page 14: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/14.jpg)
The Bayesian Way
• We need a Likelihood function: we have it• We need a Prior distribution:
e.g. all parameter values equally likely a prioriPrior(φ) ~ Uniform(0,1)
Posterior (φ) = P(φ | data) =
P(data)
x
![Page 15: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/15.jpg)
Markov Chain Monte Carlo (MCMC)
1. Initial guess for parameter was A2. Guess a random number (B)3. Calculate Prior x Likelihood4. Accept B with probability:
5. If accept, add it to list of guesses6. If reject, add previous guess to list of guesses
If all guesses have the same probability, this is 1.
![Page 16: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/16.jpg)
MCMC
Propose: φ = 0.2
Prior ~ Unif (0,1)Likelihood = φ15 ✕ (1-φ)3
![Page 17: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/17.jpg)
MCMC
Previous: φ = 0.2Propose: φ = 0.1
Prior ~ Unif (0,1)Likelihood = φ15 ✕ (1-φ)3
Previous Likelihood
Proposed Likelihood
Previous Prior
Proposed Prior
![Page 18: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/18.jpg)
MCMC
Previous: φ = 0.2Propose: φ = 0.66
Prior ~ Unif (0,1)Likelihood = φ15 ✕ (1-φ)3
![Page 19: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/19.jpg)
MCMC
Previous: φ = 0.66Propose: φ = 0.57
Prior ~ Unif (0,1)Likelihood = φ15 ✕ (1-φ)3
![Page 20: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/20.jpg)
MCMC
Previous: φ = 0.66Propose: φ = 0.57
Prior ~ Unif (0,1)Likelihood = φ15 ✕ (1-φ)3
![Page 21: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/21.jpg)
MCMC
Prior ~ Unif (0,1)Likelihood = φ15 ✕ (1-φ)3
φ = 0.55φ = 0.62…
![Page 22: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/22.jpg)
MCMC
Prior ~ Unif (0,1)Likelihood = φ15 ✕ (1-φ)3
φ = 0.55φ = 0.62φ = 0.44φ = 0.44φ = 0.44φ = 0.57φ = 0.82φ = 0.71
![Page 23: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/23.jpg)
Sampling the MCMCIgnore initial numbers:Still far from optimum
BURN-IN
These numbers should be a good sample of the Posterior (φ | data)
![Page 24: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/24.jpg)
Posterior Distribution
Survival Estimate (φ)
![Page 25: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/25.jpg)
Posterior Distribution
Survival Estimate (φ)
![Page 26: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/26.jpg)
Posterior Distribution
Survival Estimate (φ)
95% area
φ = 0.81 (0.60-0.94)
95% CREDIBLEINTERVAL
![Page 27: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/27.jpg)
Uninformative Prior
• We assumed no prior knowledge of φ
Survival Estimate (φ)
Prior Distribution
Posterior Distribution
![Page 28: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/28.jpg)
Informative Priors
Posterior
Prior
Posterior
Prior
• What if I had prior information?– Previous experiment– Literature or expert knowledge
![Page 29: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/29.jpg)
More than one Parameter
e.g. Logistic population growthProblem: Best value for r depends on value of K
![Page 30: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/30.jpg)
Marginalizing
• e.g. Probability of dying given that I am male if…?– P(death | infected) = 0.3– P(death | attacked) = 0.7– P(death | lightning) = 0.99– P(death | bored) = 0.01– P (infected | male) = 0.2– P(attacked | male) = 0.1– P(lightning | male) = 0.00001– P(bored | male)= 0.99
P(death|male) = 0.3x0.2+0.7x0.1
+0.99x0.00001+0.01x0.99
= 0.14
![Page 31: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/31.jpg)
Marginal LikelihoodIntegrate (sum) over all possible values of θ1
Likelihood be for a given value of θ2
Probability of θ2 for any given value of θ1
Posterior (r) P (data | K=200, r)
X = P(data |K=200)
![Page 32: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/32.jpg)
Gibbs Sampling
1. Propose value for parameter θ1 and keep all other parameters at latest value
2. Accept or reject proposed value θ1
3. Move on to updating value of θ2 while keeping latest values of other parameters
4. Keep on until all parameters are updated5. Update θ1 keeping all other parameters at latest
updated valueEtc…
![Page 33: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/33.jpg)
Gibbs Sampling
![Page 34: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/34.jpg)
r = 1.01 (0.61-1.61)
K = 202.1 (200.6-206.5)
N0 = 103.9 (98.0 -123.2)
![Page 35: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/35.jpg)
More than one Parameter
e.g. Logisic population growth
![Page 36: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/36.jpg)
The Power of Marginalizing
• Complex random effect structures– Condition a random effect on another
• Missing data– Condition parameters on missing data values– Missing values updated in Gibbs Sampling: we can
get estimates!• Latent Variables– Models with unmeasured variables that influence
the data
![Page 37: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/37.jpg)
Latent Variables
e.g. Dispersal strategies in frogs– Two types of individuals: dispersive and sedentary– Data on number of dispersal movements– Cannot measure ‘type’ of individual
![Page 38: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/38.jpg)
Latent Variables
N.moves ~ Poisson(λ)λ = λ1 if type = sedentary
λ = λ2 if type = dispersive
P(dispersive) ~ Binom(p)
Estimate Credible Interval
λ1 2.34 1.84-2.90
λ2 11.13 9.90-12.36
p 0.56 0.45-0.67
![Page 39: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/39.jpg)
Latent Variables
• Gibbs Sampling marginalized over the probability of each data point being each type
• Can get estimates of the ‘type’ of each data point• Maybe I want to test if type correlates with mating
success?
![Page 40: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/40.jpg)
Bayesian Model Comparison
• Deviance Information Criterion– DIC = -2logLik + var(logLik)– The more parameters, the more variance the
likelihood has• Reverse Jump MCMC– Create a ‘supermodel’ where each model is weighted by
the probability of that model being true– Gibbs Sampling on all the model estimates and model
probabilities at the same time.– Marginalizes over model probabilities– Straightforward to do model averaging of parameters
![Page 41: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/41.jpg)
Reverse Jump MCMC
N.moves ~ Poisson(λ)Model 1
λ = λ1 if type = sedentary
λ = λ2 if type = dispersive
P(dispersive) ~ Binom(p)Model 2
λ = λ0 same for all
Estimate Credible Interval
M1 0.78
λ1 2.34 1.84-2.90
λ2 11.13 9.90-12.36
p 0.56 0.45-0.67
M2 0.22
λ0 6.17 5.67-6.63
![Page 42: Foundations in Statistics for Ecology and Evolution 8. Bayesian Statistics](https://reader034.fdocuments.in/reader034/viewer/2022051514/548328ddb07959600c8b4985/html5/thumbnails/42.jpg)
Take Home
• Bayesians calculate probabilities of parameters and hypotheses
• Priors can be informative or uninformative• Parameters are probabilistic: hierarchical models• Pros: – Bayesian methods marginalize over unknowns– Complex hierarchical models and latent variables
• Cons:– Computation time is long– Choosing priors