The Stan Modeling Languagemlss2014.hiit.fi/mlss_files/2-stan.pdf · 2014-05-03 · A Stan model is...
Transcript of The Stan Modeling Languagemlss2014.hiit.fi/mlss_files/2-stan.pdf · 2014-05-03 · A Stan model is...
![Page 1: The Stan Modeling Languagemlss2014.hiit.fi/mlss_files/2-stan.pdf · 2014-05-03 · A Stan model is defined by five program blocks data! transformed data! parameters (required)! transformed](https://reader033.fdocuments.in/reader033/viewer/2022041715/5e4a8ccc6b28656f35282f54/html5/thumbnails/1.jpg)
The Stan Modeling Language
![Page 2: The Stan Modeling Languagemlss2014.hiit.fi/mlss_files/2-stan.pdf · 2014-05-03 · A Stan model is defined by five program blocks data! transformed data! parameters (required)! transformed](https://reader033.fdocuments.in/reader033/viewer/2022041715/5e4a8ccc6b28656f35282f54/html5/thumbnails/2.jpg)
Stan
Hamiltonian Monte Carlo
Modeling Language
Automatic Differentiation Adaptation
![Page 3: The Stan Modeling Languagemlss2014.hiit.fi/mlss_files/2-stan.pdf · 2014-05-03 · A Stan model is defined by five program blocks data! transformed data! parameters (required)! transformed](https://reader033.fdocuments.in/reader033/viewer/2022041715/5e4a8ccc6b28656f35282f54/html5/thumbnails/3.jpg)
A Stan model is defined by five program blocks
data!transformed data!parameters (required)!transformed parameters!model (required)!generated quantities
![Page 4: The Stan Modeling Languagemlss2014.hiit.fi/mlss_files/2-stan.pdf · 2014-05-03 · A Stan model is defined by five program blocks data! transformed data! parameters (required)! transformed](https://reader033.fdocuments.in/reader033/viewer/2022041715/5e4a8ccc6b28656f35282f54/html5/thumbnails/4.jpg)
The “data” block reads external information
data { int N; int x[N]; int offset;}
![Page 5: The Stan Modeling Languagemlss2014.hiit.fi/mlss_files/2-stan.pdf · 2014-05-03 · A Stan model is defined by five program blocks data! transformed data! parameters (required)! transformed](https://reader033.fdocuments.in/reader033/viewer/2022041715/5e4a8ccc6b28656f35282f54/html5/thumbnails/5.jpg)
The “transformed data” block allows for preprocessing of the data
transformed data { int y[N]; for (n in 1:N) y[n] <- x[n] - offset;}
![Page 6: The Stan Modeling Languagemlss2014.hiit.fi/mlss_files/2-stan.pdf · 2014-05-03 · A Stan model is defined by five program blocks data! transformed data! parameters (required)! transformed](https://reader033.fdocuments.in/reader033/viewer/2022041715/5e4a8ccc6b28656f35282f54/html5/thumbnails/6.jpg)
The “parameters” block defines the sampling space
parameters { real<lower=0> lambda1; real<lower=0> lambda2;}
![Page 7: The Stan Modeling Languagemlss2014.hiit.fi/mlss_files/2-stan.pdf · 2014-05-03 · A Stan model is defined by five program blocks data! transformed data! parameters (required)! transformed](https://reader033.fdocuments.in/reader033/viewer/2022041715/5e4a8ccc6b28656f35282f54/html5/thumbnails/7.jpg)
The “transformed parameters” block allows for parameter processing before the posterior is computed
transformed parameters { real<lower=0> lambda; lambda <- lambda1 + lambda2;}
![Page 8: The Stan Modeling Languagemlss2014.hiit.fi/mlss_files/2-stan.pdf · 2014-05-03 · A Stan model is defined by five program blocks data! transformed data! parameters (required)! transformed](https://reader033.fdocuments.in/reader033/viewer/2022041715/5e4a8ccc6b28656f35282f54/html5/thumbnails/8.jpg)
In the “model” block we get to define our posterior
model { y ~ poisson(lambda); lambda1 ~ cauchy(0, 2.5); lambda2 ~ cauchy(0, 2.5);}
![Page 9: The Stan Modeling Languagemlss2014.hiit.fi/mlss_files/2-stan.pdf · 2014-05-03 · A Stan model is defined by five program blocks data! transformed data! parameters (required)! transformed](https://reader033.fdocuments.in/reader033/viewer/2022041715/5e4a8ccc6b28656f35282f54/html5/thumbnails/9.jpg)
Lastly, the “generated quantities” block allows for postprocessing
generated quantities { int x_predict; x_predict <- poisson_rng(lambda) + offset;}
![Page 10: The Stan Modeling Languagemlss2014.hiit.fi/mlss_files/2-stan.pdf · 2014-05-03 · A Stan model is defined by five program blocks data! transformed data! parameters (required)! transformed](https://reader033.fdocuments.in/reader033/viewer/2022041715/5e4a8ccc6b28656f35282f54/html5/thumbnails/10.jpg)
data { int N; int x[N]; int offset;}!transformed data { int y[N]; for (n in 1:N) y[n] <- x[n] - offset;}!parameters { real<lower=0> lambda1; real<lower=0> lambda2;}!transformed parameters { real<lower=0> lambda; lambda <- lambda1 + lambda2;}!model { y ~ poisson(lambda); lambda1 ~ cauchy(0, 2.5); lambda2 ~ cauchy(0, 2.5);}!generated quantities { int x_predict x_predict <- poisson_rng(lambda) + offset;}
![Page 11: The Stan Modeling Languagemlss2014.hiit.fi/mlss_files/2-stan.pdf · 2014-05-03 · A Stan model is defined by five program blocks data! transformed data! parameters (required)! transformed](https://reader033.fdocuments.in/reader033/viewer/2022041715/5e4a8ccc6b28656f35282f54/html5/thumbnails/11.jpg)
data transformed data
parameters transformed parameters
model generated quantities
Execution Per chain Per chain NA Per leapfrog Per leapfrog Per sample
Variable Declarations
Yes Yes Yes Yes Yes Yes
Variable Scope
Global Global Global Global Local Local
Variables Saved?
No No Yes Yes No Yes
Modify Posterior?
No No No No Yes No
Random Variables
No No No No No Yes
![Page 12: The Stan Modeling Languagemlss2014.hiit.fi/mlss_files/2-stan.pdf · 2014-05-03 · A Stan model is defined by five program blocks data! transformed data! parameters (required)! transformed](https://reader033.fdocuments.in/reader033/viewer/2022041715/5e4a8ccc6b28656f35282f54/html5/thumbnails/12.jpg)
Stan has two primitive types
int is an integer type !
real is a floating point precision type
![Page 13: The Stan Modeling Languagemlss2014.hiit.fi/mlss_files/2-stan.pdf · 2014-05-03 · A Stan model is defined by five program blocks data! transformed data! parameters (required)! transformed](https://reader033.fdocuments.in/reader033/viewer/2022041715/5e4a8ccc6b28656f35282f54/html5/thumbnails/13.jpg)
Both can be bounded
int<lower=1> N;!real<upper=5> alpha;real<lower=-1,upper=1> beta;!real gamma;real<upper=gamma> zeta;
![Page 14: The Stan Modeling Languagemlss2014.hiit.fi/mlss_files/2-stan.pdf · 2014-05-03 · A Stan model is defined by five program blocks data! transformed data! parameters (required)! transformed](https://reader033.fdocuments.in/reader033/viewer/2022041715/5e4a8ccc6b28656f35282f54/html5/thumbnails/14.jpg)
Reals extend to linear algebra types
vector[10] a; // Column vectormatrix[10, 1] b;!row_vector[10] c; // Row vectormatrix[1, 10] d;
![Page 15: The Stan Modeling Languagemlss2014.hiit.fi/mlss_files/2-stan.pdf · 2014-05-03 · A Stan model is defined by five program blocks data! transformed data! parameters (required)! transformed](https://reader033.fdocuments.in/reader033/viewer/2022041715/5e4a8ccc6b28656f35282f54/html5/thumbnails/15.jpg)
Arrays of int, reals, vectors, and matrices are available
real a[10];!vector[10] b[10];!matrix[10, 10] c[10];
![Page 16: The Stan Modeling Languagemlss2014.hiit.fi/mlss_files/2-stan.pdf · 2014-05-03 · A Stan model is defined by five program blocks data! transformed data! parameters (required)! transformed](https://reader033.fdocuments.in/reader033/viewer/2022041715/5e4a8ccc6b28656f35282f54/html5/thumbnails/16.jpg)
Stan also implements a variety of constrained types
simplex[5] theta; // sum(theta) = 1!ordered[5] o; // o[1] < ... < o[5]positive_ordered[5] p;!corr_matrix[5] C; // Symmetric andcov_matrix[5] Sigma; // positive-definite
![Page 17: The Stan Modeling Languagemlss2014.hiit.fi/mlss_files/2-stan.pdf · 2014-05-03 · A Stan model is defined by five program blocks data! transformed data! parameters (required)! transformed](https://reader033.fdocuments.in/reader033/viewer/2022041715/5e4a8ccc6b28656f35282f54/html5/thumbnails/17.jpg)
All of your favorite statements are available, too
if/then/else!for (i in 1:I)!while (i < I)
![Page 18: The Stan Modeling Languagemlss2014.hiit.fi/mlss_files/2-stan.pdf · 2014-05-03 · A Stan model is defined by five program blocks data! transformed data! parameters (required)! transformed](https://reader033.fdocuments.in/reader033/viewer/2022041715/5e4a8ccc6b28656f35282f54/html5/thumbnails/18.jpg)
There are two ways to modify the posterior
y ~ normal(0, 1);!increment_log_posterior(log_normal(y, 0, 1));
![Page 19: The Stan Modeling Languagemlss2014.hiit.fi/mlss_files/2-stan.pdf · 2014-05-03 · A Stan model is defined by five program blocks data! transformed data! parameters (required)! transformed](https://reader033.fdocuments.in/reader033/viewer/2022041715/5e4a8ccc6b28656f35282f54/html5/thumbnails/19.jpg)
parameters { real mu[N]; real<lower=0> sigma[N];}!model {! for (n in 1:N) y[n] ~ normal(mu[n], sigma[n]);! !}
Many sampling statements are vectorized
![Page 20: The Stan Modeling Languagemlss2014.hiit.fi/mlss_files/2-stan.pdf · 2014-05-03 · A Stan model is defined by five program blocks data! transformed data! parameters (required)! transformed](https://reader033.fdocuments.in/reader033/viewer/2022041715/5e4a8ccc6b28656f35282f54/html5/thumbnails/20.jpg)
parameters { real mu[N]; real<lower=0> sigma[N];}!model {! for (n in 1:N) y[n] ~ normal(mu[n], sigma[n]);! y ~ normal(mu, sigma);!}
Many sampling statements are vectorized
![Page 21: The Stan Modeling Languagemlss2014.hiit.fi/mlss_files/2-stan.pdf · 2014-05-03 · A Stan model is defined by five program blocks data! transformed data! parameters (required)! transformed](https://reader033.fdocuments.in/reader033/viewer/2022041715/5e4a8ccc6b28656f35282f54/html5/thumbnails/21.jpg)
Because of the huge number of possible configurations, Stan uses hierarchical arguments
![Page 22: The Stan Modeling Languagemlss2014.hiit.fi/mlss_files/2-stan.pdf · 2014-05-03 · A Stan model is defined by five program blocks data! transformed data! parameters (required)! transformed](https://reader033.fdocuments.in/reader033/viewer/2022041715/5e4a8ccc6b28656f35282f54/html5/thumbnails/22.jpg)
Because of the huge number of possible configurations, Stan uses hierarchical arguments
![Page 23: The Stan Modeling Languagemlss2014.hiit.fi/mlss_files/2-stan.pdf · 2014-05-03 · A Stan model is defined by five program blocks data! transformed data! parameters (required)! transformed](https://reader033.fdocuments.in/reader033/viewer/2022041715/5e4a8ccc6b28656f35282f54/html5/thumbnails/23.jpg)
![Page 24: The Stan Modeling Languagemlss2014.hiit.fi/mlss_files/2-stan.pdf · 2014-05-03 · A Stan model is defined by five program blocks data! transformed data! parameters (required)! transformed](https://reader033.fdocuments.in/reader033/viewer/2022041715/5e4a8ccc6b28656f35282f54/html5/thumbnails/24.jpg)
Interactive example
http://www.mc-stan.org/mlss14/support.data.R
http://www.mc-stan.org/mlss14/support-data.json
http://www.mc-stan.org/mlss14/support-data.pkl
![Page 25: The Stan Modeling Languagemlss2014.hiit.fi/mlss_files/2-stan.pdf · 2014-05-03 · A Stan model is defined by five program blocks data! transformed data! parameters (required)! transformed](https://reader033.fdocuments.in/reader033/viewer/2022041715/5e4a8ccc6b28656f35282f54/html5/thumbnails/25.jpg)
Younger Americans are more likely to support the statewide legalization of gay marriage
0
0.2
0.4
0.6
0.8
1
10 20 30 40 50 60 70 80 90 100
Supp
ort F
or G
ay M
arria
ge (k
/ n)
Age
http://www.mc-stan.org/mlss14/support.data.R
![Page 26: The Stan Modeling Languagemlss2014.hiit.fi/mlss_files/2-stan.pdf · 2014-05-03 · A Stan model is defined by five program blocks data! transformed data! parameters (required)! transformed](https://reader033.fdocuments.in/reader033/viewer/2022041715/5e4a8ccc6b28656f35282f54/html5/thumbnails/26.jpg)
The GP models the data well
0
0.2
0.4
0.6
0.8
1
10 20 30 40 50 60 70 80 90 100
Supp
ort F
or G
ay M
arria
ge (%
)
Age
GP MeanGP Mean
http://www.mc-stan.org/mlss14/support.data.R
![Page 27: The Stan Modeling Languagemlss2014.hiit.fi/mlss_files/2-stan.pdf · 2014-05-03 · A Stan model is defined by five program blocks data! transformed data! parameters (required)! transformed](https://reader033.fdocuments.in/reader033/viewer/2022041715/5e4a8ccc6b28656f35282f54/html5/thumbnails/27.jpg)
And the posterior predictive checks indicate consistency
0
0.2
0.4
0.6
0.8
1
10 20 30 40 50 60 70 80 90 100
Supp
ort F
or G
ay M
arria
ge (k~ /
n)
Age
PPDPPD
http://www.mc-stan.org/mlss14/support.data.R
![Page 28: The Stan Modeling Languagemlss2014.hiit.fi/mlss_files/2-stan.pdf · 2014-05-03 · A Stan model is defined by five program blocks data! transformed data! parameters (required)! transformed](https://reader033.fdocuments.in/reader033/viewer/2022041715/5e4a8ccc6b28656f35282f54/html5/thumbnails/28.jpg)
But what about the covariates we’ve ignored?
0
0.2
0.4
0.6
0.8
1
10 20 30 40 50 60 70 80 90 100
Supp
ort F
or G
ay M
arria
ge (%
)
Age
Male
Female
http://www.mc-stan.org/mlss14/support.data.R
![Page 29: The Stan Modeling Languagemlss2014.hiit.fi/mlss_files/2-stan.pdf · 2014-05-03 · A Stan model is defined by five program blocks data! transformed data! parameters (required)! transformed](https://reader033.fdocuments.in/reader033/viewer/2022041715/5e4a8ccc6b28656f35282f54/html5/thumbnails/29.jpg)
0
0.2
0.4
0.6
0.8
1
10 20 30 40 50 60 70 80 90 100
Supp
ort F
or G
ay M
arria
ge (%
)
Age
MaleFemale
http://www.mc-stan.org/mlss14/support.data.R
![Page 30: The Stan Modeling Languagemlss2014.hiit.fi/mlss_files/2-stan.pdf · 2014-05-03 · A Stan model is defined by five program blocks data! transformed data! parameters (required)! transformed](https://reader033.fdocuments.in/reader033/viewer/2022041715/5e4a8ccc6b28656f35282f54/html5/thumbnails/30.jpg)
0
0.2
0.4
0.6
0.8
1
10 20 30 40 50 60 70 80 90 100
Supp
ort F
or G
ay M
arria
ge (k~ /
n)
Age
MaleFemale
http://www.mc-stan.org/mlss14/support.data.R