Bayesian Inference & Neural Networks
-
Upload
l-k -
Category
Technology
-
view
256 -
download
2
Transcript of Bayesian Inference & Neural Networks
Click to edit the title text format
Click to edit the outline text formatSecond Outline LevelThird Outline LevelFourth Outline LevelFifth Outline LevelSixth Outline LevelSeventh Outline Level
/
Click to edit the title text format
Click to edit the outline text formatSecond Outline LevelThird Outline LevelFourth Outline LevelFifth Outline LevelSixth Outline LevelSeventh Outline Level
/
Bayesian Inference & Neural Networks
Lukasz Krawczyk, 1st March 2017
HomoApriorius
HomoPragmaticus
HomoFriquentistus
HomoSapiens
HomoBayesianis
Good eveningMy name is Today Id like to talk about...
Agenda
About me
The Problem
Bayesian Inference
Hierarchical Models
Bayesian Inference & Neural Networks
About me
Data Scientist at Asurion Japan Holdings
Previous: Data Scientist at Abeja Inc.
MSc degree from Jagiellonian University, Poland
Contributor to several ML libraries
Currently Im working as a ...
PART 1
The Problem
Missing Uncertainty
Making a confident error is the worst thing we can do
With DL models we generally only have point estimates of parameters and predictions
Hard to make decisions when were not able to tell whether aDL model is certain about its output or not
Trust and adoption of DL is still low
PART 2
Bayesian Inference
Bayesian Inference
Inference
Posterior
Data
Credibile Region
Uncertainity
Better Insights
Prior
Model
Assumptions about datacontrolled by the prior
Bayes formula
P(true | D): The posteriorthe probability of the model parameters given the data: this is the result we want to compute.
P(D | true): The likelihoodproportional to the likelihood estimation in the frequentist approach.
P(true): The model priorencodes what we knew about the model prior to the application of the data D.
P(D): The data probabilitywhich in practice amounts to simply a normalization term.
There is no presentation about without Bayes Formula, so just as a reminder
Bayesian Inference
BayesianInferenceGeneral purpose framework
Generative models
Clarity of FS + Power of MLWhite-box modelling
Black-box fitting (NUTS, ADVI)
Uncertainity Intuitive insights
Learning from very small datasets
Probabilistic Programming
Automatic Differentation Variational InferenceNo U-Turn Sampler
Bayesian Inference
Bayesian Optimization (GP)
Hierarchical models (badass models)
Bonus pointsRobust in high dimensions
Minibatches
Knowledge transfer
BayesianInference
Bayesian Inference
Very easy way to cook your laptop
PART 3
Hierarchical Models
Hierarchical Models parameter pooling
Pooled
Unpooled
Partial-pooling
More accurate fittingNot enough dataGeneralization
Small datasetsMissing variations among groups
Example call duration model
Each advisor has his/her own distribution
Overall Call Center distribution is controlled by hyper parameter
}
Hierarchical Models - benefits
Modelling is very easy and intuitive
Natural hierarchical structure of observational data
Variation among individual groups
Knowledge transfer between groups
PART 4
Bayesian Inference & Neural Networks
Synergy
Replace weights with probability distributions
Example standard NN
x1 x2 y0.1 1.0 00.1 -1.3 1
2 hidden layers
sigmoid
tanh
Data
Backpropagation
Example NN with Bayesian Backpropagation
n=2
BayesianBackpropagation
2 hidden layers
Data
x1 x2 y0.1 1.0 [0,1,...]0.1 -1.3 [1,1,...]
Results
Uncertainity
Standard NN
NN with Bayesian Backpropagation
Synergy going deeper
~
~ Bayesian Hierarchical Model
Weight regularization similar to L2
Synergy going deeper
~
~ Bayesian Hierarchical Model
RegularizationWeight regularization similar to L2
Synergy going deeper
Bayesian Hierarchical Model
~ ~
Synergy going deeper
Bayesian Hierarchical Model
~ ~
Knowledge transfer
Why is this important?
Scientific perspectiveNN models with small datasets
Complex hierarchical neural networks (Bayesian CNN)
Minibatches
Knowledge transfer
Business perspectiveClear and intuitive models
Uncertainity in Finance & Insurance is extremely important
Better trust and adoption of Neural Network-based models
Thank you!