Forest Learning from Data
-
Upload
joe-suzuki -
Category
Science
-
view
212 -
download
1
Transcript of Forest Learning from Data
Forest Learning from Data
Joe Suzuki
July 17, 2017
Road Map
PART-II: July 24, 2017 (based on PART-I)
1. Estimating Mutual Information (15 mins)
2. Learning Forests from Data (25 mins)
3. Learning Bayesian Networks from Data (5 mins)
4. Exercise (45 mins)
PART-I: July 17, 2017
A Bayesian Approach to Data Compression
Entropy
Mutual Information (MI)
Correlation may not detect independence!
ML Estimator of MI
Bayesian Testing of Independence
Bayesian Estimation of MI
From Stirling’s formula
For large n
Experiments 500 trials for binary seq. of length n=200
BNSL: a CRAN package (J. Suzuki and J. Kawahara, 2017)
Bayesian Network Learning Structure
https://cran.r-project.org/web/packages/BNSL/index.html
collects research results by Joe Suzuki.
install(“BNSL”)
library(BNSL)
n=200; p=0.5; x=rbinom(n,1,p); y=rbinom(n,1,p) # seqs are generated
mi(x,y, proc=9) # I_n
mi(x,y) # J_n
Tree Approximation
Factorization w.r.t. A Tree
Find E s.t. D(P||P’) is minimized
Kruskal’s Algorithm
Chow-Liu Algorithm
Experiments using Asia data set
• library(BNSL)
• mm=mi_matrix(asia, proc=9) # I_n is used
• edge.list=kruskal(mm)
• g=graph_from_edgelist(edge.list, directed=FALSE)
• plot(g)
• mm=mi_matrix(asia) # J_n is used
• edge.list=kruskal(mm)
• g=graph_from_edgelist(edge.list, directed=FALSE)
• plot(g)
Asia (8 variables)
S. Lauritzen, D. Spiegelhalter. Local
Computation with Probabilities on
Graphical Structures and their
Application to Expert Systems (with
discussion). Journal of the Royal
Statistical Society: Series B
(Statistical Methodology), 50(2):157-224, 1988
Asia Data Set
I. A. Beinlich, H. J. Suermondt, R. M. Chavez, and G. F. Cooper. The ALARM Monitoring System: A Case Study with Two Probabilistic Inference Techniques for Belief Networks. In Proceedings of the 2nd European Conference on Artificial Intelligence in Medicine, pages 247-256. Springer-Verlag, 1989.
Alarm (37 varibles)
Alarm Data Set
Learning Bayesian Networks from Data
The # of candidate structures with p nodes is more than exponential with p
25 DAGs exist for p=3 but only 11 BNs are considered
7 local scores and 11 global scores
• Estimating Mutual Information
• Learning Forests from Data
• Learning Bayesian Networks from Data
Summary
Problem Set #2