Mathematical Statistics Team - Riken...IX. ICCS 2018. Springer Proceedings in Complexity. Springer),...
Transcript of Mathematical Statistics Team - Riken...IX. ICCS 2018. Springer Proceedings in Complexity. Springer),...
-
•
•
•
•
•
数理統計学チーム 下平英寿Mathematical Statistics Team
Hidetoshi Shimodaira
@
• selective inference••••••
Thong PhamKIM GEEWOOK
View-1 View-2 View-Dℝ𝑝1 ℝ𝑝2 ℝ𝑝𝐷
Neural networks
ℝ𝐾Shared space
IPS
Stu
den
t/C
ours
eU
niv
ersi
ties
Hyperbolic SIPS IPDS WIPS
●
●●
●●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●●
●
●
●●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●● ●
●
●
●
●
●
●●
●
●
●
●
●
●● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
● ●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●●
-4 -2 0 2 4
-4-2
02
4
Y
A (c
ase
1)
: z=1, : z=0
Y
density
-3 -2 -1 0 1 2 3
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
Y, ZYY, A (case 1)
-4 -2 0 2 4
-4-2
02
4
Y
A (c
ase
2)
: z=1, : z=0
Y
density
-3 -2 -1 0 1 2 3
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
Y, ZYY, A (case 2)
●●●●●●●●
●
●
●
●
●
0 2 4 6 8
−0.5
0.0
0.5
1.0
1.5
2.0
2.5
extrapolation k=2poly.3
σ2
ψ(σ
2 )
k.2
A Fitting to Tree T1 B Fitting to Edge E2
●●●●●●
●●
●
●
●
●
0 2 4 6 8
−1.6
−1.4
−1.2
−1.0
−0.8
−0.6
extrapolation k=2poly.2+poly.3+sing.3
σ2
ψ(σ
2 )
k.2
1 human
2 seal3 cow
4 rabbit
5 mouse6 opssum
1 human
2 seal
3 cow
4 rabbit 5 mouse
6 opssum0.1
E1 E3
E2 E6
T7T1
E8E1
❗
linear algebra learn study -ing !number
learnalgebralinear algebra
endeavor
learn
studymathmatics
algebra
linear algebra
studying
Corpus
(a)
(b)
(c)
xi�nleft , · · · , xi�2, xi�1, xi, xi+1, · · · , xj�1, xj , xj+1, xj+2, · · · , xj+nrightTarget n-gramContext n-gram Context n-gram
xi, xi+1, xi+2, xi+3, · · · , xj�3, xj�2, xj�1, xjSub-n-grams 2 S(x (i:j)) ⇢ V
AAACk3ichVFBS+NAFP6MuqvVXesuguAlWJV6KS9eVtyLuB68CGptFayUJDvtzpomIZkWNfQP7HnBgycFEfEfeNXD/oE9+BPEo4IXD76mAVFRX8i8b75535tvZizfkaEiuuzQOru6P3zs6U319X/6PJAe/FIMvXpgi4LtOV6wbpmhcKQrCkoqR6z7gTBrliPWrK0frfW1hghC6bmrascXmzWz6sqKtE3FVDmdGytJV89nS0psK6sSxVmqaLvZLEdZOfN7sjmpl8K6FQqlF8fK6QzlKA79JTASkEESS176GCX8hAcbddQg4EIxdmAi5G8DBgg+c5uImAsYyXhdoIkUa+tcJbjCZHaLxyrPNhLW5XmrZxirbd7F4T9gpY5x+k8ndEP/6JSu6P7VXlHco+Vlh7PV1gq/PPBnOH/3rqrGWeHXo+pNzwoVTMdeJXv3Y6Z1Crutb+zu3eRnVsajCTqka/Z/QJd0zidwG7f20bJY2UeKH8B4ft0vQXEqZ1DOWJ7KzM4lT9GDEYwiy/f9DbNYwBIKvO9fnOEcF9qQ9l2b0+bbpVpHovmKJ6EtPgAbLps9AAACk3ichVFBS+NAFP6MuqvVXesuguAlWJV6KS9eVtyLuB68CGptFayUJDvtzpomIZkWNfQP7HnBgycFEfEfeNXD/oE9+BPEo4IXD76mAVFRX8i8b75535tvZizfkaEiuuzQOru6P3zs6U319X/6PJAe/FIMvXpgi4LtOV6wbpmhcKQrCkoqR6z7gTBrliPWrK0frfW1hghC6bmrascXmzWz6sqKtE3FVDmdGytJV89nS0psK6sSxVmqaLvZLEdZOfN7sjmpl8K6FQqlF8fK6QzlKA79JTASkEESS176GCX8hAcbddQg4EIxdmAi5G8DBgg+c5uImAsYyXhdoIkUa+tcJbjCZHaLxyrPNhLW5XmrZxirbd7F4T9gpY5x+k8ndEP/6JSu6P7VXlHco+Vlh7PV1gq/PPBnOH/3rqrGWeHXo+pNzwoVTMdeJXv3Y6Z1Crutb+zu3eRnVsajCTqka/Z/QJd0zidwG7f20bJY2UeKH8B4ft0vQXEqZ1DOWJ7KzM4lT9GDEYwiy/f9DbNYwBIKvO9fnOEcF9qQ9l2b0+bbpVpHovmKJ6EtPgAbLps9AAACk3ichVFBS+NAFP6MuqvVXesuguAlWJV6KS9eVtyLuB68CGptFayUJDvtzpomIZkWNfQP7HnBgycFEfEfeNXD/oE9+BPEo4IXD76mAVFRX8i8b75535tvZizfkaEiuuzQOru6P3zs6U319X/6PJAe/FIMvXpgi4LtOV6wbpmhcKQrCkoqR6z7gTBrliPWrK0frfW1hghC6bmrascXmzWz6sqKtE3FVDmdGytJV89nS0psK6sSxVmqaLvZLEdZOfN7sjmpl8K6FQqlF8fK6QzlKA79JTASkEESS176GCX8hAcbddQg4EIxdmAi5G8DBgg+c5uImAsYyXhdoIkUa+tcJbjCZHaLxyrPNhLW5XmrZxirbd7F4T9gpY5x+k8ndEP/6JSu6P7VXlHco+Vlh7PV1gq/PPBnOH/3rqrGWeHXo+pNzwoVTMdeJXv3Y6Z1Crutb+zu3eRnVsajCTqka/Z/QJd0zidwG7f20bJY2UeKH8B4ft0vQXEqZ1DOWJ7KzM4lT9GDEYwiy/f9DbNYwBIKvO9fnOEcF9qQ9l2b0+bbpVpHovmKJ6EtPgAbLps9AAACk3ichVFBS+NAFP6MuqvVXesuguAlWJV6KS9eVtyLuB68CGptFayUJDvtzpomIZkWNfQP7HnBgycFEfEfeNXD/oE9+BPEo4IXD76mAVFRX8i8b75535tvZizfkaEiuuzQOru6P3zs6U319X/6PJAe/FIMvXpgi4LtOV6wbpmhcKQrCkoqR6z7gTBrliPWrK0frfW1hghC6bmrascXmzWz6sqKtE3FVDmdGytJV89nS0psK6sSxVmqaLvZLEdZOfN7sjmpl8K6FQqlF8fK6QzlKA79JTASkEESS176GCX8hAcbddQg4EIxdmAi5G8DBgg+c5uImAsYyXhdoIkUa+tcJbjCZHaLxyrPNhLW5XmrZxirbd7F4T9gpY5x+k8ndEP/6JSu6P7VXlHco+Vlh7PV1gq/PPBnOH/3rqrGWeHXo+pNzwoVTMdeJXv3Y6Z1Crutb+zu3eRnVsajCTqka/Z/QJd0zidwG7f20bJY2UeKH8B4ft0vQXEqZ1DOWJ7KzM4lT9GDEYwiy/f9DbNYwBIKvO9fnOEcF9qQ9l2b0+bbpVpHovmKJ6EtPgAbLps9
2 VAAACanichVG7SgNBFD1Z3/EVY6OkCSYRq3DXRrESbSx95QGJyO466uBmd9ndBDT4A3ZWgqkURMTPsPEHLPIJol0EGwtvNguiQb3DzJw5c8+dMzO6Y0rPJ2pGlJ7evv6BwaHo8Mjo2HhsIp737KpriJxhm7Zb1DVPmNISOV/6pig6rtAquikK+tFqe79QE64nbWvbP3bETkU7sOS+NDSfqUK6LK1kPr0bS1GWgkh2AzUEKYSxbsduUcYebBioogIBCz5jExo8biWoIDjM7aDOnMtIBvsCp4iytspZgjM0Zo94POBVKWQtXrdreoHa4FNM7i4rk8jQE91Rix7pnp7p49da9aBG28sxz3pHK5zd8bOprfd/VRWefRx+qf707GMfi4FXyd6dgGnfwujoaycXra2lzUx9lq7phf1fUZMe+AZW7c242RCbDUT5A9Sfz90N8vNZlbLqxnxqeSX8ikEkMIM5fu8FLGMN68gF7s5xiUbkVYkr00qik6pEQs0kvoWS/gTObourAAACanichVG7SgNBFD1Z3/EVY6OkCSYRq3DXRrESbSx95QGJyO466uBmd9ndBDT4A3ZWgqkURMTPsPEHLPIJol0EGwtvNguiQb3DzJw5c8+dMzO6Y0rPJ2pGlJ7evv6BwaHo8Mjo2HhsIp737KpriJxhm7Zb1DVPmNISOV/6pig6rtAquikK+tFqe79QE64nbWvbP3bETkU7sOS+NDSfqUK6LK1kPr0bS1GWgkh2AzUEKYSxbsduUcYebBioogIBCz5jExo8biWoIDjM7aDOnMtIBvsCp4iytspZgjM0Zo94POBVKWQtXrdreoHa4FNM7i4rk8jQE91Rix7pnp7p49da9aBG28sxz3pHK5zd8bOprfd/VRWefRx+qf707GMfi4FXyd6dgGnfwujoaycXra2lzUx9lq7phf1fUZMe+AZW7c242RCbDUT5A9Sfz90N8vNZlbLqxnxqeSX8ikEkMIM5fu8FLGMN68gF7s5xiUbkVYkr00qik6pEQs0kvoWS/gTObourAAACanichVG7SgNBFD1Z3/EVY6OkCSYRq3DXRrESbSx95QGJyO466uBmd9ndBDT4A3ZWgqkURMTPsPEHLPIJol0EGwtvNguiQb3DzJw5c8+dMzO6Y0rPJ2pGlJ7evv6BwaHo8Mjo2HhsIp737KpriJxhm7Zb1DVPmNISOV/6pig6rtAquikK+tFqe79QE64nbWvbP3bETkU7sOS+NDSfqUK6LK1kPr0bS1GWgkh2AzUEKYSxbsduUcYebBioogIBCz5jExo8biWoIDjM7aDOnMtIBvsCp4iytspZgjM0Zo94POBVKWQtXrdreoHa4FNM7i4rk8jQE91Rix7pnp7p49da9aBG28sxz3pHK5zd8bOprfd/VRWefRx+qf707GMfi4FXyd6dgGnfwujoaycXra2lzUx9lq7phf1fUZMe+AZW7c242RCbDUT5A9Sfz90N8vNZlbLqxnxqeSX8ikEkMIM5fu8FLGMN68gF7s5xiUbkVYkr00qik6pEQs0kvoWS/gTObourAAACanichVG7SgNBFD1Z3/EVY6OkCSYRq3DXRrESbSx95QGJyO466uBmd9ndBDT4A3ZWgqkURMTPsPEHLPIJol0EGwtvNguiQb3DzJw5c8+dMzO6Y0rPJ2pGlJ7evv6BwaHo8Mjo2HhsIp737KpriJxhm7Zb1DVPmNISOV/6pig6rtAquikK+tFqe79QE64nbWvbP3bETkU7sOS+NDSfqUK6LK1kPr0bS1GWgkh2AzUEKYSxbsduUcYebBioogIBCz5jExo8biWoIDjM7aDOnMtIBvsCp4iytspZgjM0Zo94POBVKWQtXrdreoHa4FNM7i4rk8jQE91Rix7pnp7p49da9aBG28sxz3pHK5zd8bOprfd/VRWefRx+qf707GMfi4FXyd6dgGnfwujoaycXra2lzUx9lq7phf1fUZMe+AZW7c242RCbDUT5A9Sfz90N8vNZlbLqxnxqeSX8ikEkMIM5fu8FLGMN68gF7s5xiUbkVYkr00qik6pEQs0kvoWS/gTObour
• Okuno, Hada, Shimodaira (ICML 2018) A probabilistic framework for multi-view feature learning with many-to-many associations via neural networks
• Okuno, Shimodaira (ICML workshop Theoretical Foundations and Applications of Deep Generative Models 2018) On representation power of neural network-based graph embedding and beyond
• Okuno, Kim, Shimodaira (to appear AISTATS 2019) Graph Embedding with Shifted Inner Product Similarity and Its Improved Approximation Capability
• Kim, Okuno, Fukui, Shimodaira (arXiv:1902.10409) Representation Learning with Weighted Inner Product for Universal Approximation of General Similarities
• Okuno, Shimodaira (to appear AISTATS 2019) Robust Graph Embedding with Noisy Link Weights
•
Universal Approximation Mercer
• mini-batch SGD)
•
•
• (IPS) positive definite, PD) ( )
• Shifted Inner Product Similarity (SIPS)
• SIPS (conditionally PD, CPD)
• CPD ( ) PoincareWasserstein
• Inner Product Difference Similarity (IPDS)
• IPDS (indefinite)
IPS ( SIPS (
• Weighted Inner Product Similarity (WIPS)
•(indefinite)
•
• (β- )( β-
• density power divergence
Cosine sim., Gauss kernel, …
Negative Poincare distance, …
Negative Jeffrey’s divergence, …CPD similarities
General similarities
PD similarities IPS
SIPS
IPDS and WIPS (Proposed)
β=0 β=0.5 β=1.0
β
• Kim, Fukui, Shimodaira (EMNLP Workshop on Noisy User-generated Text W-NUT 2018), Word-like character n-gram embedding
• Kim, Fukui, Shimodaira (to appear NAACL-HLT 2019), Segmentation-free compositional n-gram embedding
• Shimodaira, Terada (arXiv:1902.04964), Selective Inference for Testing Trees and Edges in Phylogenetics
•
p
•(selective inference)
• (Terada, Shimodaira arXiv:1711.00949)(Shimodaira 2002, 2004,
2008)p-
•p-
• Pham, Sheridan, Shimodaira (to appear Journal of Statistical Software 2019), PAFit: an R Package for Estimating Preferential Attachment and Node Fitness in Temporal Complex Networks
• Inoue, Pham, Shimodaira (Unifying Themes in Complex Systems IX. ICCS 2018. Springer Proceedings in Complexity. Springer), Transitivity vs Preferential Attachment: Determining the Driving Force Behind the Evolution of Scientific Co-Authorship Networks
•(PAFit)
•)
•
• Imori, Shimodaira (arXiv:1902.07954), An information criterion for auxiliary variable selection in incomplete data analysis
•
•
•
•
•
• AIC
• CMP:• HEP:• SMJ: