Sparse Bundle Adjustment for Dense Data · Sparse Bundle Adjustment for Dense Data ... Sparse...
Transcript of Sparse Bundle Adjustment for Dense Data · Sparse Bundle Adjustment for Dense Data ... Sparse...
Sparse Bundle Adjustment for Dense Data
Kurt Konolige
Willow Garage, Stanford CS
Bundle Adjustment
𝑝1 𝑝2
𝑞2 … 𝑞3 𝑞1
𝑝𝑟𝑗 𝑝𝑟𝑗 𝑝𝑟𝑗 𝑝𝑟𝑗 𝑝𝑟𝑗 𝑝𝑟𝑗
𝑒𝑘 = 𝑝𝑟𝑗𝑘(𝑝𝑖 , 𝑞𝑗) − 𝑧 𝑘
Minimize sum of squared error
Photo tourism [Snavely, Seitz, Szeliski 2006]
Large nonlinear estimation problem: Bundle Adjustment
Thousands of camera poses, millions of points
Sparse Bundle Adjustment
𝑒𝑘 = 𝑧𝑘(𝑥𝑖 , 𝑥𝑗) − 𝑧 𝑘 Error Equation
𝑱𝑇𝑱𝒙 = 𝑱𝑇𝒆 Normal Equation
𝐽𝑘 =𝜕𝑒𝑘𝜕𝑥 𝑥 Jacobians
i
i
j
j
Lourakis and Argyros [TR 2004] Konolige [BMVC 2010] Jeong et al. [CVPR 2010]
1 10 100 1000
Jeong
Konolige
LA
Time to convergence, seconds
2,010,363 vars, 2,857,277 constraints
[Gutmann and Konolige CIRA 1999]
g2o: A General Framework for Graph Optimization Kummerle, Grisetti, Strasdat, Konolige, Burgard [ICRA 2011] Open Source!
Issues
• Growth of dataset
• How to incorporate ICP for dense depth
• How to set up a "surface" database
Hauke Strasdat et al., ICCV 2011
Hauke Strasdat et al., ICCV 2011
Generalized ICP [Segal et al. 2005]
+ Matching of neighbors
T d
𝑝1 𝑝2 ℎ
ℎ
ℎ
…
…
𝑒𝑘 = ℎ𝑘(𝑝𝑖 , 𝑝𝑗) − 𝑧 𝑘
𝑝1 𝑝2
𝑞2
𝑓 𝑝1, 𝑝2, 𝑞1, 𝑞2, 𝑞3, … = 𝑝𝑟𝑗𝑖 𝑝1, 𝑞𝑖 𝑝𝑟𝑗𝑖 𝑝2, 𝑞𝑖 𝑖𝑐𝑝𝑖 𝑝1, 𝑝2𝑖𝑖
… 𝑞3 𝑞1
𝑝𝑟𝑗 𝑝𝑟𝑗 𝑝𝑟𝑗 𝑝𝑟𝑗 𝑝𝑟𝑗 𝑝𝑟𝑗
𝑖𝑐𝑝
𝑖𝑐𝑝
𝑖𝑐𝑝
Plane-plane ICP Point-plane ICP
Visual Features and ICP
ICP & Visual Features†
Global BA
• Incremental pairwise
alignment;
• Global optimization performed with all the
accumulated constraints;
• 72 nodes, 77899 edges => 346ms
Visual Features†
only
Timing Performance
Based on 823
pairwise alignment
tests ӿ.
Results
ӿ GICP is an open-source implementation of plane-to-plane generalized ICP
(Segal et al. “Generalized-ICP”, 2009)
† BRIEF features from FAST keypoints.
Combined ICP and Visual Features [Fioraio and Konolige 2011]
𝑝1 𝑝2
𝑞2 … 𝑞3 𝑞1
𝑝𝑟𝑗
𝑟𝑒𝑔 𝑟𝑒𝑔
𝑝𝑟𝑗 𝑝𝑟𝑗 𝑝𝑟𝑗 𝑝𝑟𝑗
Sparse Surface Adjustment [Ruhnke et al. 2011]
+ Matching of neighbors
T
𝑒𝑚𝑒𝑎𝑠
𝑒𝑠𝑢𝑟𝑓
M M
Relation to Dense Methods
Dense tracking – stabilizes the front end [ESM, Comport et al. 2010, DTAM]
Dense surface methods have difficulty with “adjustment” after
loop closure
ICP bundle adjustment doesn’t make surfaces first-class objects
Adding in visual features
Sparse Surface Patch Adjustment
T
𝑒𝑚𝑒𝑎𝑠
M
𝑒𝑚𝑒𝑎𝑠