Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian...
Transcript of Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian...
![Page 1: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/1.jpg)
Justin SolomonPrinceton University
David BommesRWTH Aachen University
![Page 2: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/2.jpg)
Optimization.
Synonym(-ish): Variational methods.
![Page 3: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/3.jpg)
Optimization.
Synonym(-ish): Variational methods.
Caveat: Slightly different connotation in ML
![Page 4: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/4.jpg)
![Page 5: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/5.jpg)
ClientWhich optimization tool is relevant?
DesignerCan I design an algorithm for this problem?
![Page 6: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/6.jpg)
Optimization is a huge field.
Patterns, algorithms, & examples common in geometry processing.
![Page 7: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/7.jpg)
Vocabulary
Simple examples
Unconstrained optimization
Equality-constrained optimization
Part I (Justin)
![Page 8: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/8.jpg)
Inequality constraints
Advanced algorithms
Discrete problems
Conclusion
Part II (David)
![Page 9: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/9.jpg)
Vocabulary
Simple examples
Unconstrained optimization
Equality-constrained optimization
Part I (Justin)
![Page 10: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/10.jpg)
Objective (“Energy Function”)
![Page 11: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/11.jpg)
Equality Constraints
![Page 12: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/12.jpg)
Inequality Constraints
![Page 13: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/13.jpg)
Gradienthttps://en.wikipedia.org/?title=Gradient
![Page 14: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/14.jpg)
Hessianhttp://math.etsu.edu/multicalc/prealpha/Chap2/Chap2-5/10-3a-t3.gif
![Page 15: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/15.jpg)
Jacobianhttps://en.wikipedia.org/wiki/Jacobian_matrix_and_determinant
![Page 16: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/16.jpg)
Critical point
(unconstrained)
Saddle point
Local min
Local max
![Page 17: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/17.jpg)
Critical points
may not be minima.
![Page 18: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/18.jpg)
![Page 19: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/19.jpg)
Numerical Algorithms, Solomon
More later
![Page 20: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/20.jpg)
Vocabulary
Simple examples
Unconstrained optimization
Equality-constrained optimization
Part I (Justin)
![Page 21: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/21.jpg)
![Page 22: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/22.jpg)
How effective are generic
optimization tools?
![Page 23: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/23.jpg)
Try the
simplest solver first.
![Page 24: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/24.jpg)
(assume A is symmetric and positive definite)
![Page 25: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/25.jpg)
Normal equations(better solvers for this case!)
![Page 26: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/26.jpg)
G. Peyré, mesh processing course slides
![Page 27: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/27.jpg)
𝒘𝒊𝒋 ≡ 𝟏: Tutte embedding
𝒘𝒊𝒋 from mesh: Harmonic embedding
Assumption: 𝒘 symmetric.
![Page 28: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/28.jpg)
Laplacian matrix!
![Page 29: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/29.jpg)
K. Crane, brickisland.net
Leads to famous cotangent weights!Useful for interpolation.
![Page 30: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/30.jpg)
Never construct 𝑨−𝟏 explicitly(if you can avoid it)
Added structure helpsSparsity, symmetry, positive definite
![Page 31: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/31.jpg)
Direct (explicit matrix)
Dense: Gaussian elimination/LU, QR for least-squares
Sparse: Reordering (SuiteSparse, Eigen)
Iterative (apply matrix repeatedly)
Positive definite: Conjugate gradients
Symmetric: MINRES, GMRES
Generic: LSQR
![Page 32: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/32.jpg)
Induced by the connectivity of the triangle mesh.
Iteration of CG has local effect⇒ Precondition!
![Page 33: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/33.jpg)
What if 𝑽𝟎 = {}?
![Page 34: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/34.jpg)
Prevents trivial solution 𝒙 ≡ 𝟎.
Extract the smallest eigenvalue.
![Page 35: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/35.jpg)
Prevents trivial solution 𝒙 ≡ 𝟎.N contains basis for null space of A.
Extract the smallest nonzero eigenvalue.
![Page 36: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/36.jpg)
Mullen et al. “Spectral Conformal Parameterization.” SGP 2008.
![Page 37: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/37.jpg)
2 3 4 5 6 7 8 9 10
“Laplace-Beltrami Eigenfunctions”
![Page 38: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/38.jpg)
![Page 39: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/39.jpg)
Roughly:
1.Extract Laplace-Beltrami eigenfunctions:
2.Find mapping matrix (linear solve!):
Ovsjanikov et al. “Functional Maps.” SIGGRAPH 2012.
![Page 40: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/40.jpg)
Vocabulary
Simple examples
Unconstrained optimization
Equality-constrained optimization
Part I (Justin)
![Page 41: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/41.jpg)
![Page 42: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/42.jpg)
Unstructured.
![Page 43: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/43.jpg)
Gradient descent
![Page 44: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/44.jpg)
Gradient descent
Line search
![Page 45: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/45.jpg)
Gradient descent
Multiple optima!
![Page 46: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/46.jpg)
Accelerated gradient descent
Quadratic convergence on convex problems!(Nesterov 1983)
![Page 47: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/47.jpg)
Newton’s Method
1
2
3
Line search for stability
![Page 48: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/48.jpg)
Quasi-Newton: BFGS and friends
Hessian approximation
(Often sparse) approximation from previous samples and gradients
Inverse in closed form!
![Page 49: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/49.jpg)
Often continuous gradient descent
M. Kazhdan
![Page 50: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/50.jpg)
Fröhlich and Botsch. “Example-Driven Deformations Based on Discrete Shells.” CGF 2011.
![Page 51: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/51.jpg)
Roughly:
1. Linearly interpolate edge lengths and dihedral angles.
2. Nonlinear optimization for vertex positions.
Sum of squares: Gauss-Newton
![Page 52: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/52.jpg)
Matlab: fminunc or minfunc C++: libLBFGS, dlib, others
Typically provide functions for function and gradient (and optionally, Hessian).
Try several!
![Page 53: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/53.jpg)
Vocabulary
Simple examples
Unconstrained optimization
Equality-constrained optimization
Part I (Justin)
![Page 54: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/54.jpg)
![Page 55: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/55.jpg)
- Decrease f: −𝛁𝒇- Violate constraint: ±𝛁𝒈
![Page 56: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/56.jpg)
Want:
![Page 57: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/57.jpg)
![Page 58: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/58.jpg)
Turns constrained optimization into
unconstrained root-finding.
![Page 59: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/59.jpg)
ReparameterizationEliminate constraints to reduce to unconstrained case
Newton’s methodApproximation: quadratic function with linear constraint
Penalty methodAugment objective with barrier term, e.g. 𝒇 𝒙 + 𝝆|𝒈 𝒙 |
![Page 60: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/60.jpg)
(assume A is symmetric and positive definite)
![Page 61: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/61.jpg)
No longer positive definite!
![Page 62: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/62.jpg)
Example: Levenberg-Marquardt
Fix (or adjust) damping parameter
𝝀 > 𝟎.
![Page 63: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/63.jpg)
Huang et al. “L1-Based Construction of Polycube Maps from Complex Shapes.” TOG 2014.
![Page 64: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/64.jpg)
Align with coordinate axes
Preserve area
Note: Final method includes several more terms!
![Page 65: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/65.jpg)
![Page 66: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/66.jpg)
Try lightweight options
versus
![Page 67: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/67.jpg)
Try lightweight options
versus
![Page 68: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/68.jpg)
Repeatedly solve linear systems
“Geometric median”
![Page 69: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/69.jpg)
d can be aBregman divergence
![Page 70: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/70.jpg)
Decompose as sum of hard part f and easy part g.
https://blogs.princeton.edu/imabandit/2013/04/11/orf523-ista-and-fista/
![Page 71: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/71.jpg)
Add constraint to objective
Does nothing when constraint is satisfied
![Page 72: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/72.jpg)
https://web.stanford.edu/~boyd/papers/pdf/admm_slides.pdf
![Page 73: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/73.jpg)
Want two easy subproblems
Augmented part
Solomon et al. “Earth Mover’s Distances on Discrete Surfaces.” SIGGRAPH 2014.
![Page 74: Justin Solomon David Bommes · 2015-07-06 · Direct (explicit matrix) Dense: Gaussian elimination/LU, QR for least-squares Sparse: Reordering (SuiteSparse, Eigen) Iterative (apply](https://reader033.fdocuments.in/reader033/viewer/2022050512/5f9ce8d927d46f06f25dbfb8/html5/thumbnails/74.jpg)
https://en.wikipedia.org/wiki/Frank%E2%80%93Wolfe_algorithm
Linearize objective, not constraints