Solving Polynomial Systems by Homotopy Continuation · 9/13/2006 · nAlgebraic Geometry and...
Transcript of Solving Polynomial Systems by Homotopy Continuation · 9/13/2006 · nAlgebraic Geometry and...
Algebraic Geometry and Applications Seminar IMA, September 13, 2006
Solving Polynomial Systems by Homotopy Continuation
Andrew SommeseUniversity of Notre Dame
www.nd.edu/~sommese
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 2
n Reference on the area up to 2005:n A.J. Sommese and C.W. Wampler, Numerical
solution of systems of polynomials arising in engineering and science, (2005), World Scientific Press.
n Survey covering other topics n T.Y. Li, Numerical solution of polynomial
systems by homotopy continuation methods, in Handbook of Numerical Analysis, Volume XI, 209-304, North-Holland, 2003.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 3
Overview
n Solving Polynomial Systemsn Computing Isolated Solutions
n Homotopy Continuationn Case Study: Alt’s nine-point path synthesis problem for planar four-bars
n Positive Dimensional Solution Setsn How to represent themn Decomposing them into irreducible components
n Numerical issues posed by multiplicity greater than one componentsn Deflation and Endgamesn Bertini and the need for adaptive precision
n A Motivating Problem and an Approach to Itn Fiber Productsn A positive dimensional approach to finding isolated solutions equation-by-
equation
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 4
Solving Polynomial Systems
n Find all solutions of a polynomial system on:
0),...,(f
),...,(f
1n
11
=
N
N
xx
xxM
NC
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 5
Why?
n To solve problems from engineering and science.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 6
Characteristics of Engineering Systems
n systems are sparse: they often have symmetries and have much smaller solution sets than would be expected.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 7
Characteristics of Engineering Systems
n systems are sparse: they often have symmetries and have much smaller solution sets than would be expected.
n systems depend on parameters: typically they need to be solved many times for different values of the parameters.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 8
Characteristics of Engineering Systems
n systems are sparse: they often have symmetries and have much smaller solution sets than would be expected.
n systems depend on parameters: typically they need to be solved many times for different values of the parameters.
n usually only real solutions are interesting
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 9
Characteristics of Engineering Systems
n systems are sparse: they often have symmetries and have much smaller solution sets than would be expected.
n systems depend on parameters: typically they need to be solved many times for different values of the parameters.
n usually only real solutions are interesting.n usually only finite solutions are interesting.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 10
Characteristics of Engineering Systems
n systems are sparse: they often have symmetries and have much smaller solution sets than would be expected.
n systems depend on parameters: typically they need to be solved many times for different values of the parameters.
n usually only real solutions are interesting.n usually only finite solutions are interesting.n nonsingular isolated solutions were the center of
attention.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 11
Computing Isolated Solutions
n Find all isolated solutions in of a system on n polynomials:
NC
0),...,(f
),...,(f
1n
11
=
N
N
xx
xxM
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 12
Solving a system
n Homotopy continuation is our main tool: n Start with known solutions of a known start
system and then track those solutions as we deform the start system into the system that we wish to solve.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 13
Path Tracking
This method takes a system g(x) = 0, whose solutions we know, and makes use of a homotopy, e.g.,
Hopefully, H(x,t) defines “nice paths” x(t) as t runs from 1 to 0. They start at known solutions of
g(x) = 0 and end at the solutions of f(x) at t = 0.
tg(x). t)f(x)-(1 t)H(x, +=
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 14
n The paths satisfy the Davidenko equation
n To compute the paths: use ODE methods to predict and Newton’s method to correct.
tH
dtdx
xH
dt t)dH(x(t),
0N
1
i
i ∂∂
+∂∂
== ∑=i
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 15
nSolutions of
n f(x)=0
nKnown solutions of g(x)=0
nt=0 nt=1nH(x,t) = (1-t) f(x) + t g(x)
nx3(t)
nx1(t)
nx2(t)
nx4(t)
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 16
nNewton correction
nprediction
{
n∆ t
nxj(t)
nx*
n01
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 17
Algorithms
n middle 80’s: Projective space was beginning to be used, but the methods were a combination of differential topology and numerical analysis with homotopies tracked exclusively through real parameters.
n early 90’s: algebraic geometric methods worked into the theory: great increase in security, efficiency, and speed.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 18
Uses of algebraic geometry
Simple but extremely useful consequence of algebraicity [A. Morgan (GM R. & D.) and S.]
n Instead of the homotopy H(x,t) = (1-t)f(x) + tg(x) use H(x,t) = (1-t)f(x) + γtg(x)
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 19
Genericity
n Morgan + S. : if the parameter space is irreducible, solving the system at a random point simplifies subsequent solves: in practice speedups by factors of 100.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 20
Endgames (Morgan, Wampler, and S.)
n Example: (x – 1)2 - t = 0We can uniformize around a solution at t = 0. Lettingt = s2, knowing the solutionat t = 0.01, we can trackaround |s| = 0.1 and useCauchy’s Integral Theorem to compute x at s = 0.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 21
n Special Homotopies to take advantage of sparseness
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 22
Multiprecision
n Not practical in the early 90’s!n Highly nontrivial to design and dependent on
hardwaren Hardware too slow
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 23
Hardware
n Continuation is computationally intensive. On average:n in 1985: 3 minutes/path on largest mainframes.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 24
Hardware
n Continuation is computationally intensive. On average:n in 1985: 3 minutes/path on largest mainframes.n in 1991: over 8 seconds/path, on an IBM 3081;
2.5 seconds/path on a top-of-the-line IBM 3090.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 25
Hardware
n Continuation is computationally intensive. On average:n in 1985: 3 minutes/path on largest mainframes.n in 1991: over 8 seconds/path, on an IBM 3081;
2.5 seconds/path on a top-of-the-line IBM 3090.n 2006: about 10 paths a second on a single
processor desktop CPU; 1000’s of paths/second on moderately sized clusters.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 26
A Guiding Principle then and now
n Algorithms must be structured – when possible – to avoid extra paths and especially those paths leading to singular solutions: find a way to never follow the paths in the first place.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 27
Continuation’s Core Computation
n Given a system f(x) = 0 of n polynomials in n unknowns, continuation computes a finite set S of solutions such that:n any isolated root of f(x) = 0 is contained in S; n any isolated root “occurs” a number of times
equal to its multiplicity as a solution of f(x) = 0;n S is often larger than the set of isolated
solutions.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 28
Case Study: Alt’s Problem
n We follow
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 29
n A four-bar planar linkage is a planar quadrilateral with a rotational joint at each vertex.
n They are useful for converting one type of motion to another.
n They occur everywhere.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 30
How Do Mechanical Engineers Find Mechanisms?
n Pick a few points in the plane (called precision points)
n Find a coupler curve going through those points
n If unsuitable, start over.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 31
n Having more choices makes the process faster.
n By counting constants, there will be no coupler curves going through more than nine points.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 32
Nine Point Path-Synthesis Problem
H. Alt, Zeitschrift für angewandte Mathematikund Mechanik, 1923:
n Given nine points in the plane, find the set of all four-bar linkages, whose coupler curves pass through all these points.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 33
n First major attack in 1963 by Freudensteinand Roth.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 34
D'
Pj
dj
?j
µj
u b v
CD
x
y
P0
C'
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 35
Pj
dj
µj
b-dj
v
C
y
P0
? j
yei?j
b
v = y – b
veiµj = yei?j - (b - dj)
= yei?j + dj - b
C'
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 36
We use complex numbers (as is standard in this area)
Summing over vectors we have 16 equations
plus their 16 conjugates
byeeby jii jj −+=− δθµ)(
axeeax jii jj −+=− δθλ)(
beyeby jii jj −+=− −− δθµ)(
aexeax jii jj −+=− −− δθλ)(
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 37
This gives 8 sets of 4 equations:
in the variables a, b, x, y, and for j from 1 to 8.
byeeby jii jj −+=− δθµ)(
axeeax jii jj −+=− δθλ)(
beyeby jii jj −+=− −− δθµ)(
aexeax jii jj −+=− −− δθλ)(
,y ,x ,b ,a
jjj ,,? θµ
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 38
Multiplying each side by its complex conjugate and letting we get 8 sets of 3 equations
in the 24 variables with j from 1 to 8.
[ ] [ ] 0dd - x)- a(d )x - a(d? x)d - (a ?)xd - a( jjjjjjjj =+++
[ ] [ ] 0dd - y) - b(d )y - b(d? y)d - (b ?)yd - b( jjjjjjjj =+++
0???? jjjj =++
jj ?, ?and y ,x ,b ,a y, x,b, a,
1e? ji?j −=
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 39
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 40
in the 24 variableswith j from 1 to 8.
[ ] [ ] 0dd - x) -a (d )x - a(d? x)d -(a ?)xd - a( jjjjjjjj =+++
[ ] [ ] 0dd - y)- b(d )y - b(d? y)d - (b ?)yd - b( jjjjjjjj =+++
0???? jjjj =++
jj ?, ?and y ,x ,b ,a y, x,b, a,
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 41
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 42
Using Cramer’s rule and substitution we have what is essentially the Freudenstein-Roth system consisting of 8 equations of degree 7.Impractical to solve in early 90’s:
78 = 5,764,801solutions.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 43
n Newton’s method doesn’t find many solutions: Freudenstein and Roth used a simple form of continuation combined with heuristics.
n Tsai and Lu using methods introduced by Li, Sauer, and Yorke found only a small fraction of the solutions. That method requires starting from scratch each time the problem is solved for different parameter values
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 44
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 45
Solve by Continuation
All 2-homog.systems
All 9-pointsystems
“numerical reduction” to test case (done 1 time)synthesis program (many times)
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 46
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 47
n Intermission
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 48
Positive Dimensional Solution Sets
We now turn to finding the positive dimensional solution sets of a system
0),...,(f
),...,(f
1n
11
=
N
N
xx
xxM
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 49
How to represent positive dimensional components?
n S. + Wampler in ’95: n Use the intersection of a component with
generic linear space of complementary dimension.
n By using continuation and deforming the linear space, as many points as are desired can be chosen on a component.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 50
n Use a generic flag of affine linear spaces to get witness point supersets
n This approach has 19th
century roots in algebraic geometry
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 51
The Numerical Irreducible Decomposition
Carried out in a sequence of articles with Jan Verschelde (University at Illinois at Chicago) and Charles Wampler (General Motors Research and Development)n Efficient Computation of “Witness Supersets’’
n S. and V., Journal of Complexity 16 (2000), 572-602.
n Numerical Irreducible Decompositionn S., V., and W., SIAM Journal on Numerical Analysis, 38
(2001), 2022-2046.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 52
n An efficient algorithm using monodromyn S., V., and W., SIAM Journal on Numerical Analysis 40
(2002), 2026-2046.
n Intersection of algebraic setsn S., V., and W., SIAM Journal on Numerical Analysis 42
(2004), 1552-1571.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 53
Symbolic Approach with same classical roots
Two nonnumerical articles in this direction:n M. Giusti and J. Heintz, Symposia Mathematica XXXIV,
pages 216-256. Cambridge UP, 1993.n G. Lecerf, Journal of Complexity 19 (2003), 564-596.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 54
The Irreducible Decomposition
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 55
Witness Point Sets
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 56
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 57
Basic Steps in the Algorithm
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 58
Example
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 59
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 60
From Sommese, Verschelde, and Wampler,SIAM J. Num. Analysis, 38 (2001), 2022-2046.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 61
Numerical issues posed by multiple components
Consider a toy homotopy
Continuation is a problem because the Jacobian withrespect to the x variables is singular.
How do we deal with this?
0),,(2
21
21 =
−=
txx
txxH
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 62
Deflation
The basic idea introduced by Ojika in 1983 is to differentiate the multiplicity away. Leykin, Verschelde, and Zhao gave an algorithm for an isolated point that they showed terminated. Given a system f, replace it with
0bzAzJf(x)
f(x)=
+⋅⋅
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 63
Bates, Hauenstein, Sommese, and Wampler:To make a viable algorithm for multiple components,
it is necessary to make decisions on ranks of singular matrices. To do this reliably, endgames are needed.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 64
Bertini and the need for adaptive precision
n Why use Multiprecision?n to ensure that the region where an endgame
works is not contained in the region where the numerics break down;
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 65
Bertini and the need for adaptive precision
n Why use Multiprecision?n to ensure that the region where an endgame
works is not contained in the region where the numerics break down;
n to ensure that a polynomial being zero at a point is the same as the polynomial numerically being approximately zero at the point;
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 66
Bertini and the need for adaptive precision
n Why use Multiprecision?n to ensure that the region where an endgame
works is not contained in the region where the numerics break down;
n to ensure that a polynomial being zero at a point is the same as the polynomial numerically being approximately zero at the point;
n to prevent the linear algebra in continuation from falling apart.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 67
Evaluation
n To 15 digits of accuracy one of the roots of this polynomial is a = 27.9999999999999. Evaluating p(a) exactly to 15 digits, we find that p(a) = - 0.0578455953407660.
n Even with 17 digit accuracy, the approximate root is a = 27.999999999999905 and we still only have p(a) = -0.0049533155737293130.
128)( 910 +−= zzzp
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 68
Wilkinson’s Theorem in Numerical Linear Algebra
n Solving Ax = f, with A an N by N matrix,we must expect to lose digits ofaccuracy. Geometrically, ison the order of the inverse of the distance in from A to to the set defined by det(A) = 0.
)](cond[log10 A
||||||||)( cond 1−⋅= AAA1−× NNP
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 69
n One approach is to simply run paths that fail over at a higher precision, e.g., this is an option in Jan Verschelde’s code, PHC.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 70
n One approach is to simply run paths that fail over at a higher precision, e.g., this is an option in Jan Verschelde’s code, PHC.
n Bertini is designed to dynamically adjust the precision to achieve a solution with a prespecified error. Bertini is being developed by Dan Bates, Jon Hauenstein, Charles Wampler, and myself (with some early work by Chris Monico). First release scheduled for October 1.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 71
Issues
n You need to stay on the parameter space where your problem is: this means you must adjust the coefficients of your equations dynamically.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 72
Issues
n You need to stay on the parameter space where your problem is: this means you must adjust the coefficients of your equations dynamically.
n You need rules to decide when to change precision and by how much to change it.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 73
n The theory we use is presented in the articlen D. Bates, A.J. Sommese, and C.W. Wampler,
Multiprecision path tracking, preprint.available at www.nd.edu/~sommese
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 74
A Motivating Problem and an Approach to It
n This is joint work with Charles Wampler. The problem is to find the families of overconstrained mechanisms of specified types.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 75
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 76
If the lengths of the six legs are fixed the platform robot is usually rigid.
Husty and Karger made a study of exceptional lengths when the robot will move: one interesting case is when the top joints and the bottom joints are in a configuration of equilateral triangles.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 77
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 78
Another Example
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 79
Overconstrained Mechanisms
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 80
To automate the finding of such mechanisms, we need to solve the following problem:n Given an algebraic map p between irreducible
algebraic affine varieties X and Y, find the irreducible components of the algebraic subset of X consisting of points x with the dimension of the fiber of p at x greater than the generic fiber dimension of the map p.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 81
An approach
n A method to find the exceptional setsn A.J. Sommese and C.W. Wampler, Exceptional
sets and fiber products, preprint.
n An approach to large systems with few solutionsn A.J. Sommese, J. Verschelde and C.W.
Wampler, Solving polynomial systems equation by equation, preprint.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 82
Summary
n Many Problems in Engineering and Science are naturally phrased as problems about algebraic sets and maps.
n Numerical analysis (continuation) gives a method to manipulate algebraic sets and give practical answers.
n Increasing speedup of computers, e.g., the recent jump into multicore processors, continually expands the practical boundary into the purely theoretical region.
nAlgebraic Geometry and Applications SeminarnIMA, September 13, 2006 83
Newton failure
The polynomial system of Griewank and Osborne (Analysis of Newton's method at irregular singularities, SIAM J. Numer. Anal. 20(4): 747--773, 1983):
Newton’s Method fails for any point sufficiently near the origin (other than the origin).
02
2
31629
=
−−
zwzwz