Complexity of Polynomial Systems

David PritchardWaterloo URA Seminar

May 16, 2007

Preliminaries

• Polynomial equation: any expression using variables, constants, addition/subtraction, multiplication, whole powers, and one “=”

• Polynomial system: multiple equations in a common set of variables. E.g.,

• Solution: a value for each variable, so after substituting, all equations are true. E.g.,

x3 ¡ 4y2 +5= 0; x = yz; 3(x +y)100 = ab

x = ¡ 1;y = 1;z = ¡ 1;a = 0;b= 2007:

Motivation

• Polynomial systems come up all over the place. Want general technique to answer: is there a solution? what is it?

• Special case, e.g.: quadratic formula.• For this talk “general technique”

means “computer program”• Want programs to be fast. Leads us to

talk about computational complexity

Terminology

• Important special case of polynomial is linear: no multiplication/powers of variables allowed. E.g.,

• In high school, learn how to solve 2x2 systems of linear equalities

x +3y = 5; (x +1)y+3z = 5; x2 ¡ x = 0

Game Plan

Real variables, linear

polynomials

Integer variables, linear

polynomialsReal* variables,

arbitrary polynomials

Integer variables, arbitrary

polynomials• Will discuss equalities and inequalities

* or complex

Crash Course in Computational Complexity

• A computer algorithm (program) is polynomial* time (P-time) if its running time on input I is at most

for some choice of constants C1, C2

• Math: “time” = steps on Turing machine. Us: “time” = ms of (say) Java program. Both notions are the same!

C1size(I )C2

* Distinct usage from polynomial system

Real Linear Equality Systems

• Mentioned 2 var, 2 equation case• General strategy: Gaussian Elimination

– solve for one variable x in terms of others– eliminate all occurrences of x– repeat: solve & eliminate another var, etc

• Elimination ever gives “0=1”: no sol'n • Else back-substitute at end, get sol'n

Gaussian Elimination: Time

• Say n vars, m eqns. Input size = n+m• # arithm’c operations to perform one

round of substitution: 2nm• Each round elims a var n rounds• Total operations: 2n2m < 2(input size)3

hence Gaussian elim’n is P-time• (We cheated - coefficient sizes, inter-

mediate expression swell! Still P-time)

Real Linear Inequality Systems

• Related to “linear programs:”maximize f(x, y, …) subject to c1(x, y, …) ≥ 0, c2(x, y, …) ≥ 0,…

where f, c1, c2, …, are linear functions• In fact optimizing is no harder than

solving; use binary search likesolve f(x, y, …) ≥ 10, f(x, y, …) ≤20,

c1(x, y, …) ≥ 0, c2(x, y, …) ≥ 0,…

Linear Program Duality

• Notice (beef=1/2, lamb=1/4, oil=1/4) gives cost $2.75. Is it optimal?

• Yes! Add equations (1)+(2)+0.5*(3):

minimize $3¢beef +$4¢lamb+$1¢oil

s.t. beef + lamb+oil ¸ 1 (1)

beef + lamb¸ 3=4 (2)

2¢beef +4¢lamb¸ 2 (3)

3¢beef +4¢lamb+oil ¸ 1+ 3=4+0:5¢2= 2:75

• We can get a P-time solution, tricky

Scorecard

Real variables, linear polynom’s

P-time

Integer variables, linear

polynom’s

Real variables, any polynomials

Integer variables, any polynomials

[Khachiyan 1972]

Integer Linear Systems

• Application: similar to LP food example, but where can't break units (e.g., scheduling employees)

• Hard! Even in special case where all variables must be 0 or 1

• Naive algorithm: try all possibilities

• n vars: need to try 2n combinations,

not P-time algorithm. Is there one?

What is NP-hardness? (NPh)

• A problem (not algorithm) can be NPh• Doesn’t mean Non-Polynomial — actually

“n” stands for nondeterministic• Yet, accepted evidence that no polynomial

algorithm exists for the problem• Proof: reduce another NPh problem to it.

Many, many problems are known to be NPh• Why evidence? Despite trying, no known P-

time solution to any NPh problem• Suspicion: none exist! (i.e., P != NP)

Integer Linear Programming

• 0-1 integer linear programming is NPh (Karp, 1972)

• Removing “0-1” only makes harder!• Effort still focused on practical

solutions; for NPh problems, “real-life” instances may not be the hardest ones

• E.g., used to solve traveling salesman problem (NPh) on 100000s of cities

• One difficulty: no duality =(

Scorecard

Real variables, linear polynom’s

P-time

Integer variables, linear

polynom’sNP-hard

Real variables, any polynomials

Integer variables, any polynomials

[Khachiyan 1972] [Karp 1972]

Real/Complex Polynomial Systems

• Real vs complex leads to vastly different approaches to solve a polynomial system

• Both are NP-hard! Proof…• Reduce to 0-1 integer linear programming:

– Suppose we can solve real polynomial systems– Then we could solve 0-1 integer linear

programs too: take linear constraints, add constraint x2=x for each variable x

– Increases problem size (#constraints, #vars) by only a little bit (a polynomial amount)

Systems in : Gröbner Bases

• Simplest Q: does a system have sol’n?• Recall: if we can manipulate system’s

equations to get “0=1”, no solution• Basic idea: normalize system so that

we can compute remainders (like mod)• Division by a set of polynomials is

straightforward but can give multiple answers if set is not a Gröbner basis

Sketch of Complex Case

1. Normalize system to a Gröbner basis2. Compute remainder of 1 in radical3. If remainder is 0, there’s no solution;

else remainder is 1, there’s a solution!• Depends on the fact that complex

numbers are algebraically closed• Not as straightforward to numerically

compute a solution

Reals: Cylindrical Algebraic Decomposition

• Idea: polynomial system decomposes Euclidean space into several parts.

• Compute parts recursively: use elimination (resultant) to reduce #vars

y4 ¡ 2y3 + y2 ¡ 3x2y + 2x4 = 0

e.g. from [Arnon, Collins, McCallum 1984]

Reals: Cylindrical Algebraic Decomposition

Decomp’n of R by g:

f (x;y) = y4 ¡ 2y3 + y2 ¡ 3x2y + 2x4

) y-resultant g(x) = 2048x12 ¡ 4608x10 + 37x8 + 12x6

Decomp’n of R2 by f:

Scorecard

Real variables, linear polynom’s

P-time

Integer variables, linear

polynom’sNP-hard

Real variables, any polynomials

NP-hard

Integer variables, any polynomials

[Khachiyan 1972] [Karp 1972]

[Buchberger 1965; Tarski 1930s]

Integer Variables + Polynomials = Diophantine

Equations• 1900, David Hilbert gave 23

problems to the international mathematics congress in Paris

• Problem 10: “Find an algorithm to determine whether a given polynomial Diophantine equation with integer coefficients has an integer solution”

• Exactly what we want to do!

(Different Formulations)

• Whole number variables vs integer variables? Can write one using other:(whole number) – (whole number)

= any integer [obvious](integer)2 + (integer)2 + (integer)2 + (integer)2 = any whole number[Lagrange’s 4-squares theorem]

• Combine multiple eqn’s: {a=b, c=d} (a-b)2+(c-d)2=0

How to Solve Diophantine Eq’ns?

• Can’t just try all possible solutions: if none found, can’t tell when to stop looking.

• Any other technique works?• No!• Matiyasevich, 1970: No computer

program exists that can determine solvability of Diophantine systems

Matiyasevich’s Theorem

• Main contribution: proved anything a computer can do, a Diophantine sys’m can do

• Random e.g.:this system has solutionif and only ifk+2 is prime

0 = wz + h + j - q0 = (gk + 2g + k + 1)(h + j) + h - z0 = 16(k + 1)3(k + 2)(n + 1)2 + 1 - f2

0 = 2n + p + q + z - e0 = e3(e + 2)(a + 1)2 + 1 - o2

0 = (a2 - 1)y2 + 1 – x2

0 = 16r2y4(a2 - 1) + 1 - u2

0 = n + l + v - y0 = (a2 - 1)l2 + 1 - m2

0 = ai + k + 1 - l - i0 = ((a + u2(u2 - a))2 - 1)(n + 4dy)2 + 1 - (x + cu)2

0 = p + l(a - n - 1) + b(2an + 2a - n2 - 2n - 2) - m0 = q + y(a - p - 1) + s(2ap + 2a - p2 - 2p - 2) - x0 = z + pl(a - p) + t(2ap - p2 - 1) - pm

• Halting problem: Is there a computer program which takes other computer programs as input, and outputs whether or not they ever stop?

• Answer: No! Proof: pretty! [not here](Turing, 1936)

• Hence: no program to solve Diophantine systems since (using simulation result) it would solve halting problem

Scorecard

Real variables, linear polynom’s

P-time

Integer variables, linear

polynom’sNP-hard

Real variables, any polynomials

NP-hard

Integer variables, any polynomials

Uncomputable

[Khachiyan 1972] [Karp 1972]

[Tarski 1930s; Buchberger 1965]

[Matiyasevich 1970]

Endgame

• Many linear program solvers exist and simplex method doable by hand

• Many integer LP solvers; can be much slower

• Gröbner bases in Maple and…• CAD in Mathematica and…• Too bad for Diophantus and Hilbert

=(