Post on 16-Apr-2022
1
Amir Ali AhmadiPrinceton, ORFE
(Affiliated member of PACM, COS, MAE)
CDCβ17, Tutorial LectureMelbourne
Sum of Squares Optimizationand Applications
2
Optimization over nonnegative polynomials
Basic semialgebraic set:
Why do this?!
Example: When is
nonnegative?nonnegative over a given basic semialgebraic set?
π₯ β βπ ππ π₯ β₯ 0}
Ex: π₯13 β 2π₯1π₯2
4 β₯ 0π₯14 + 3π₯1π₯2 β π₯2
6 β₯ 0
3
Optimization over nonnegative polynomials
Is π π₯ β₯ 0 on {π1 π₯ β₯ 0,β¦ , ππ π₯ β₯ 0}?
Optimization Statistics Control
β’ Lower bounds on polynomial optimization problems
β’ Fitting a polynomial to data subject to shape constraints(e.g., convexity, or monotonicity)
ππ(π₯)
ππ₯πβ₯ 0, βπ₯ β π΅
β’ Stabilizing controllers
Implies that
π₯ π π₯ β€ π½}is in the region of attraction
π π₯ > 0,π π₯ β€ π½ β π»π π₯ ππ π₯ < 0
π₯ = π(π₯)
maxπΎ
πΎ
s.t. π π₯ β πΎ β₯ 0,βπ₯ β {ππ π₯ β₯ 0}
4
How would you prove nonnegativity?
Ex. Decide if the following polynomial is nonnegative:
Not so easy! (In fact, NP-hard for degree β₯ 4)
But what if I told you:
β’Is it any easier to test for a sum of squares (SOS) decomposition?
5
SOSSDP
where
Example coming up in Antonisβ talkFully automated in YALMIP, SOSTOOLS, SPOTLESS, GloptiPoly, β¦
6
How to prove nonnegativity over a basic semialgebraic set?
Positivstellensatz: Certifies that
Putinarβs Psatz:(1993) under Archimedean condition
Stengleβs Psatz (1974)Schmudgenβs Psatz (1991)β¦
Search for ππ is an SDP when we bound the degree.
βπ π₯ = π0 π₯ + π ππ π₯ ππ π₯ ,
where ππ , π = 0,β¦ ,π are sos
All use sos polynomialsβ¦
[Lasserre, Parrilo]
Dynamics and Control
7
Lyapunov theory with sum of squares (sos) techniques
8
Lyapunov function
Ex. Lyapunovβs stability theorem.
(similar local version)
GAS
9
Global stability
GASExample.
Couple lines of code in SOSTOOLS, YALMIP, SPOTLESS, etc.
Output of SDP solver:
Theoretical limitations: converse implications may fail
10
β’ Globally asymptotically stable.
β’ But no polynomial Lyapunov function of any degree! [AAA, Krstic, Parrilo]
[AAA,Parrilo]
β’ Testing asymptotic stability of cubic vector fields is strongly NP-hard. [AAA]
Converse statements possible in special cases
11
1. Asymptotically stable homogeneous polynomial vector field Rational Lyapunov function with an SOS certificate.[AAA, El Khadir]
2. Exponentially stable polynomial vector field on a compact set Polynomial Lyapunov function.[Peet, Papachristodoulou]
3. Asymptotically stable switched linear system Polynomial Lyapunov function with an SOS certificate. [Parrilo, Jadbabaie]
4. Asymptotically stable switched linear system Convex polynomial Lyapunovfunction with an SOS certificate.[AAA, Jungers]
12
Local stability β SOS on the Acrobot
[Majumdar, AAA, Tedrake ](Best paper award - IEEE Conf. on Robotics and Automation)
Swing-up:
Balance:
Controller designed by SOS
(4-state system)
Statistics and Machine Learning
13
Monotone regression: problem definition
14
NP-hardness and SOS relaxation
15
[AAA, Curmei, Hall]
Approximation theorem
16
[AAA, Curmei, Hall]
Numerical experiments
17
Polynomial Optimization
18
Theorem: Let πΎπ,2ππ be a sequence of sets of homogeneous polynomials in
π variables and of degree 2π. If:
(1) πΎπ,2ππ β ππ,2π βπ and β π π,2π pd in πΎπ,2π
0
(2) π > 0 β βπ β β s. t. π β πΎπ,2ππ
(3) πΎπ,2ππ β πΎπ,2π
π+1 βπ
(4) π β πΎπ,2ππ β π + ππ π,2π β πΎπ,2π
π , βπ β [0,1]
A meta-theorem for producing hierarchies
19
ππ,2π
πΎπ,2ππ
=opt. val.
minπ₯ββπ
π(π₯)
π . π‘. ππ π₯ β₯ 0, π = 1,β¦ ,π
POPmaxπΎ
πΎ
π . π‘. ππΎ π§ β1
ππ π+π+3,4π π§ β πΎπ+π+3,4π
π
π β βCompactness assumptions
2π = maximum degree of π, ππ
Then,
where ππΎ is a form which can be written down explicitly from π, ππ .
Example: Artin cones π΄π,2ππ = π π β π is sos for some sos π of degree 2π}
[AAA, Hall]
20
An optimization-free converging hierarchy
[AAA, Hall]
π π₯ > 0, βπ₯ β π₯ β βπ ππ π₯ β₯ 0, π = 1, β¦ ,π}
2π =maximum degree of π, ππ
β
β π β β such that
π π£2 β π€2 β1
π π π£π
2 β π€π2 2 π
+1
2π π π£π
4 + π€π4 π
β π π£π2 + ππ€π
2 π2
has nonnegative coefficients,
where π is a form in π +π + 3 variables and of degree 4π, which can be explicitly written from π, ππ and π .
(also leads to DSOS/SDSOS-based converging hierarchies for POP)
21
Ongoing directions: large-scale/real-time verification
β’ 30 states, 14 control inputs, cubic dynamics
β’ Done with SDSOS optimization (see Georginaβs talk)
Two promising approaches:
1. LP and SOCP-based alternatives to SOS, Georginaβs talkLess powerful than SOS (Jamesβ talk), but good enough for some applications
2. Exploiting problem structure and designing customized algorithms
Antonisβ talk (next), and Pablo Parriloβs plenary (Thu. 8:30am)
Slides/references available at: aaa.princeton.edu