Polynomial Embeddings for Quadratic Algebraic Equations
description
Transcript of Polynomial Embeddings for Quadratic Algebraic Equations
Polynomial Embeddings for Quadratic Algebraic Equations
Radu Balan
University of Maryland, College Park, MD 20742
Math-CS Joint Lecture, Drexel University Monday April 23, 2012
Overview
1. Introduction: Motivation and statement of problem
2. Invertibility Results in the Real Case3. The Algebraic Approach:
a. Quasi-Linear Embeddingsb. Hierarchical Embeddingsc. Numerical Analysis
4. Modified Least Square Estimator5. Theoretical Bounds: CRLB6. Performance Analysis
2/50
1. Introduction: Motivation
Byx BA=Identity
Question: What if |y| is known instead (that is, one looses the phase information) ?
?|| xAxz
Where is important: X-Ray Crystallography, Speech Processing
knowns?
Inversion of Nonlinear Transformations
y = A x
3/50
1. Introduction: Statement of the Problem
Reconstruction from magnitudes of frame coefficients
a complete set of vectors (frame) for the n-dimensional Hilbert space H (H=Cn or H=Rn).
Equivalence relation: x,yH, x~y iff there is a scalar z, |z|=1 so that x=zy(real case: x=y ; complex case: x=eiy). Let .
Define
Problems: 1. Is an injective map?2. If it is, how to invert it efficiently?
4/50
H
Rm
x(x)
f1
f2
fm
5/50
Real Case: K=RTheorem [R.B.,Casazza, Edidin, ACHA(2006)] is injective iff for any subset JF either J or F\J
spans Rn.Corollaries [2006]• if m 2n-1, and a generic frame set F, then is
injective;• if m2n-2 then for any set F, cannot be injective;• if any n-element subset of F is linearly independent,
then is injective; for m=2n-1 this is a necessary and sufficient condition.
2. Invertibility Results: Real Case (1)
6/50
Invertibility Results: Real Case (2)Real Case: K=R
Theorem [R.B.(2012)] is injective iff any one of the following equivalent conditions holds true:
• For any x,yRn, x≠0, y≠0,
• There is a constant a>0 so that for all x, RI
7/50
Invertibility Results: Real Case (3) complete set of vectors (frame) for H=Rn.
One would expect that if is injective and m>2n-1 then there is a strict subsetJ{1,2,…,m} so that π is injective, where π:RmR|J| is the restriction to J index.
However the next example shows this is not the case.
Example. Consider n=3, m=6, and F the set of columns of the following matrix
F =
Note that for any subset J of 3 columns, either J or F\J is linearly dependent.Thus is injective but removing any column makes π not injective.
8/50
3. The Algebraic Approach3.1 Quasi-Linear EmbeddingsExample (a)Consider the real case: n=3 , m=6.Frame vectors:
Need to solve a system of the form:
9/50
[1 2 2 1 2 11 4 −2 4 −2 11 −2 −4 1 4 41 −4 −6 4 12 91 2 −14 1 −14 491 0 2 0 0 1
] [𝑦1
𝑦2
𝑦3
𝑦 4
𝑦5
𝑦6
]=[4949
1964
]𝑠 𝑜𝑙𝑣𝑒 𝑓𝑜𝑟 𝑦→𝑦=[
1−121−24
]Then factor:
Thus, we obtain:
10/50
Summary of this approach:
where
11/50
Summary of this approach:
where
12/50
Example (b)Consider the real case: n=3 , m=5.Frame vectors:
Need to solve the system:
Is it possible? How?
X
13/50
|𝑥1+𝑥2+𝑥3|=2
|𝑥1+2 𝑥2−𝑥3|=3|𝑥1−𝑥2−2𝑥3|=2
|𝑥1−2𝑥2−3𝑥3|=3
|𝑥1+𝑥2−7 𝑥3|=14
→
𝑥12+2𝑥1𝑥2+2𝑥1𝑥3+𝑥2
2+2𝑥2𝑥3+𝑥32 ¿ 4
𝑥12+4 𝑥1𝑥2−2 𝑥1𝑥3+4 𝑥2
2−2𝑥2𝑥3+𝑥32 ¿ 9
𝑥12−2 𝑥1 𝑥2−4 𝑥1 𝑥3+𝑥2
2+4 𝑥2 𝑥3+4 𝑥32 ¿ 4
𝑥12−4 𝑥1𝑥2−6 𝑥1𝑥3+4 𝑥2
2+12𝑥2𝑥3+9 𝑥32 ¿ 9
𝑥12+2 𝑥1𝑥2−14 𝑥1𝑥3+𝑥2
2−14 𝑥2𝑥3+49𝑥32 ¿ 196
Let’s square again:
We obtained 5 linear equations with 6 unknown monomials.
Idea: Let’s multiply again these equations (square and cross)
New equations: : 15 equations
New variables (monomials): : 15 unknowns
14/50
Summary of this approach:
where
15/50
3.2 Hierarchical Embeddings
Primary data:
Level d embedding: ,
Identify:
Then
a homogeneous polynomial of total degree 2d.
16/50
How many monomials?
Real case: number of monomials of degree 2d in n variables:
Complex case: …Number of degree (d,d) in n variables:
Define redundancy at level d:
(R.B. [SampTA2009])
17/50
Real case:
18/50
Real case:
19/50
Real case:
20/50
Complex case:
21/50
Complex case:
22/50
Complex case:
23/50
Fundamental question: How many equations are linearly independent?
Recall: , Note:
and
The matrix is not canonical, and so is * for d>1.However its range is basis independent.
We are going to compute this range in terms of a canonical matrix.
24/50
Theorem The following hold true:1. (as a quadratic form)
2. Rows of are linearly independent iff
where is the mdxmd matrix given by
Real case:Complex case:
25/50
Let denote the Gram matrixAnd for integer p.
Theorem In either real or complex case:
Hence for d=1, the number of independent quadratics is given by:
Theorem For d=2,
Remark Note the k1=k2,l1=l2 submatrix of is 26/50
3.3. Numerical analysisResults for the complex case: random frames
n=3,m=627/50
n=3,m=6 28/50
n=16m=64
29/50
n=32m=128
30/50
n=4m=16
31/50
n=4m=14
Note 6 zero eigenvalues of instead of 5.32/50
n=4m=15
Number of zero eigenvalues = 20 =120-100, as expected.33/50
4. Modified Least Square Error Estimator
• Model: mifxd iii 1 , ,2
“Vanilla” Least-Square-Estimator (LSE):
m
iiixLSE fxdx
1
22,minargˆ
Tm
i
Tiiiii xxXffFdXFtr
, ,
:asly equivalentcriterion theRewrite
1
2
We modify this criterion in two ways:1. Replace X by a rank r positive matrix Y2. Regularize the criterion by adding a norm of Y
34/50
)ˆ(.)ˆ(.ˆ
)()(minargˆ1
2)(,0
QeigvectprincYeigvalprincx
YtrdYFtrYm
iiirYrankY
Use a rank-r factorization of Y to account for constraint:
)1(:,ˆˆ :ionfactorizat SDV
minargˆ
1,1
1
2
UxVUL
LLtrdLFLtrL Tm
iii
TRL rn
35/50
Optimization procedure
LLtrdLFLtrL Tm
iii
TRL rn
1
2minargˆ
Our approach: 1. Start with a large and decrease its value over
time;2. Replace one L in the inner quadratic term by a
previous estimate3. Penalize large successive variations.
36/50
KKtrKLKLtrLLtrdKFLtrKLJ Tt
m
i
Tt
Ttii
Tt
1
2),(
1
,minarg )()1(
ttLLJL t
tLt
Algorithm (Part I)Step I: Initialization
Step II: Iteration
Step III: Factorization
1
00)0(
,for policy adaptationan Choose0,,, Initialize
ttt
tL
)1(:,ˆ 1,1
)(
UxVULSVD
How to initialize?How to adapt?
Convergence?
37/50
Initialization
m
iii
m
ii
Tm
i
Tii
T
FdQ
dLQLtrLLtrdLFLtrLJ
L
1
1
2
1
2
:where
22)(
: small and largeFor
21 , : of eigenpairs be ,Let eeveQvQve kkkkk
rr vvvL ||| 2211)0(
Set:
m
i
r
k ik
krfv
rree1 1
20
10,2
,/... 38/50
Convergence
KKtrKLKLtrLLtrdKFLtrKLJ Tt
m
i
Tt
Ttii
Tt
1
2),(
Consider the iterative process:
)()1(
)()1(
,:
,minarg:tt
tt
ttL
t
LLJj
LLJL
Theorem Assume (t)t,(µt)t are monotonically decreasing non-negative sequences. Then (jt)t≥0 is a monotonically decreasing convergent sequence.
LKJKLJ tt ,, :Note
39/50
5. Theoretical bounds: CRLB
m
iii
iiii
mmdfxxdpoodLoglikelih
diiNmifxdModel
1
22
2
22
)log()2log(2
,2
1)|(log :
...),,0(~ ; 1 , , :
jk
jk xxxdpEI )|(log: :Matrixn InformatioFisher
2
,
m
i
Tiii fffxRRI
1
2
2 , where, 4
12
1
4CO : RItimatorUnbiasedEsVARCRLB
𝐸 [‖𝑥− �̂�‖2 ]≥ 𝜎2
4𝑡𝑟 (𝑅−1 )
𝐸 [‖𝑥 𝑥𝑇− �̂� �̂�𝑇‖2 ]≥ 𝜎2
2[‖𝑥‖2 𝑡𝑟 (𝑅−1 )+𝑥𝑇 𝑅− 1𝑥 ] 40/50
The LSE/MLE is a biased estimator. Modified CRLB for biased estimators:
xxBiasxEx
LBModifiedCRBiasBiasx
Ix
xxxxE TT
T
)( , ˆ)(
:ˆˆ 1
Asymptotically (for high SNR):
,
,41 ,
2
1
112
Idx
fRfRffxBiasm
iii
Tii
).(..44
6114
12
10
tohRRRMSE
MSE
T
MSE
41/50
6. Performance Analysis
n=3 , m=9 , d = 1 (dlevel) , r = 2 and , decrease by 5% every step w/ saturation (subspace)
Mintrace algorithm: Candes, Strohmer, Voroninski (2011)
42/50
n=3 , m=9, d=1, r=243/50
n=3 , m=9, d=1, r=244/50
n=8 , m=24, d=3, r=245/50
n=8 , m=24, d=3, r=246/50
n=8 , m=24, d=3, r=247/50
n=9 , m=27, d=3, r=248/50
n=9 , m=27, d=3, r=249/50
n=9 , m=27, d=3, r=250/50