Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not...
Transcript of Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not...
![Page 1: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/1.jpg)
Context Over. Orthog. Invertible Under.
Tensor problems in Engineering
Pierre Comon
July 18, 2008
Pierre Comon Tensor problems in EE 1
![Page 2: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/2.jpg)
Context Over. Orthog. Invertible Under.
Contents I
1 ContextStatistical frameworkCharacteristic functionsIdentifiability & UniquenessCumulants
2 Overdetermined mixturesGeneralIndependenceStandardization
3 Algorithms for orthogonal decompositionCriteriaPair sweepingOther
4 Algorithms for invertible decompositionProbabilistic approach
Pierre Comon Tensor problems in EE 2
![Page 3: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/3.jpg)
Context Over. Orthog. Invertible Under.
Contents II
Alternate Least SquaresDiagonally dominantJoint Schur
5 Underdetermined mixturesBinaryRanksBiomeFoobiCAFIterative algorithmsEnding
Pierre Comon Tensor problems in EE 3
![Page 4: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/4.jpg)
Context Over. Orthog. Invertible Under.
Lecture 1/3
Pierre Comon Tensor problems in EE 4
![Page 5: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/5.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Linear statistical model
y = As + b (1)
with
y : K × 1 randoms : P × 1 random with stat. independent entriesA : K × P deterministicb : errors (may be removed for P large enough)
Pierre Comon Tensor problems in EE 5
![Page 6: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/6.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Taxinomy
K ≥ P: “over-determined”
can be reduced to a square P × P regular mixtureA orthogonal or unitaryA square invertible
K < P: “under-determined”
A rectangular with pairwise lin. independent columns
Pierre Comon Tensor problems in EE 6
![Page 7: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/7.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Taxinomy
K ≥ P: “over-determined”
can be reduced to a square P × P regular mixture
A orthogonal or unitaryA square invertible
K < P: “under-determined”
A rectangular with pairwise lin. independent columns
Pierre Comon Tensor problems in EE 6
![Page 8: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/8.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Taxinomy
K ≥ P: “over-determined”
can be reduced to a square P × P regular mixtureA orthogonal or unitary
A square invertible
K < P: “under-determined”
A rectangular with pairwise lin. independent columns
Pierre Comon Tensor problems in EE 6
![Page 9: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/9.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Taxinomy
K ≥ P: “over-determined”
can be reduced to a square P × P regular mixtureA orthogonal or unitaryA square invertible
K < P: “under-determined”
A rectangular with pairwise lin. independent columns
Pierre Comon Tensor problems in EE 6
![Page 10: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/10.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Taxinomy
K ≥ P: “over-determined”
can be reduced to a square P × P regular mixtureA orthogonal or unitaryA square invertible
K < P: “under-determined”
A rectangular with pairwise lin. independent columns
Pierre Comon Tensor problems in EE 6
![Page 11: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/11.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Taxinomy
K ≥ P: “over-determined”
can be reduced to a square P × P regular mixtureA orthogonal or unitaryA square invertible
K < P: “under-determined”
A rectangular with pairwise lin. independent columns
Pierre Comon Tensor problems in EE 6
![Page 12: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/12.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Goals
In the stochastic framework: solely from realizations ofobservation vector y,
Estimate matrix A: Blind identification
Estimate realizations of the “source” vector s: Blindseparation/extraction/equalization
In the deterministic framework:
Decompose the data tensor (cf. subsequent course)
Pierre Comon Tensor problems in EE 7
![Page 13: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/13.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Goals
In the stochastic framework: solely from realizations ofobservation vector y,
Estimate matrix A: Blind identification
Estimate realizations of the “source” vector s: Blindseparation/extraction/equalization
In the deterministic framework:
Decompose the data tensor (cf. subsequent course)
Pierre Comon Tensor problems in EE 7
![Page 14: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/14.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Goals
In the stochastic framework: solely from realizations ofobservation vector y,
Estimate matrix A: Blind identification
Estimate realizations of the “source” vector s: Blindseparation/extraction/equalization
In the deterministic framework:
Decompose the data tensor (cf. subsequent course)
Pierre Comon Tensor problems in EE 7
![Page 15: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/15.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Goals
In the stochastic framework: solely from realizations ofobservation vector y,
Estimate matrix A: Blind identification
Estimate realizations of the “source” vector s: Blindseparation/extraction/equalization
In the deterministic framework:
Decompose the data tensor (cf. subsequent course)
Pierre Comon Tensor problems in EE 7
![Page 16: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/16.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Goals
In the stochastic framework: solely from realizations ofobservation vector y,
Estimate matrix A: Blind identification
Estimate realizations of the “source” vector s: Blindseparation/extraction/equalization
In the deterministic framework:
Decompose the data tensor (cf. subsequent course)
Pierre Comon Tensor problems in EE 7
![Page 17: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/17.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Application areas for symmetric tensors
1 Telecommunications (Cellular, Satellite, Military),
2 Radar, Sonar,
3 Biomedical (EchoGraphy, ElectroEncephaloGraphy,ElectroCardioGraphy)...
4 Speech,
5 Machine Learning,
6 Control...
Pierre Comon Tensor problems in EE 8
![Page 18: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/18.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Application areas for symmetric tensors
1 Telecommunications (Cellular, Satellite, Military),
2 Radar, Sonar,
3 Biomedical (EchoGraphy, ElectroEncephaloGraphy,ElectroCardioGraphy)...
4 Speech,
5 Machine Learning,
6 Control...
Pierre Comon Tensor problems in EE 8
![Page 19: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/19.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Application areas for symmetric tensors
1 Telecommunications (Cellular, Satellite, Military),
2 Radar, Sonar,
3 Biomedical (EchoGraphy, ElectroEncephaloGraphy,ElectroCardioGraphy)...
4 Speech,
5 Machine Learning,
6 Control...
Pierre Comon Tensor problems in EE 8
![Page 20: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/20.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Application areas for symmetric tensors
1 Telecommunications (Cellular, Satellite, Military),
2 Radar, Sonar,
3 Biomedical (EchoGraphy, ElectroEncephaloGraphy,ElectroCardioGraphy)...
4 Speech,
5 Machine Learning,
6 Control...
Pierre Comon Tensor problems in EE 8
![Page 21: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/21.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Application areas for symmetric tensors
1 Telecommunications (Cellular, Satellite, Military),
2 Radar, Sonar,
3 Biomedical (EchoGraphy, ElectroEncephaloGraphy,ElectroCardioGraphy)...
4 Speech,
5 Machine Learning,
6 Control...
Pierre Comon Tensor problems in EE 8
![Page 22: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/22.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Application areas for symmetric tensors
1 Telecommunications (Cellular, Satellite, Military),
2 Radar, Sonar,
3 Biomedical (EchoGraphy, ElectroEncephaloGraphy,ElectroCardioGraphy)...
4 Speech,
5 Machine Learning,
6 Control...
Pierre Comon Tensor problems in EE 8
![Page 23: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/23.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Example: Antenna Array Processing (1)
Transmitter
Receiver array
Pierre Comon Tensor problems in EE 9
![Page 24: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/24.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Example: Antenna Array Processing (1)
Transmitter
Receiver array
Pierre Comon Tensor problems in EE 9
![Page 25: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/25.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Example: Antenna Array Processing (2)
Modeling the signals received on an array of antennasgenerally leads to a matrix decomposition:
Tij
p
=∑q
∑`
aiq`
∑k
hq`k
p
skqj
i : space k: symbol # a: receiver geometryj : time q: user # h: global channel impulse response
`: path # s: Transmitted signal
But in the presence of additional diversity, a tensor can beconstructed, thanks to new variable p
Pierre Comon Tensor problems in EE 10
![Page 26: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/26.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Example: Antenna Array Processing (2)
Modeling the signals received on an array of antennasgenerally leads to a matrix decomposition:
Tijp =∑q
∑`
aiq`
∑k
hq`kpskqj
i : space k: symbol # a: receiver geometryj : time q: user # h: global channel impulse response
`: path # s: Transmitted signal
But in the presence of additional diversity, a tensor can beconstructed, thanks to new variable p
Pierre Comon Tensor problems in EE 10
![Page 27: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/27.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Example: Antenna Array Processing (3)
New variable p can represent:
Oversampling (sample index),
Spreading code (chip index),
Frequency (multicarrier),
Geometrical invariance (subarray index),
Polarization...
Warning: tensor should not have proportional matrix slices(degeneration)
Pierre Comon Tensor problems in EE 11
![Page 28: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/28.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Example: Antenna Array Processing (3)
New variable p can represent:
Oversampling (sample index),
Spreading code (chip index),
Frequency (multicarrier),
Geometrical invariance (subarray index),
Polarization...
Warning: tensor should not have proportional matrix slices(degeneration)
Pierre Comon Tensor problems in EE 11
![Page 29: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/29.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Link with tensors
In the stochastic framework:
use of the characteristic function
use of cumulants
The obtained tensor enjoys symmetry properties
→ another motivation to study symmetric tensors
Pierre Comon Tensor problems in EE 12
![Page 30: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/30.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Link with tensors
In the stochastic framework:
use of the characteristic function
use of cumulants
The obtained tensor enjoys symmetry properties→ another motivation to study symmetric tensors
Pierre Comon Tensor problems in EE 12
![Page 31: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/31.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Characteristic functions
First c.f.
Real Scalar: Φx(t)def= E{e tx} =
∫u e tu dFx(u)
Real Multivariate: Φx(t)def= E{e tTx} =
∫u e tTu dFx(u)
Second c.f.
Ψ(t)def= log Φ(t)
Properties:
Always exists in the neighborhood of 0Uniquely defined as long as Φ(t) 6= 0
Pierre Comon Tensor problems in EE 13
![Page 32: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/32.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Characteristic functions
First c.f.
Real Scalar: Φx(t)def= E{e tx} =
∫u e tu dFx(u)
Real Multivariate: Φx(t)def= E{e tTx} =
∫u e tTu dFx(u)
Second c.f.
Ψ(t)def= log Φ(t)
Properties:
Always exists in the neighborhood of 0Uniquely defined as long as Φ(t) 6= 0
Pierre Comon Tensor problems in EE 13
![Page 33: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/33.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Characteristic functions
First c.f.
Real Scalar: Φx(t)def= E{e tx} =
∫u e tu dFx(u)
Real Multivariate: Φx(t)def= E{e tTx} =
∫u e tTu dFx(u)
Second c.f.
Ψ(t)def= log Φ(t)
Properties:
Always exists in the neighborhood of 0Uniquely defined as long as Φ(t) 6= 0
Pierre Comon Tensor problems in EE 13
![Page 34: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/34.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Characteristic functions
First c.f.
Real Scalar: Φx(t)def= E{e tx} =
∫u e tu dFx(u)
Real Multivariate: Φx(t)def= E{e tTx} =
∫u e tTu dFx(u)
Second c.f.
Ψ(t)def= log Φ(t)
Properties:
Always exists in the neighborhood of 0Uniquely defined as long as Φ(t) 6= 0
Pierre Comon Tensor problems in EE 13
![Page 35: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/35.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Characteristic functions
First c.f.
Real Scalar: Φx(t)def= E{e tx} =
∫u e tu dFx(u)
Real Multivariate: Φx(t)def= E{e tTx} =
∫u e tTu dFx(u)
Second c.f.
Ψ(t)def= log Φ(t)
Properties:
Always exists in the neighborhood of 0Uniquely defined as long as Φ(t) 6= 0
Pierre Comon Tensor problems in EE 13
![Page 36: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/36.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Characteristic functions
First c.f.
Real Scalar: Φx(t)def= E{e tx} =
∫u e tu dFx(u)
Real Multivariate: Φx(t)def= E{e tTx} =
∫u e tTu dFx(u)
Second c.f.
Ψ(t)def= log Φ(t)
Properties:
Always exists in the neighborhood of 0Uniquely defined as long as Φ(t) 6= 0
Pierre Comon Tensor problems in EE 13
![Page 37: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/37.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Characteristic functions
First c.f.
Real Scalar: Φx(t)def= E{e tx} =
∫u e tu dFx(u)
Real Multivariate: Φx(t)def= E{e tTx} =
∫u e tTu dFx(u)
Second c.f.
Ψ(t)def= log Φ(t)
Properties:
Always exists in the neighborhood of 0
Uniquely defined as long as Φ(t) 6= 0
Pierre Comon Tensor problems in EE 13
![Page 38: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/38.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Characteristic functions
First c.f.
Real Scalar: Φx(t)def= E{e tx} =
∫u e tu dFx(u)
Real Multivariate: Φx(t)def= E{e tTx} =
∫u e tTu dFx(u)
Second c.f.
Ψ(t)def= log Φ(t)
Properties:
Always exists in the neighborhood of 0Uniquely defined as long as Φ(t) 6= 0
Pierre Comon Tensor problems in EE 13
![Page 39: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/39.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Characteristic functions (cont’d)
Properties of the 2nd Characteristic function (cont’d):
if a c.f. Ψx(t) is a polynomial, then its degree is at most 2 andx is Gaussian (Marcinkiewicz’1938) [Luka70]if (x , y) statistically independent, then
Ψx,y (u, v) = Ψx(u) + Ψy (v) (2)
Proof.
Ψx ,y (u, v) = log[E{exp(ux + vy)}]= log[E{exp(ux)}E{exp(vy)}].
Pierre Comon Tensor problems in EE 14
![Page 40: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/40.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Characteristic functions (cont’d)
Properties of the 2nd Characteristic function (cont’d):
if a c.f. Ψx(t) is a polynomial, then its degree is at most 2 andx is Gaussian (Marcinkiewicz’1938) [Luka70]if (x , y) statistically independent, then
Ψx,y (u, v) = Ψx(u) + Ψy (v) (2)
Proof.
Ψx ,y (u, v) = log[E{exp(ux + vy)}]= log[E{exp(ux)}E{exp(vy)}].
Pierre Comon Tensor problems in EE 14
![Page 41: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/41.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Characteristic functions (cont’d)
Properties of the 2nd Characteristic function (cont’d):
if a c.f. Ψx(t) is a polynomial, then its degree is at most 2 andx is Gaussian (Marcinkiewicz’1938) [Luka70]
if (x , y) statistically independent, then
Ψx,y (u, v) = Ψx(u) + Ψy (v) (2)
Proof.
Ψx ,y (u, v) = log[E{exp(ux + vy)}]= log[E{exp(ux)}E{exp(vy)}].
Pierre Comon Tensor problems in EE 14
![Page 42: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/42.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Characteristic functions (cont’d)
Properties of the 2nd Characteristic function (cont’d):
if a c.f. Ψx(t) is a polynomial, then its degree is at most 2 andx is Gaussian (Marcinkiewicz’1938) [Luka70]if (x , y) statistically independent, then
Ψx,y (u, v) = Ψx(u) + Ψy (v) (2)
Proof.
Ψx ,y (u, v) = log[E{exp(ux + vy)}]= log[E{exp(ux)}E{exp(vy)}].
Pierre Comon Tensor problems in EE 14
![Page 43: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/43.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Characteristic functions (cont’d)
Properties of the 2nd Characteristic function (cont’d):
if a c.f. Ψx(t) is a polynomial, then its degree is at most 2 andx is Gaussian (Marcinkiewicz’1938) [Luka70]if (x , y) statistically independent, then
Ψx,y (u, v) = Ψx(u) + Ψy (v) (2)
Proof.
Ψx ,y (u, v) = log[E{exp(ux + vy)}]= log[E{exp(ux)}E{exp(vy)}].
Pierre Comon Tensor problems in EE 14
![Page 44: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/44.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Problem posed in terms of Characteristic Functions
If sp independent and x = As, we have the core equation:
Ψx(u) =∑p
Ψsp
(∑q
uq Aqp
)(3)
Proof.
Plug x = As, in definition of Ψx and get
Φx(u)def= E{exp(uTAs)} = E{exp(
∑p,q
uq Aqp sp)}
Since sp independent, Φx(u) =∏
p E{exp(∑
q uq Aqp sp)}Taking the log concludes.
Problem: Decompose a mutlivariate function into a sum ofunivariate ones
Pierre Comon Tensor problems in EE 15
![Page 45: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/45.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Problem posed in terms of Characteristic Functions
If sp independent and x = As, we have the core equation:
Ψx(u) =∑p
Ψsp
(∑q
uq Aqp
)(3)
Proof.
Plug x = As, in definition of Ψx and get
Φx(u)def= E{exp(uTAs)} = E{exp(
∑p,q
uq Aqp sp)}
Since sp independent, Φx(u) =∏
p E{exp(∑
q uq Aqp sp)}Taking the log concludes.
Problem: Decompose a mutlivariate function into a sum ofunivariate ones
Pierre Comon Tensor problems in EE 15
![Page 46: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/46.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Problem posed in terms of Characteristic Functions
If sp independent and x = As, we have the core equation:
Ψx(u) =∑p
Ψsp
(∑q
uq Aqp
)(3)
Proof.
Plug x = As, in definition of Ψx and get
Φx(u)def= E{exp(uTAs)} = E{exp(
∑p,q
uq Aqp sp)}
Since sp independent, Φx(u) =∏
p E{exp(∑
q uq Aqp sp)}
Taking the log concludes.
Problem: Decompose a mutlivariate function into a sum ofunivariate ones
Pierre Comon Tensor problems in EE 15
![Page 47: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/47.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Problem posed in terms of Characteristic Functions
If sp independent and x = As, we have the core equation:
Ψx(u) =∑p
Ψsp
(∑q
uq Aqp
)(3)
Proof.
Plug x = As, in definition of Ψx and get
Φx(u)def= E{exp(uTAs)} = E{exp(
∑p,q
uq Aqp sp)}
Since sp independent, Φx(u) =∏
p E{exp(∑
q uq Aqp sp)}Taking the log concludes.
Problem: Decompose a mutlivariate function into a sum ofunivariate ones
Pierre Comon Tensor problems in EE 15
![Page 48: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/48.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Problem posed in terms of Characteristic Functions
If sp independent and x = As, we have the core equation:
Ψx(u) =∑p
Ψsp
(∑q
uq Aqp
)(3)
Proof.
Plug x = As, in definition of Ψx and get
Φx(u)def= E{exp(uTAs)} = E{exp(
∑p,q
uq Aqp sp)}
Since sp independent, Φx(u) =∏
p E{exp(∑
q uq Aqp sp)}Taking the log concludes.
Problem: Decompose a mutlivariate function into a sum ofunivariate ones
Pierre Comon Tensor problems in EE 15
![Page 49: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/49.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Darmois-Skitovich theorem (1953)
TheoremLet si be statistically independent random variables, and two linearstatistics:
y1 =∑
i
ai si and y2 =∑
i
bi si
If y1 and y2 are statistically independent, then random variables skfor which ak bk 6= 0 are Gaussian.
NB: holds in both R or C
Pierre Comon Tensor problems in EE 16
![Page 50: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/50.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Sketch of proof
Let charatecteristic functions
Ψ1,2(u, v) = log E{exp( y1 u + y2 v)}Ψk(w) = log E{exp( yk w)}ϕp(w) = log E{exp( sp w)}
1 Independence between sp’s implies:
Ψ1,2(u, v) =∑P
k=1 ϕk(u ak + v bk)
Ψ1(u) =∑P
k=1 ϕk(u ak)
Ψ2(v) =∑P
k=1 ϕk(v bk)
2 Independence between y1 and y2 implies
Ψ1,2(u, v) = Ψ1(u) + Ψ2(v)
Pierre Comon Tensor problems in EE 17
![Page 51: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/51.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Sketch of proof
Let charatecteristic functions
Ψ1,2(u, v) = log E{exp( y1 u + y2 v)}Ψk(w) = log E{exp( yk w)}ϕp(w) = log E{exp( sp w)}
1 Independence between sp’s implies:
Ψ1,2(u, v) =∑P
k=1 ϕk(u ak + v bk)
Ψ1(u) =∑P
k=1 ϕk(u ak)
Ψ2(v) =∑P
k=1 ϕk(v bk)
2 Independence between y1 and y2 implies
Ψ1,2(u, v) = Ψ1(u) + Ψ2(v)
Pierre Comon Tensor problems in EE 17
![Page 52: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/52.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Sketch of proof
Let charatecteristic functions
Ψ1,2(u, v) = log E{exp( y1 u + y2 v)}Ψk(w) = log E{exp( yk w)}ϕp(w) = log E{exp( sp w)}
1 Independence between sp’s implies:
Ψ1,2(u, v) =∑P
k=1 ϕk(u ak + v bk)
Ψ1(u) =∑P
k=1 ϕk(u ak)
Ψ2(v) =∑P
k=1 ϕk(v bk)
2 Independence between y1 and y2 implies
Ψ1,2(u, v) = Ψ1(u) + Ψ2(v)
Pierre Comon Tensor problems in EE 17
![Page 53: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/53.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Does not restrict generality to assume that [ak , bk ] notcollinear. To simplify, assume also ϕp differentiable.
3 Hence∑P
k=1 ϕp(u ak + v bk) =∑P
k=1 ϕk(u ak) + ϕk(v bk)Trivial for terms for which akbk = 0.From now on, restrict the sum to terms akbk 6= 0
4 Write this at u + α/aP and v − α/bP :
P∑k=1
ϕk
(u ak + v bk + α(
ak
aP− bk
bP)
)= f (u) + g(v)
5 Subtract to cancel Pth term, divide by α, and let α→ 0:
P−1∑k=1
(ak
aP− bk
bP)ϕ
(1)k (u ak + v bk) = f (1)(u) + g (1)(v)
for some univariate functions f (1)(u) and g (1)(u).
Conclusion: We have one term less
Pierre Comon Tensor problems in EE 18
![Page 54: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/54.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Does not restrict generality to assume that [ak , bk ] notcollinear. To simplify, assume also ϕp differentiable.
3 Hence∑P
k=1 ϕp(u ak + v bk) =∑P
k=1 ϕk(u ak) + ϕk(v bk)Trivial for terms for which akbk = 0.From now on, restrict the sum to terms akbk 6= 0
4 Write this at u + α/aP and v − α/bP :
P∑k=1
ϕk
(u ak + v bk + α(
ak
aP− bk
bP)
)= f (u) + g(v)
5 Subtract to cancel Pth term, divide by α, and let α→ 0:
P−1∑k=1
(ak
aP− bk
bP)ϕ
(1)k (u ak + v bk) = f (1)(u) + g (1)(v)
for some univariate functions f (1)(u) and g (1)(u).
Conclusion: We have one term less
Pierre Comon Tensor problems in EE 18
![Page 55: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/55.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Does not restrict generality to assume that [ak , bk ] notcollinear. To simplify, assume also ϕp differentiable.
3 Hence∑P
k=1 ϕp(u ak + v bk) =∑P
k=1 ϕk(u ak) + ϕk(v bk)Trivial for terms for which akbk = 0.From now on, restrict the sum to terms akbk 6= 0
4 Write this at u + α/aP and v − α/bP :
P∑k=1
ϕk
(u ak + v bk + α(
ak
aP− bk
bP)
)= f (u) + g(v)
5 Subtract to cancel Pth term, divide by α, and let α→ 0:
P−1∑k=1
(ak
aP− bk
bP)ϕ
(1)k (u ak + v bk) = f (1)(u) + g (1)(v)
for some univariate functions f (1)(u) and g (1)(u).
Conclusion: We have one term less
Pierre Comon Tensor problems in EE 18
![Page 56: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/56.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Does not restrict generality to assume that [ak , bk ] notcollinear. To simplify, assume also ϕp differentiable.
3 Hence∑P
k=1 ϕp(u ak + v bk) =∑P
k=1 ϕk(u ak) + ϕk(v bk)Trivial for terms for which akbk = 0.From now on, restrict the sum to terms akbk 6= 0
4 Write this at u + α/aP and v − α/bP :
P∑k=1
ϕk
(u ak + v bk + α(
ak
aP− bk
bP)
)= f (u) + g(v)
5 Subtract to cancel Pth term, divide by α, and let α→ 0:
P−1∑k=1
(ak
aP− bk
bP)ϕ
(1)k (u ak + v bk) = f (1)(u) + g (1)(v)
for some univariate functions f (1)(u) and g (1)(u).
Conclusion: We have one term less
Pierre Comon Tensor problems in EE 18
![Page 57: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/57.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Does not restrict generality to assume that [ak , bk ] notcollinear. To simplify, assume also ϕp differentiable.
3 Hence∑P
k=1 ϕp(u ak + v bk) =∑P
k=1 ϕk(u ak) + ϕk(v bk)Trivial for terms for which akbk = 0.From now on, restrict the sum to terms akbk 6= 0
4 Write this at u + α/aP and v − α/bP :
P∑k=1
ϕk
(u ak + v bk + α(
ak
aP− bk
bP)
)= f (u) + g(v)
5 Subtract to cancel Pth term, divide by α, and let α→ 0:
P−1∑k=1
(ak
aP− bk
bP)ϕ
(1)k (u ak + v bk) = f (1)(u) + g (1)(v)
for some univariate functions f (1)(u) and g (1)(u).
Conclusion: We have one term less
Pierre Comon Tensor problems in EE 18
![Page 58: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/58.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
6 Repeat the procedure (P − 1) times and get:
P∏j=2
(a1
aj− b1
bj) ϕ
(P−1)1 (u a1 + v b1) = f (P−1)(u) + g (P−1)(v)
7 Hence ϕ(P−1)1 (u a1 + v b1) is linear, as a sum of two univariate
functions (ϕ(P)1 is a constant because a1b1 6= 0).
8 Eventually ϕ1 is a polynomial.
9 Lastly invoke Marcinkiewicz theorem to conclude that s1 isGaussian.
10 Same is true for any ϕp such that apbp 6= 0: sp is Gaussian.
NB: also holds if ϕp not differentiable
Pierre Comon Tensor problems in EE 19
![Page 59: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/59.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
6 Repeat the procedure (P − 1) times and get:
P∏j=2
(a1
aj− b1
bj) ϕ
(P−1)1 (u a1 + v b1) = f (P−1)(u) + g (P−1)(v)
7 Hence ϕ(P−1)1 (u a1 + v b1) is linear, as a sum of two univariate
functions (ϕ(P)1 is a constant because a1b1 6= 0).
8 Eventually ϕ1 is a polynomial.
9 Lastly invoke Marcinkiewicz theorem to conclude that s1 isGaussian.
10 Same is true for any ϕp such that apbp 6= 0: sp is Gaussian.
NB: also holds if ϕp not differentiable
Pierre Comon Tensor problems in EE 19
![Page 60: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/60.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
6 Repeat the procedure (P − 1) times and get:
P∏j=2
(a1
aj− b1
bj) ϕ
(P−1)1 (u a1 + v b1) = f (P−1)(u) + g (P−1)(v)
7 Hence ϕ(P−1)1 (u a1 + v b1) is linear, as a sum of two univariate
functions (ϕ(P)1 is a constant because a1b1 6= 0).
8 Eventually ϕ1 is a polynomial.
9 Lastly invoke Marcinkiewicz theorem to conclude that s1 isGaussian.
10 Same is true for any ϕp such that apbp 6= 0: sp is Gaussian.
NB: also holds if ϕp not differentiable
Pierre Comon Tensor problems in EE 19
![Page 61: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/61.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
6 Repeat the procedure (P − 1) times and get:
P∏j=2
(a1
aj− b1
bj) ϕ
(P−1)1 (u a1 + v b1) = f (P−1)(u) + g (P−1)(v)
7 Hence ϕ(P−1)1 (u a1 + v b1) is linear, as a sum of two univariate
functions (ϕ(P)1 is a constant because a1b1 6= 0).
8 Eventually ϕ1 is a polynomial.
9 Lastly invoke Marcinkiewicz theorem to conclude that s1 isGaussian.
10 Same is true for any ϕp such that apbp 6= 0: sp is Gaussian.
NB: also holds if ϕp not differentiable
Pierre Comon Tensor problems in EE 19
![Page 62: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/62.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
6 Repeat the procedure (P − 1) times and get:
P∏j=2
(a1
aj− b1
bj) ϕ
(P−1)1 (u a1 + v b1) = f (P−1)(u) + g (P−1)(v)
7 Hence ϕ(P−1)1 (u a1 + v b1) is linear, as a sum of two univariate
functions (ϕ(P)1 is a constant because a1b1 6= 0).
8 Eventually ϕ1 is a polynomial.
9 Lastly invoke Marcinkiewicz theorem to conclude that s1 isGaussian.
10 Same is true for any ϕp such that apbp 6= 0: sp is Gaussian.
NB: also holds if ϕp not differentiable
Pierre Comon Tensor problems in EE 19
![Page 63: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/63.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
6 Repeat the procedure (P − 1) times and get:
P∏j=2
(a1
aj− b1
bj) ϕ
(P−1)1 (u a1 + v b1) = f (P−1)(u) + g (P−1)(v)
7 Hence ϕ(P−1)1 (u a1 + v b1) is linear, as a sum of two univariate
functions (ϕ(P)1 is a constant because a1b1 6= 0).
8 Eventually ϕ1 is a polynomial.
9 Lastly invoke Marcinkiewicz theorem to conclude that s1 isGaussian.
10 Same is true for any ϕp such that apbp 6= 0: sp is Gaussian.
NB: also holds if ϕp not differentiable
Pierre Comon Tensor problems in EE 19
![Page 64: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/64.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Equivalent representations
Let y admit two representations
y = As and y = Bz
where s (resp. z) have independent components, and A (resp. B)have pairwise noncollinear columns.
These representations are equivalent if every column of A isproportional to some column of B, and vice versa.
If all representations of y are equivalent, they are said to beessentially unique (permutation & scale ambiguities only).
Pierre Comon Tensor problems in EE 20
![Page 65: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/65.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Equivalent representations
Let y admit two representations
y = As and y = Bz
where s (resp. z) have independent components, and A (resp. B)have pairwise noncollinear columns.
These representations are equivalent if every column of A isproportional to some column of B, and vice versa.
If all representations of y are equivalent, they are said to beessentially unique (permutation & scale ambiguities only).
Pierre Comon Tensor problems in EE 20
![Page 66: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/66.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Equivalent representations
Let y admit two representations
y = As and y = Bz
where s (resp. z) have independent components, and A (resp. B)have pairwise noncollinear columns.
These representations are equivalent if every column of A isproportional to some column of B, and vice versa.
If all representations of y are equivalent, they are said to beessentially unique (permutation & scale ambiguities only).
Pierre Comon Tensor problems in EE 20
![Page 67: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/67.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Identifiability & uniqueness theorems s
Let y be a random vector of the form y = As, where sp areindependent, and A has non pairwise collinear columns.
Identifiability theorem y can be represented asy = A1 s1 + A2 s2, where s1 is non Gaussian, s2 is Gaussianindependent of s1, and A1 is essentially unique.
Uniqueness theorem If in addition the columns of A1 arelinearly independent, then the distribution of s1 is unique upto scale and location indeterminacies.
Remark 1: if s2 is 1-dimensional, then A2 is also essentially uniqueRemark 2: the proofs are not constructive [KagaLR73]
Pierre Comon Tensor problems in EE 21
![Page 68: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/68.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Example of uniqueness s
Let si be independent with no Gaussian component, and bi beindependent Gaussian. Then the linear model below is identifiable,but A2 is not essentially unique whereas A1 is:(
s1 + s2 + 2 b1
s1 + 2 b2
)= A1 s+A2
(b1
b2
)= A1 s+A3
(b1 + b2
b1 − b2
)with
A1 =
(1 11 0
), A2 =
(2 00 2
)and A3 =
(1 11 −1
)Hence the distribution of s is essentially unique.But (A1, A2) not equivalent to (A1, A3).
Pierre Comon Tensor problems in EE 22
![Page 69: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/69.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Example of non uniqueness s
Let si be independent with no Gaussian component, and bi beindependent Gaussian. Then the linear model below is identifiable,but the distribution of s is not unique because a 2× 4 matrixcannot be full column rank:
(s1 + s3 + s4 + 2 b1
s2 + s3 − s4 + 2 b2
)= A
s1s2
s3 + b1 + b2
s4 + b1 − b2
= A
s1 + 2 b1
s2 + 2 b2
s3s4
with
A =
(1 0 1 10 1 1 −1
)
Pierre Comon Tensor problems in EE 23
![Page 70: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/70.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Definition of Cumulants
Moments:
µrdef= E{x r} = (−)r ∂rΦ(t)
∂tr
∣∣∣∣t=0
Cumulants:
Cx (r)
def= Cum{x , . . . , x︸ ︷︷ ︸
r times
} = (−)r ∂rΨ(t)
∂tr
∣∣∣∣t=0
Relationship between Moments and Cumulants obtained byexpanding both sides in Taylor series:
log Φx(t) = Ψx(t)
Needs existence. Counter example: Cauchy
px(u) =1
π (1 + u2)
Pierre Comon Tensor problems in EE 24
![Page 71: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/71.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Definition of Cumulants
Moments:
µrdef= E{x r} = (−)r ∂rΦ(t)
∂tr
∣∣∣∣t=0
Cumulants:
Cx (r)
def= Cum{x , . . . , x︸ ︷︷ ︸
r times
} = (−)r ∂rΨ(t)
∂tr
∣∣∣∣t=0
Relationship between Moments and Cumulants obtained byexpanding both sides in Taylor series:
log Φx(t) = Ψx(t)
Needs existence. Counter example: Cauchy
px(u) =1
π (1 + u2)
Pierre Comon Tensor problems in EE 24
![Page 72: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/72.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Definition of Cumulants
Moments:
µrdef= E{x r} = (−)r ∂rΦ(t)
∂tr
∣∣∣∣t=0
Cumulants:
Cx (r)
def= Cum{x , . . . , x︸ ︷︷ ︸
r times
} = (−)r ∂rΨ(t)
∂tr
∣∣∣∣t=0
Relationship between Moments and Cumulants obtained byexpanding both sides in Taylor series:
log Φx(t) = Ψx(t)
Needs existence. Counter example: Cauchy
px(u) =1
π (1 + u2)
Pierre Comon Tensor problems in EE 24
![Page 73: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/73.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Definition of Cumulants
Moments:
µrdef= E{x r} = (−)r ∂rΦ(t)
∂tr
∣∣∣∣t=0
Cumulants:
Cx (r)
def= Cum{x , . . . , x︸ ︷︷ ︸
r times
} = (−)r ∂rΨ(t)
∂tr
∣∣∣∣t=0
Relationship between Moments and Cumulants obtained byexpanding both sides in Taylor series:
log Φx(t) = Ψx(t)
Needs existence. Counter example: Cauchy
px(u) =1
π (1 + u2)
Pierre Comon Tensor problems in EE 24
![Page 74: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/74.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
First Cumulants
C(2) is the variance: C(2) = µ(2) − µ2(1)
For zero-mean r.v.: C(3) = µ(3), and C(4) = µ(4) − 3µ2(2)
Standardized cumulants:
K(r) = Cum(r)
{x − µ′(1)√
µ(2)
}
e.g. Skewness K3, and Kurtosis K4.
Pierre Comon Tensor problems in EE 25
![Page 75: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/75.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
First Cumulants
C(2) is the variance: C(2) = µ(2) − µ2(1)
For zero-mean r.v.: C(3) = µ(3), and C(4) = µ(4) − 3µ2(2)
Standardized cumulants:
K(r) = Cum(r)
{x − µ′(1)√
µ(2)
}
e.g. Skewness K3, and Kurtosis K4.
Pierre Comon Tensor problems in EE 25
![Page 76: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/76.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
First Cumulants
C(2) is the variance: C(2) = µ(2) − µ2(1)
For zero-mean r.v.: C(3) = µ(3), and C(4) = µ(4) − 3µ2(2)
Standardized cumulants:
K(r) = Cum(r)
{x − µ′(1)√
µ(2)
}
e.g. Skewness K3, and Kurtosis K4.
Pierre Comon Tensor problems in EE 25
![Page 77: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/77.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Examples of cumulants (1)
Example: Zero-mean Gaussian
Moments
µ(2r) = µr(2)
(2r)!
r ! 2r
In particular:
µ(4) = 3µ2(2), µ(6) = 15µ3
(2)
C(4) = 0, K(4) = 0.
All Cumulants of order r > 2 are null
Pierre Comon Tensor problems in EE 26
![Page 78: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/78.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Examples of Cumulants (2) s
Example: Uniform
uniformly distributed in [−a,+a] with probability 12a
Moments: µ(2k) = a2k
2k+1
4th order Cumulant: C(4) = a4
5 − 3 a4
9 = −2 a4
15
Kurtosis: K(4) = −65 .
a-
1 / 2a
−a
6
Pierre Comon Tensor problems in EE 27
![Page 79: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/79.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Examples of Cumulants (3) s
Example: Zero-mean standardized binary
x takes two values x1 = −a and x2 = 1/a with probabilities
P1 = 11+a2 , P2 = a2
1+a2
Skewness is K(3) =1
a− a
Kurtosis is K(4) =1
a2+ a2 − 4
Extreme values (bound)
Minimum Kurtosisfor a = 1 (symmetric):K(4) = −2
Pierre Comon Tensor problems in EE 28
![Page 80: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/80.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Multivariate Cumulants
Notation: Cij ..`def= Cum{Xi , Xj , ...X`}
First cumulants:
µ′i = Ciµ′ij = Cij + CiCjµ′ijk = Cijk + [3] CiCjk + CiCjCk
with [n]: Mccullagh’s bracket notation.
Next, for zero-mean variables:
µijk` = Cijk` + [3] CijCk`
µijk`m = Cijk`m + [10] CijCk`m
Again, general formula of Leonov-Shiryayev obtained byTaylor expansion of both sides of Ψ(t) = log Φ(t)...
Pierre Comon Tensor problems in EE 29
![Page 81: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/81.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Multivariate Cumulants
Notation: Cij ..`def= Cum{Xi , Xj , ...X`}
First cumulants:
µ′i = Ciµ′ij = Cij + CiCjµ′ijk = Cijk + [3] CiCjk + CiCjCk
with [n]: Mccullagh’s bracket notation.
Next, for zero-mean variables:
µijk` = Cijk` + [3] CijCk`
µijk`m = Cijk`m + [10] CijCk`m
Again, general formula of Leonov-Shiryayev obtained byTaylor expansion of both sides of Ψ(t) = log Φ(t)...
Pierre Comon Tensor problems in EE 29
![Page 82: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/82.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Multivariate Cumulants
Notation: Cij ..`def= Cum{Xi , Xj , ...X`}
First cumulants:
µ′i = Ciµ′ij = Cij + CiCjµ′ijk = Cijk + [3] CiCjk + CiCjCk
with [n]: Mccullagh’s bracket notation.
Next, for zero-mean variables:
µijk` = Cijk` + [3] CijCk`
µijk`m = Cijk`m + [10] CijCk`m
Again, general formula of Leonov-Shiryayev obtained byTaylor expansion of both sides of Ψ(t) = log Φ(t)...
Pierre Comon Tensor problems in EE 29
![Page 83: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/83.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Multivariate Cumulants
Notation: Cij ..`def= Cum{Xi , Xj , ...X`}
First cumulants:
µ′i = Ciµ′ij = Cij + CiCjµ′ijk = Cijk + [3] CiCjk + CiCjCk
with [n]: Mccullagh’s bracket notation.
Next, for zero-mean variables:
µijk` = Cijk` + [3] CijCk`
µijk`m = Cijk`m + [10] CijCk`m
Again, general formula of Leonov-Shiryayev obtained byTaylor expansion of both sides of Ψ(t) = log Φ(t)...
Pierre Comon Tensor problems in EE 29
![Page 84: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/84.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Properties of Cumulants
Multi-linearity (also enjoyed by moments):
Cum{αX , Y , ..,Z} = α Cum{X , Y , ..,Z} (4)
Cum{X1 + X2, Y , ..,Z} = Cum{X1, Y , ..,Z}+ Cum{X2, Y , ..,Z}
Cancellation: If {Xi} can be partitioned into 2 groups ofindependent r.v., then
Cum{X1, X2, ..,Xr} = 0
Additivity: If X and Y are independent, then
Cum{X1 + Y1, X2 + Y2, ..,Xr + Yr} = Cum{X1, X2, ..,Xr}+ Cum{Y1, Y2, ..,Yr}
Inequalities, e.g.:K2
(3) ≤ K(4) + 2
Pierre Comon Tensor problems in EE 30
![Page 85: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/85.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Properties of Cumulants
Multi-linearity (also enjoyed by moments):
Cum{αX , Y , ..,Z} = α Cum{X , Y , ..,Z} (4)
Cum{X1 + X2, Y , ..,Z} = Cum{X1, Y , ..,Z}+ Cum{X2, Y , ..,Z}
Cancellation: If {Xi} can be partitioned into 2 groups ofindependent r.v., then
Cum{X1, X2, ..,Xr} = 0
Additivity: If X and Y are independent, then
Cum{X1 + Y1, X2 + Y2, ..,Xr + Yr} = Cum{X1, X2, ..,Xr}+ Cum{Y1, Y2, ..,Yr}
Inequalities, e.g.:K2
(3) ≤ K(4) + 2
Pierre Comon Tensor problems in EE 30
![Page 86: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/86.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Properties of Cumulants
Multi-linearity (also enjoyed by moments):
Cum{αX , Y , ..,Z} = α Cum{X , Y , ..,Z} (4)
Cum{X1 + X2, Y , ..,Z} = Cum{X1, Y , ..,Z}+ Cum{X2, Y , ..,Z}
Cancellation: If {Xi} can be partitioned into 2 groups ofindependent r.v., then
Cum{X1, X2, ..,Xr} = 0
Additivity: If X and Y are independent, then
Cum{X1 + Y1, X2 + Y2, ..,Xr + Yr} = Cum{X1, X2, ..,Xr}+ Cum{Y1, Y2, ..,Yr}
Inequalities, e.g.:K2
(3) ≤ K(4) + 2
Pierre Comon Tensor problems in EE 30
![Page 87: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/87.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Properties of Cumulants
Multi-linearity (also enjoyed by moments):
Cum{αX , Y , ..,Z} = α Cum{X , Y , ..,Z} (4)
Cum{X1 + X2, Y , ..,Z} = Cum{X1, Y , ..,Z}+ Cum{X2, Y , ..,Z}
Cancellation: If {Xi} can be partitioned into 2 groups ofindependent r.v., then
Cum{X1, X2, ..,Xr} = 0
Additivity: If X and Y are independent, then
Cum{X1 + Y1, X2 + Y2, ..,Xr + Yr} = Cum{X1, X2, ..,Xr}+ Cum{Y1, Y2, ..,Yr}
Inequalities, e.g.:K2
(3) ≤ K(4) + 2
Pierre Comon Tensor problems in EE 30
![Page 88: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/88.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Problem posed in terms of Cumulants
Input-output relations If y = As, where sp are independent,then multi-linearity of cumulants yields:
Cy,ijk..` =P∑
p=1
Aip Ajp Akp..A`p Cs,ppp..p (5)
Can one identify A form tensor Cy?
Remark
Tensor Cy does not contain all the information whereas the c.f(3) did.
Possibility to choose cumulant order(s)
Pierre Comon Tensor problems in EE 31
![Page 89: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/89.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Problem posed in terms of Cumulants
Input-output relations If y = As, where sp are independent,then multi-linearity of cumulants yields:
Cy,ijk..` =P∑
p=1
Aip Ajp Akp..A`p Cs,ppp..p (5)
Can one identify A form tensor Cy?
Remark
Tensor Cy does not contain all the information whereas the c.f(3) did.
Possibility to choose cumulant order(s)
Pierre Comon Tensor problems in EE 31
![Page 90: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/90.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Problem posed in terms of Cumulants
Input-output relations If y = As, where sp are independent,then multi-linearity of cumulants yields:
Cy,ijk..` =P∑
p=1
Aip Ajp Akp..A`p Cs,ppp..p (5)
Can one identify A form tensor Cy?
Remark
Tensor Cy does not contain all the information whereas the c.f(3) did.
Possibility to choose cumulant order(s)
Pierre Comon Tensor problems in EE 31
![Page 91: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/91.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Problem posed in terms of Cumulants
Input-output relations If y = As, where sp are independent,then multi-linearity of cumulants yields:
Cy,ijk..` =P∑
p=1
Aip Ajp Akp..A`p Cs,ppp..p (5)
Can one identify A form tensor Cy?
Remark
Tensor Cy does not contain all the information whereas the c.f(3) did.
Possibility to choose cumulant order(s)
Pierre Comon Tensor problems in EE 31
![Page 92: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/92.jpg)
Context Over. Orthog. Invertible Under. Stat. c.f. Uniqueness Cumulants
Problem posed in terms of Cumulants
Input-output relations If y = As, where sp are independent,then multi-linearity of cumulants yields:
Cy,ijk..` =P∑
p=1
Aip Ajp Akp..A`p Cs,ppp..p (5)
Can one identify A form tensor Cy?
Remark
Tensor Cy does not contain all the information whereas the c.f(3) did.
Possibility to choose cumulant order(s)
Pierre Comon Tensor problems in EE 31
![Page 93: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/93.jpg)
Context Over. Orthog. Invertible Under. General Independence Stand.
Over-determined mixtures
In that framework, the statistical model involves at most as manysources as the dimension of the observation space:
y = As, with Kdef= dim{y} ≥ dim{s} def
= P
That is, A admits a left inverse.
Warning:
In practice, Cy or Ψy(u) are estimated from noisymeasurements, so that (5) or (3) are never exactly satisfied ifK ≥ P: they become approximations
Pierre Comon Tensor problems in EE 32
![Page 94: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/94.jpg)
Context Over. Orthog. Invertible Under. General Independence Stand.
Over-determined mixtures
In that framework, the statistical model involves at most as manysources as the dimension of the observation space:
y = As, with Kdef= dim{y} ≥ dim{s} def
= P
That is, A admits a left inverse.
Warning:
In practice, Cy or Ψy(u) are estimated from noisymeasurements, so that (5) or (3) are never exactly satisfied ifK ≥ P: they become approximations
Pierre Comon Tensor problems in EE 32
![Page 95: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/95.jpg)
Context Over. Orthog. Invertible Under. General Independence Stand.
Essential uniqueness
Over-determined mixtures are equivalent iff they are relatedby scale-permutation: A = BΛP
Hence in the absence of noise, the source random variablescan be recovered up to scale and permutation as:
z = ΛPs
This is an inherent indeterminacy of the problem.
Pierre Comon Tensor problems in EE 33
![Page 96: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/96.jpg)
Context Over. Orthog. Invertible Under. General Independence Stand.
Essential uniqueness
Over-determined mixtures are equivalent iff they are relatedby scale-permutation: A = BΛP
Hence in the absence of noise, the source random variablescan be recovered up to scale and permutation as:
z = ΛPs
This is an inherent indeterminacy of the problem.
Pierre Comon Tensor problems in EE 33
![Page 97: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/97.jpg)
Context Over. Orthog. Invertible Under. General Independence Stand.
Direct vs Inverse
Two formulations in terms of cumulants:
1 Direct: look for A so as to fit eq. (5):
minA||Cy,ijk..` −
P∑p=1
Aip Ajp Akp..A`p Cs,ppp..p||2
i.e. decompose Cy into a sum of P rank-one terms
2 Inverse: look for B:
minB
∑mnp..q 6=ppp..p
|∑ijk..`
Cy,ijk..`Bim Bjn Bkp..B`q|2
i.e. try to diagonalize Cy by linear transform, z = By
Pierre Comon Tensor problems in EE 34
![Page 98: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/98.jpg)
Context Over. Orthog. Invertible Under. General Independence Stand.
Direct vs Inverse
Two formulations in terms of cumulants:
1 Direct: look for A so as to fit eq. (5):
minA||Cy,ijk..` −
P∑p=1
Aip Ajp Akp..A`p Cs,ppp..p||2
i.e. decompose Cy into a sum of P rank-one terms
2 Inverse: look for B:
minB
∑mnp..q 6=ppp..p
|∑ijk..`
Cy,ijk..`Bim Bjn Bkp..B`q|2
i.e. try to diagonalize Cy by linear transform, z = By
Pierre Comon Tensor problems in EE 34
![Page 99: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/99.jpg)
Context Over. Orthog. Invertible Under. General Independence Stand.
Divide to conquer
Difficulty: many unknowns, in real or complex field
1 1st idea: address a sequence of problems of smallerdimension instead of a single one in larger dimension.
2 2nd idea: decompose A into two factors, A = LQ, andcompute L so as to exactly standardize the data. Look for thebest Q in a second stage.
ý Both are sub-optimal, but of practical value.
Pierre Comon Tensor problems in EE 35
![Page 100: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/100.jpg)
Context Over. Orthog. Invertible Under. General Independence Stand.
Divide to conquer
Difficulty: many unknowns, in real or complex field
1 1st idea: address a sequence of problems of smallerdimension instead of a single one in larger dimension.
2 2nd idea: decompose A into two factors, A = LQ, andcompute L so as to exactly standardize the data. Look for thebest Q in a second stage.
ý Both are sub-optimal, but of practical value.
Pierre Comon Tensor problems in EE 35
![Page 101: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/101.jpg)
Context Over. Orthog. Invertible Under. General Independence Stand.
Divide to conquer
Difficulty: many unknowns, in real or complex field
1 1st idea: address a sequence of problems of smallerdimension instead of a single one in larger dimension.
2 2nd idea: decompose A into two factors, A = LQ, andcompute L so as to exactly standardize the data. Look for thebest Q in a second stage.
ý Both are sub-optimal, but of practical value.
Pierre Comon Tensor problems in EE 35
![Page 102: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/102.jpg)
Context Over. Orthog. Invertible Under. General Independence Stand.
Pairwise vs. Mutual independence (1)
Components sk of s are pairwise independent ⇔ Any pair ofcomponents (sk , s`) are mutually independent.
In general, pairwise and mutual independence are notequivalent
But...
Pierre Comon Tensor problems in EE 36
![Page 103: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/103.jpg)
Context Over. Orthog. Invertible Under. General Independence Stand.
Pairwise vs. Mutual independence (2) s
Example: Pairwise but not Mutual independence
3 mutually independent binary sources, xi ∈ {−1, 1},1 ≤ i ≤ 3
Define x4 = x1x2x3. Then x4 is also binary, dependent on xi
xk are pairwise independent:p(x1 = a, x4 = b) = p(x4 = b | x1 = a).p(x1 = a) =p(x2 x3 = b/a).p(x1 = a)But x1 and x2 x3 are binary ⇒p(x2 x3 = b/a).p(x1 = a) = 1
2 ·12
NB: in particular, Cum{x1, x2, x3, x4} = 1 6= 0
Pierre Comon Tensor problems in EE 37
![Page 104: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/104.jpg)
Context Over. Orthog. Invertible Under. General Independence Stand.
Pairwise vs. Mutual independence (3)
Corollary of Darmois’s theorem for over-determined mixtures:
Let z = Cs, where si are independent r.v., and assume either
all si are non Gaussian and C is invertible, or
at most one si is Gaussian and C is orthogonal/unitary
Then the following properties are equivalent:
1 Components zi are pairwise independent
2 Components zi are mutually independent
3 C = ΛP, with Λ diagonal invertible and P permutation
Proof... 1 → 3 by contradiction
Pierre Comon Tensor problems in EE 38
![Page 105: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/105.jpg)
Context Over. Orthog. Invertible Under. General Independence Stand.
Pairwise vs. Mutual independence (3)
Corollary of Darmois’s theorem for over-determined mixtures:
Let z = Cs, where si are independent r.v., and assume either
all si are non Gaussian and C is invertible, or
at most one si is Gaussian and C is orthogonal/unitary
Then the following properties are equivalent:
1 Components zi are pairwise independent
2 Components zi are mutually independent
3 C = ΛP, with Λ diagonal invertible and P permutation
Proof... 1 → 3 by contradiction
Pierre Comon Tensor problems in EE 38
![Page 106: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/106.jpg)
Context Over. Orthog. Invertible Under. General Independence Stand.
Pairwise vs. Mutual independence (3)
Corollary of Darmois’s theorem for over-determined mixtures:
Let z = Cs, where si are independent r.v., and assume either
all si are non Gaussian and C is invertible, or
at most one si is Gaussian and C is orthogonal/unitary
Then the following properties are equivalent:
1 Components zi are pairwise independent
2 Components zi are mutually independent
3 C = ΛP, with Λ diagonal invertible and P permutation
Proof... 1 → 3 by contradiction
Pierre Comon Tensor problems in EE 38
![Page 107: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/107.jpg)
Context Over. Orthog. Invertible Under. General Independence Stand.
Standardization with covariance
Let x be a zero-mean r.v. with covariance matrix:
Γxdef= E{x xH}
and let L be any square root of Γ:
LLH = Γx
Then xdef= L−1x is a standardized random variable, i.e. it has
unit covariance.
In practice, use SVD to face ill-conditioning or singularity: xmay have then a smaller dimension.
Pierre Comon Tensor problems in EE 39
![Page 108: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/108.jpg)
Context Over. Orthog. Invertible Under. General Independence Stand.
Standardization with covariance
Let x be a zero-mean r.v. with covariance matrix:
Γxdef= E{x xH}
and let L be any square root of Γ:
LLH = Γx
Then xdef= L−1x is a standardized random variable, i.e. it has
unit covariance.
In practice, use SVD to face ill-conditioning or singularity: xmay have then a smaller dimension.
Pierre Comon Tensor problems in EE 39
![Page 109: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/109.jpg)
Context Over. Orthog. Invertible Under. General Independence Stand.
Pre-processing with SVD s
Observed random variable x of dimension K . Then ∃(U, x):
x = U x, U unitary
where “Principal Components” xi are uncorrelated.ith column ui of U is called “ith PC loading vector”
Two –among many– possible calculations from a K × Nrealization matrix X:
EVD of sample covariance Rx : Rx = UΣ2UH, x = UH xSample estimate by SVD: X = UΣVH, X = ΣVH
The latter is more accurate, and avoids the calculation of thecovariance.
Pierre Comon Tensor problems in EE 40
![Page 110: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/110.jpg)
Context Over. Orthog. Invertible Under. General Independence Stand.
Lecture 2/3
Pierre Comon Tensor problems in EE 41
![Page 111: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/111.jpg)
Context Over. Orthog. Invertible Under. General Independence Stand.
Orthogonal decomposition
If Q orthogonal, the two problems are equivalent:
1 Direct:
minQ,Λ||Cijk` −
P∑p=1
Qip Qjp Qkp Q`p Λpppp||2
2 Inverse: minQ,Λ ||∑
ijk` Qip Qjq Qkr Q`s Cijk` − Λpppp δpqrs ||2 or
e.g. maxQ
∑p
|∑ijk`
Qip Qjp Qkp Q`p Cijk`|2
Proof. The Frobenius norm is invariant under orthogonal change ofcoordinates.
Pierre Comon Tensor problems in EE 42
![Page 112: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/112.jpg)
Context Over. Orthog. Invertible Under. General Independence Stand.
Orthogonal decomposition
If Q orthogonal, the two problems are equivalent:
1 Direct:
minQ,Λ||Cijk` −
P∑p=1
Qip Qjp Qkp Q`p Λpppp||2
2 Inverse: minQ,Λ ||∑
ijk` Qip Qjq Qkr Q`s Cijk` − Λpppp δpqrs ||2 or
e.g. maxQ
∑p
|∑ijk`
Qip Qjp Qkp Q`p Cijk`|2
Proof. The Frobenius norm is invariant under orthogonal change ofcoordinates.
Pierre Comon Tensor problems in EE 42
![Page 113: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/113.jpg)
Context Over. Orthog. Invertible Under. General Independence Stand.
Orthogonal decomposition
If Q orthogonal, the two problems are equivalent:
1 Direct:
minQ,Λ||Cijk` −
P∑p=1
Qip Qjp Qkp Q`p Λpppp||2
2 Inverse: minQ,Λ ||∑
ijk` Qip Qjq Qkr Q`s Cijk` − Λpppp δpqrs ||2 or
e.g. maxQ
∑p
|∑ijk`
Qip Qjp Qkp Q`p Cijk`|2
Proof. The Frobenius norm is invariant under orthogonal change ofcoordinates.
Pierre Comon Tensor problems in EE 42
![Page 114: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/114.jpg)
Context Over. Orthog. Invertible Under. General Independence Stand.
Orthogonal decomposition
If Q orthogonal, the two problems are equivalent:
1 Direct:
minQ,Λ||Cijk` −
P∑p=1
Qip Qjp Qkp Q`p Λpppp||2
2 Inverse: minQ,Λ ||∑
ijk` Qip Qjq Qkr Q`s Cijk` − Λpppp δpqrs ||2 or
e.g. maxQ
∑p
|∑ijk`
Qip Qjp Qkp Q`p Cijk`|2
Proof. The Frobenius norm is invariant under orthogonal change ofcoordinates.
Pierre Comon Tensor problems in EE 42
![Page 115: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/115.jpg)
Context Over. Orthog. Invertible Under. General Independence Stand.
What we have seen so far
The joint 2nd characteristic function of x = As contains allthe information. The problem consists of decomposing it intoa sum of univariate characteristic functions.
Cumulant tensors contain part of the information, but maysuffice. The problem reduces to decomposing one of them.
Identification of an invertible mixture generally leads to anapproximation problem
1st idea: in an inverse approach, one can process pairwise
2nd idea: two-stage by decomposing A = LQ. First computeL via an exact standardization of x. Second compute the bestorthogonal matrix Q.
Drawback: one puts an infinite weight on 2nd order statistics
Pierre Comon Tensor problems in EE 43
![Page 116: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/116.jpg)
Context Over. Orthog. Invertible Under. Criteria Pair sweep. Other
Estimation of the orthogonal matrix
Assume we have realizations of the standardized r.v. x = L−1 x.We have to:
1 Choose an optimization criterion to maximize, e.g. based onthe cumulant tensor of z.
2 Devise a numerical algorithm, e.g. proceeding pairwise
Pierre Comon Tensor problems in EE 44
![Page 117: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/117.jpg)
Context Over. Orthog. Invertible Under. Criteria Pair sweep. Other
Estimation of the orthogonal matrix
Assume we have realizations of the standardized r.v. x = L−1 x.We have to:
1 Choose an optimization criterion to maximize, e.g. based onthe cumulant tensor of z.
2 Devise a numerical algorithm, e.g. proceeding pairwise
Pierre Comon Tensor problems in EE 44
![Page 118: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/118.jpg)
Context Over. Orthog. Invertible Under. Criteria Pair sweep. Other
Estimation of the orthogonal matrix
Assume we have realizations of the standardized r.v. x = L−1 x.We have to:
1 Choose an optimization criterion to maximize, e.g. based onthe cumulant tensor of z.
2 Devise a numerical algorithm, e.g. proceeding pairwise
Pierre Comon Tensor problems in EE 44
![Page 119: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/119.jpg)
Context Over. Orthog. Invertible Under. Criteria Pair sweep. Other
Contrast criteria
1 Optimization criteria
Let s have stat. independent components, and z = Qs. A good“contrast” criterion Υ(Q) should:
be maximal if and only if the components of z areindependent, ie. iff Q is trivial (scale-permutation): Q = ΛP,
decrease if Q is not trivial.
This can be summarized by:
Υ(Q) ≤ Υ(I), with equality iff Q = ΛP (6)
Pierre Comon Tensor problems in EE 45
![Page 120: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/120.jpg)
Context Over. Orthog. Invertible Under. Criteria Pair sweep. Other
Examples of contrasts
Denote C the 4th order cumulant tensor of z. If at most 1 sourcehas a null kurtosis, then
ΥCoM(Q)def=∑
i C2iiii (maximize diagonal entries)
ΥSTO(Q)def=∑
ij C 2iiij (jointly diagonalize 3rd order slices)
ΥJAD(Q)def=∑
ijk C 2iijk (jointly diagonalize matrix slices)
are “contrast” criteria.
Pierre Comon Tensor problems in EE 46
![Page 121: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/121.jpg)
Context Over. Orthog. Invertible Under. Criteria Pair sweep. Other
Examples of contrasts (cont’d)
Proof. We need to prove (6), that is, Υ(Q) ≤ Υ(I). Denote D the(diagonal) cumulant tensor of s. First use multi-linearity (5) ofcumulant tensors: Cijk` =
∑p Qip Qjp Qkp Q`p Dp.
Then:
ΥJADdef=
∑ijk
C 2iijk =
∑ijk
|∑p
Q2ip Qjp Qkp Dp|2
=∑
i
∑pq
Q2ip Q2
iq
∑j
Qjp Qjq
∑k
Qkp Qkq Dp Dq
and because Q is orthogonal
ΥJAD =∑
i
∑p
Q4ip D2
p
Now Qij ≤ 1 yields ΥJAD ≤∑
i
∑p Q2
ip D2p =
∑i D
2i
def= ΥJAD(I)
This proves inequality (6) for ΥJAD .
Pierre Comon Tensor problems in EE 47
![Page 122: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/122.jpg)
Context Over. Orthog. Invertible Under. Criteria Pair sweep. Other
Examples of contrasts (cont’d)
Proof. We need to prove (6), that is, Υ(Q) ≤ Υ(I). Denote D the(diagonal) cumulant tensor of s. First use multi-linearity (5) ofcumulant tensors: Cijk` =
∑p Qip Qjp Qkp Q`p Dp. Then:
ΥJADdef=
∑ijk
C 2iijk =
∑ijk
|∑p
Q2ip Qjp Qkp Dp|2
=∑
i
∑pq
Q2ip Q2
iq
∑j
Qjp Qjq
∑k
Qkp Qkq Dp Dq
and because Q is orthogonal
ΥJAD =∑
i
∑p
Q4ip D2
p
Now Qij ≤ 1 yields ΥJAD ≤∑
i
∑p Q2
ip D2p =
∑i D
2i
def= ΥJAD(I)
This proves inequality (6) for ΥJAD .
Pierre Comon Tensor problems in EE 47
![Page 123: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/123.jpg)
Context Over. Orthog. Invertible Under. Criteria Pair sweep. Other
Examples of contrasts (cont’d)
Proof. We need to prove (6), that is, Υ(Q) ≤ Υ(I). Denote D the(diagonal) cumulant tensor of s. First use multi-linearity (5) ofcumulant tensors: Cijk` =
∑p Qip Qjp Qkp Q`p Dp. Then:
ΥJADdef=
∑ijk
C 2iijk =
∑ijk
|∑p
Q2ip Qjp Qkp Dp|2
=∑
i
∑pq
Q2ip Q2
iq
∑j
Qjp Qjq
∑k
Qkp Qkq Dp Dq
and because Q is orthogonal
ΥJAD =∑
i
∑p
Q4ip D2
p
Now Qij ≤ 1 yields ΥJAD ≤∑
i
∑p Q2
ip D2p =
∑i D
2i
def= ΥJAD(I)
This proves inequality (6) for ΥJAD .
Pierre Comon Tensor problems in EE 47
![Page 124: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/124.jpg)
Context Over. Orthog. Invertible Under. Criteria Pair sweep. Other
Examples of contrasts (cont’d)
Proof. We need to prove (6), that is, Υ(Q) ≤ Υ(I). Denote D the(diagonal) cumulant tensor of s. First use multi-linearity (5) ofcumulant tensors: Cijk` =
∑p Qip Qjp Qkp Q`p Dp. Then:
ΥJADdef=
∑ijk
C 2iijk =
∑ijk
|∑p
Q2ip Qjp Qkp Dp|2
=∑
i
∑pq
Q2ip Q2
iq
∑j
Qjp Qjq
∑k
Qkp Qkq Dp Dq
and because Q is orthogonal
ΥJAD =∑
i
∑p
Q4ip D2
p
Now Qij ≤ 1 yields ΥJAD ≤∑
i
∑p Q2
ip D2p =
∑i D
2i
def= ΥJAD(I)
This proves inequality (6) for ΥJAD .
Pierre Comon Tensor problems in EE 47
![Page 125: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/125.jpg)
Context Over. Orthog. Invertible Under. Criteria Pair sweep. Other
Proof. (cont’d) On the other hand, clearly
ΥCoM ≤ ΥSTO ≤ ΥJAD
so that we have also proved the inequality (6) for ΥCoM and ΥSTO .
Now if equality holds, all inequalities above are equalities, and inparticular ∑
i
∑p
(Q2ip − Q4
ip) D2p = 0
Thus |Qip| ∈ {0, 1} by Lp-norm inequality, for all i and for all psuch that Dp 6= 0. Yet, at most one Dp is null by hypothesis, andQ is orthogonal. Thus Q must be a signed permutation.
Pierre Comon Tensor problems in EE 48
![Page 126: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/126.jpg)
Context Over. Orthog. Invertible Under. Criteria Pair sweep. Other
Proof. (cont’d) On the other hand, clearly
ΥCoM ≤ ΥSTO ≤ ΥJAD
so that we have also proved the inequality (6) for ΥCoM and ΥSTO .
Now if equality holds, all inequalities above are equalities, and inparticular ∑
i
∑p
(Q2ip − Q4
ip) D2p = 0
Thus |Qip| ∈ {0, 1} by Lp-norm inequality, for all i and for all psuch that Dp 6= 0. Yet, at most one Dp is null by hypothesis, andQ is orthogonal. Thus Q must be a signed permutation.
Pierre Comon Tensor problems in EE 48
![Page 127: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/127.jpg)
Context Over. Orthog. Invertible Under. Criteria Pair sweep. Other
Proof. (cont’d) On the other hand, clearly
ΥCoM ≤ ΥSTO ≤ ΥJAD
so that we have also proved the inequality (6) for ΥCoM and ΥSTO .
Now if equality holds, all inequalities above are equalities, and inparticular ∑
i
∑p
(Q2ip − Q4
ip) D2p = 0
Thus |Qip| ∈ {0, 1} by Lp-norm inequality, for all i and for all psuch that Dp 6= 0. Yet, at most one Dp is null by hypothesis, andQ is orthogonal. Thus Q must be a signed permutation.
Pierre Comon Tensor problems in EE 48
![Page 128: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/128.jpg)
Context Over. Orthog. Invertible Under. Criteria Pair sweep. Other
Examples of contrasts (cont’d)
Interpretation.
ΥCoM ≤ ΥSTO ≤ ΥJAD shows that ΥCoM is more sensitive
ΥSTO and ΥJAD are less discriminant since they alsomaximize non diagonal terms
Example for 4× 4× 4 tensors
Matrix slices diagonalization 6= Tensor diagonalization
Pierre Comon Tensor problems in EE 49
![Page 129: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/129.jpg)
Context Over. Orthog. Invertible Under. Criteria Pair sweep. Other
Algorithms: Jacobi pair Sweeping (1)
2 Pairwise processing
Split the orthogonal matrix into a product of plane Givensrotations:
G[i , j ]def=
1√1 + θ2
(1 θ−θ 1
)acting in the subspace defined by (zi , zj).
ý the dimension has been reduced to 2, and we have a singleunknown, θ, that can be imposed to lie in (−1, 1].
Pierre Comon Tensor problems in EE 50
![Page 130: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/130.jpg)
Context Over. Orthog. Invertible Under. Criteria Pair sweep. Other
Algorithms: Jacobi pair Sweeping (2)
Cyclic sweeping with fixed ordering: Example in dimension P = 3
x x z
L−1
Carl Jacobi, 1804-1851
Pierre Comon Tensor problems in EE 51
![Page 131: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/131.jpg)
Context Over. Orthog. Invertible Under. Criteria Pair sweep. Other
Algorithms: Jacobi pair Sweeping (3)
Sweeping a 3× 3× 3 symmetric tensor X x xx x xx x x
x x x
x X xx x x
→ x x xx x xx x .
X x xx x xx x x
x x x
x . xx x x
→ x x xx x xx x X
. x xx x xx x x
x x x
x X xx x x
x x x
x x xx x X
X : maximizedx : minimized. : unchanged
by last Givens rotation
Pierre Comon Tensor problems in EE 52
![Page 132: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/132.jpg)
Context Over. Orthog. Invertible Under. Criteria Pair sweep. Other
Algorithms: Jacobi pair Sweeping (4)
Criteria ΥCoM , ΥSTO and ΥJAD , are rational functions of θ,and their absolute maxima can be computed algebraically.
To prove this, consider the elementary 2× 2 problem
z = G x,
denote Cijk` the cumulants of z, and
Gdef=
(cosβ sinβ− sinβ cosβ
)def=
1√1 + θ2
(1 θ−θ 1
)
Pierre Comon Tensor problems in EE 53
![Page 133: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/133.jpg)
Context Over. Orthog. Invertible Under. Criteria Pair sweep. Other
Solution for ΥCoM
We have ΥCoMdef= (C1111)
2 + (C2222)2
Denote ξ = θ − 1/θ. Then it is a rational function in ξ:
ψ4(ξ) = (ξ2 + 4)−24∑
i=0
bi ξi
Its stationary points are roots of a polynomial of degree 4:
ω4(ξ) =4∑
i=0
ci ξi
obtainable algebraically via Ferrari’s technique. Coefficients bi
and ci are given functions of cumulants of x.
θ is obtained from ξ by rooting a 2nd degree trinomial.
Pierre Comon Tensor problems in EE 54
![Page 134: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/134.jpg)
Context Over. Orthog. Invertible Under. Criteria Pair sweep. Other
Solution for ΥJAD
Goal: maximize squares of diagonal terms of GHN(r)G,
where matrix slices are denoted N(r) =
(ar br
cr dr
)and are
cumulants of x
Let vdef= [cos 2β, sin 2β]T. Then this amounts to maximizing
the quadratic form vT Mv where
Mdef=∑
r
[ar − dr
br + cr
][ar − dr , br + cr ]
Thus, 2β is given by the dominant eigenvector of M
and G is obtained by rooting a 2nd degree trinomial.
Pierre Comon Tensor problems in EE 55
![Page 135: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/135.jpg)
Context Over. Orthog. Invertible Under. Criteria Pair sweep. Other
Solution for ΥSTO
Goal: maximize squares of diagonal terms of 3rd order tensors
T[`]pqrdef=∑
ijk Gpi Gqj Grk Cijk`
If G =
(cosβ sinβ− sinβ cosβ
)then denote v
def= [cos 2β, sin 2β]T.
Angle 2β is given by vector v maximizing a quadratic formvTBv, where B is 2× 2 symmetric and contains sums ofproducts of cumulants of x
θ is obtained from ξ by rooting a 2nd degree trinomial.
Pierre Comon Tensor problems in EE 56
![Page 136: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/136.jpg)
Context Over. Orthog. Invertible Under. Criteria Pair sweep. Other
First conclusions
Pair sweeping can be executed thanks to the equivalencebetween pairwise and mutual independence
The cumulant tensor can be diagonalized iteratively via aJacobi-like algorithm
For each pair, there is a closed-form solution for the optimalGivens rotation (absolute maximimum of the contrastcriterion).
Questions:
But what about global convergence?
Pairwise processing holds valid if model is exact
Pierre Comon Tensor problems in EE 57
![Page 137: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/137.jpg)
Context Over. Orthog. Invertible Under. Criteria Pair sweep. Other
Stationary points: symmetric matrix case
Given a matrix m with components mij , it is sought for anorthogonal matrix Q such that Υ2 is maximized:
Υ2(Q) =∑
i
M2ii ; Mij =
∑p,q
QipQjq mpq.
Stationary points of Υ2 satisfy for any pair of indices(q, r), q 6= r :
MqqMqr = MrrMqr
Next, d2Υ2 < 0⇔ M2qr < (Mqq −Mrr )
2, which proves that
Mqr = 0, ∀q 6= r yields a maximumMqq = Grr , ∀q, r yields a minimumOther stationary points are saddle points
Pierre Comon Tensor problems in EE 58
![Page 138: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/138.jpg)
Context Over. Orthog. Invertible Under. Criteria Pair sweep. Other
Stationary points: symmetric tensor case
Similarly, one can look at relations characterizing localmaxima of criterion Υ
TqqqqTqqqr − TrrrrTqrrr = 0,
4T 2qqqr + 4T 2
qrrr − (Tqqqq −3
2Tqqrr )
2
−(Trrrr −3
2Tqqrr )
2 < 0.
for any pair of indices (p, q), p 6= q.
As a conclusion, contrary to Υ2 in the matrix case, Υ mighthave theoretically spurious local maxima in the tensor case(order > 2).
Pierre Comon Tensor problems in EE 59
![Page 139: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/139.jpg)
Context Over. Orthog. Invertible Under. Criteria Pair sweep. Other
Problem P2
1 At each step, a plane rotation is computed and yields theglobal maximum of the objective Υ restricted to one variable
2 There is no proof that the sequence of successive planerotations yields the global maximum, in the general case(tensors that are not necessarily diagonalizable)
3 Yet, no counter-example has been found since 1991
Pierre Comon Tensor problems in EE 60
![Page 140: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/140.jpg)
Context Over. Orthog. Invertible Under. Criteria Pair sweep. Other
Problem P2
1 At each step, a plane rotation is computed and yields theglobal maximum of the objective Υ restricted to one variable
2 There is no proof that the sequence of successive planerotations yields the global maximum, in the general case(tensors that are not necessarily diagonalizable)
3 Yet, no counter-example has been found since 1991
Pierre Comon Tensor problems in EE 60
![Page 141: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/141.jpg)
Context Over. Orthog. Invertible Under. Criteria Pair sweep. Other
Problem P2
1 At each step, a plane rotation is computed and yields theglobal maximum of the objective Υ restricted to one variable
2 There is no proof that the sequence of successive planerotations yields the global maximum, in the general case(tensors that are not necessarily diagonalizable)
3 Yet, no counter-example has been found since 1991
Pierre Comon Tensor problems in EE 60
![Page 142: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/142.jpg)
Context Over. Orthog. Invertible Under. Criteria Pair sweep. Other
Other algorithms for orthogonal diagonalization
The 3 previous criteria summarize most ways to address tensorapproximate diagonalization.But the orthogonal matrix can be treated differently, e.g. via otherparameterizations
express an orthogonal matrix as the exponential of askew-symmetric matrix: Q = expS
or use the Cayley parameterization: Q = (I− S)(I + S)−1,
...
Pierre Comon Tensor problems in EE 61
![Page 143: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/143.jpg)
Context Over. Orthog. Invertible Under. Criteria Pair sweep. Other
Joint Approximate Diagonalization s
The idea is to consider the symmetric tensor of dimension Kand order d as a collection of Kd−2 symmetric K ×K matrices
This is the Joint Approximate Diagonalization (JAD) problem
Pierre Comon Tensor problems in EE 62
![Page 144: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/144.jpg)
Context Over. Orthog. Invertible Under. Criteria Pair sweep. Other
Bibliographical comments s
Orthogonal diagonalization of symmetric tensors:
JAD in 2 modes: [DeLe78] (R), [CardS93] (C), with matrixexponential [TanaF07]JAD 2 modes with positive definite matrices: [Flur86]JAD in 3 modes: [DelaDV01]direct diago without slicing (pairs): [Como92] [Como94]
Orthogonal diagonalization of non symmetric tensors:
ALS type [Kroo83] [Kier92]ALS on pairs: [MartV08] [SoreC08]JAD in 2 modes (R): [Pesq01]Matrix exponential [SoreICD08]
Pierre Comon Tensor problems in EE 63
![Page 145: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/145.jpg)
Context Over. Orthog. Invertible Under. Proba. ALS Diag. dom. Joint Schur
Approximate diagonalization by invertible transform
Joint Approximate Diagonalization (JAD) of a collection of:
symmetric matricessymmetric diagonally dominant matricessymmetric positive definite matricesfor a collection of matrices, not necessarily symmetric (2invertible transforms)...
Direct approaches without slicing the tensor into a collectionof matrices: algorithms devised for underdetermined mixturesapply (cf. subsequent lecture)
Pierre Comon Tensor problems in EE 64
![Page 146: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/146.jpg)
Context Over. Orthog. Invertible Under. Proba. ALS Diag. dom. Joint Schur
Problem P3
Ill-poseness of optimization over set of invertible matrices
Possible solution:impose a constraint like detA ≥ η > 0 ?
Pierre Comon Tensor problems in EE 65
![Page 147: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/147.jpg)
Context Over. Orthog. Invertible Under. Proba. ALS Diag. dom. Joint Schur
Problem P3
Ill-poseness of optimization over set of invertible matrices
Possible solution:impose a constraint like detA ≥ η > 0 ?
Pierre Comon Tensor problems in EE 65
![Page 148: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/148.jpg)
Context Over. Orthog. Invertible Under. Proba. ALS Diag. dom. Joint Schur
Survey of 4 algorithms
1 Iterative algorithm based on a probabilistic criterion
2 An algorithm of ALS type: ACDC
3 An algorithm with a multiplicative update, valid if A isdiagonally dominant
4 An algorithm based on Joint triangularization
Pierre Comon Tensor problems in EE 66
![Page 149: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/149.jpg)
Context Over. Orthog. Invertible Under. Proba. ALS Diag. dom. Joint Schur
Survey of 4 algorithms
1 Iterative algorithm based on a probabilistic criterion
2 An algorithm of ALS type: ACDC
3 An algorithm with a multiplicative update, valid if A isdiagonally dominant
4 An algorithm based on Joint triangularization
Pierre Comon Tensor problems in EE 66
![Page 150: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/150.jpg)
Context Over. Orthog. Invertible Under. Proba. ALS Diag. dom. Joint Schur
Survey of 4 algorithms
1 Iterative algorithm based on a probabilistic criterion
2 An algorithm of ALS type: ACDC
3 An algorithm with a multiplicative update, valid if A isdiagonally dominant
4 An algorithm based on Joint triangularization
Pierre Comon Tensor problems in EE 66
![Page 151: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/151.jpg)
Context Over. Orthog. Invertible Under. Proba. ALS Diag. dom. Joint Schur
Survey of 4 algorithms
1 Iterative algorithm based on a probabilistic criterion
2 An algorithm of ALS type: ACDC
3 An algorithm with a multiplicative update, valid if A isdiagonally dominant
4 An algorithm based on Joint triangularization
Pierre Comon Tensor problems in EE 66
![Page 152: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/152.jpg)
Context Over. Orthog. Invertible Under. Proba. ALS Diag. dom. Joint Schur
Probabilistic approach (1) s
Let T(q) be a collection of symmetric positive semidefinitematricesLook for B such that M(q)
def= BT(q)BT are as diagonal as
possible.
1 Choose criterion to maximize Υdef=∑
q αq log detM(q)
detDiag{M(q)}We have Υ ≤ 0 from Hadamard’s inequality.
Avoids singularityLinked to Maximum Likelihood
2 Use multiplicative update as B(`+1) = UB(`)
3 Criterion after update: Υ =∑
q αq log detM(q) det2 U
detDiag{UM(q)UT}
Pierre Comon Tensor problems in EE 67
![Page 153: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/153.jpg)
Context Over. Orthog. Invertible Under. Proba. ALS Diag. dom. Joint Schur
Probabilistic approach (2) s
4 Variation of Υ during one update:∑q αq
[2 log detU− log detDiag{UM(q)UT}+ log detDiagM(q)
]5 Update two rows at a time, i.e. U is equal to Identity except
for entries (i , i), (i , j), (j , i), (j , j).By concavity of log, get a lower bound on variation:∑
q
αq
[2 log detU− log(UPUT)11 − log(UQUT)22
]where P and Q are the 2× 2 matrices:P =
∑q
pq
M(q)iiM(q)[i , j ] and Q =
∑q
pq
M(q)jjM(q)[i , j ]
6 Maximize this bound instead. This leads to rooting a 2nddegree trinomial. Sweep all the pairs in turns
Pierre Comon Tensor problems in EE 68
![Page 154: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/154.jpg)
Context Over. Orthog. Invertible Under. Proba. ALS Diag. dom. Joint Schur
Alternate Least Squares
1 Two writings of the criterion:
Υ =∑q
||T(q)− BΛ(q)BH||2
Υ =∑q
||t(q)− Bλ(q)||2
2 Stationary values for DiagΛ(q): λ(q) = {BH B}−1BH t(q)
3 Stationary value for each column b[`] of matrix B is thedominant eigenvector of the Hermitean matrix
P[`] =1
2
∑q
λ`(q){T[q; `]H + T[q; `]}
where T[q; `]def= T(q)−
∑n 6=` λn(q)b[n]b[n]H.
4 ALS: calculate Λ(q) and B alternately Use LS solution whenmatrices are singular.
Pierre Comon Tensor problems in EE 69
![Page 155: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/155.jpg)
Context Over. Orthog. Invertible Under. Proba. ALS Diag. dom. Joint Schur
Alternate Least Squares
1 Two writings of the criterion:
Υ =∑q
||T(q)− BΛ(q)BH||2
Υ =∑q
||t(q)− Bλ(q)||2
2 Stationary values for DiagΛ(q): λ(q) = {BH B}−1BH t(q)
3 Stationary value for each column b[`] of matrix B is thedominant eigenvector of the Hermitean matrix
P[`] =1
2
∑q
λ`(q){T[q; `]H + T[q; `]}
where T[q; `]def= T(q)−
∑n 6=` λn(q)b[n]b[n]H.
4 ALS: calculate Λ(q) and B alternately Use LS solution whenmatrices are singular.
Pierre Comon Tensor problems in EE 69
![Page 156: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/156.jpg)
Context Over. Orthog. Invertible Under. Proba. ALS Diag. dom. Joint Schur
Alternate Least Squares
1 Two writings of the criterion:
Υ =∑q
||T(q)− BΛ(q)BH||2
Υ =∑q
||t(q)− Bλ(q)||2
2 Stationary values for DiagΛ(q): λ(q) = {BH B}−1BH t(q)
3 Stationary value for each column b[`] of matrix B is thedominant eigenvector of the Hermitean matrix
P[`] =1
2
∑q
λ`(q){T[q; `]H + T[q; `]}
where T[q; `]def= T(q)−
∑n 6=` λn(q)b[n]b[n]H.
4 ALS: calculate Λ(q) and B alternately Use LS solution whenmatrices are singular.
Pierre Comon Tensor problems in EE 69
![Page 157: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/157.jpg)
Context Over. Orthog. Invertible Under. Proba. ALS Diag. dom. Joint Schur
Alternate Least Squares
1 Two writings of the criterion:
Υ =∑q
||T(q)− BΛ(q)BH||2
Υ =∑q
||t(q)− Bλ(q)||2
2 Stationary values for DiagΛ(q): λ(q) = {BH B}−1BH t(q)
3 Stationary value for each column b[`] of matrix B is thedominant eigenvector of the Hermitean matrix
P[`] =1
2
∑q
λ`(q){T[q; `]H + T[q; `]}
where T[q; `]def= T(q)−
∑n 6=` λn(q)b[n]b[n]H.
4 ALS: calculate Λ(q) and B alternately Use LS solution whenmatrices are singular.
Pierre Comon Tensor problems in EE 69
![Page 158: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/158.jpg)
Context Over. Orthog. Invertible Under. Proba. ALS Diag. dom. Joint Schur
Diagonally dominant matrices (1)
One wishes to minimize iteratively∑
q ||T(q)− AΛq AT||2Assume A is strictly diagonally dominant: |Aii | >
∑j 6=i |Aij |
(cf. Levy-Desplanques theorem)
1 Initialize A = I
2 Update A multiplicatively as A← (I + W)A, where W iszero-diagonal
3 Compute the best W assuming that it is small and that T(q)are almost diagonal (first order approximation)
Pierre Comon Tensor problems in EE 70
![Page 159: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/159.jpg)
Context Over. Orthog. Invertible Under. Proba. ALS Diag. dom. Joint Schur
Diagonally dominant matrices (2)
Computational details:
We have: T(`+1)(q)← (I + W)T(`)(q)(I + W)T
T(`)(q)def= (D− E), where D is diagonal, and E zero-diagonal
If W and E are small:T(`+1)(q) ≈ D(q) + WD(q) + D(q)WT − E(q)
Hence minimize, wrt W:∑q
∑i 6=j
|Wij Djj(q) + Wji Dii (q)− Eij(q)|2
This is of the form minw ||Jw− e||2, where J is sparse: JTJ isblock diagonal
Thus one gets at each iteration a collection of decoupled 2× 2linear systems
Pierre Comon Tensor problems in EE 71
![Page 160: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/160.jpg)
Context Over. Orthog. Invertible Under. Proba. ALS Diag. dom. Joint Schur
Joint triangularization of matrix slices
1 From T, determine a collection of matrices (e.g. matrixslices), T(q), satisfying T(q) = AD(q)BT,
D(q)def= diag{Cq,:}.
2 Compute the Generalized Schur decompositionQT(q)Z = R(q), where R(q) are upper-triangular
3 Since QT R(q)ZT = AD(q)BT, matrices R′ def= QA and
R′′ def= BT Z are upper triangular, and can be assumed to have
a unit diagonal. Hence R′ and R′′ can be computed bysolving from the bottom to the top the triangular system, twoentries R ′
ij and R ′′ij at a time:
R(q) = R′ D(q)R′′
4 Compute A = R′ QT and BT = R′′ ZT
5 Compute matrix C from T, A and B by solving theover-determined linear system C · {(B� A)T} = TK×JI
Pierre Comon Tensor problems in EE 72
![Page 161: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/161.jpg)
Context Over. Orthog. Invertible Under. Proba. ALS Diag. dom. Joint Schur
Joint triangularization of matrix slices
1 From T, determine a collection of matrices (e.g. matrixslices), T(q), satisfying T(q) = AD(q)BT,
D(q)def= diag{Cq,:}.
2 Compute the Generalized Schur decompositionQT(q)Z = R(q), where R(q) are upper-triangular
3 Since QT R(q)ZT = AD(q)BT, matrices R′ def= QA and
R′′ def= BT Z are upper triangular, and can be assumed to have
a unit diagonal. Hence R′ and R′′ can be computed bysolving from the bottom to the top the triangular system, twoentries R ′
ij and R ′′ij at a time:
R(q) = R′ D(q)R′′
4 Compute A = R′ QT and BT = R′′ ZT
5 Compute matrix C from T, A and B by solving theover-determined linear system C · {(B� A)T} = TK×JI
Pierre Comon Tensor problems in EE 72
![Page 162: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/162.jpg)
Context Over. Orthog. Invertible Under. Proba. ALS Diag. dom. Joint Schur
Joint triangularization of matrix slices
1 From T, determine a collection of matrices (e.g. matrixslices), T(q), satisfying T(q) = AD(q)BT,
D(q)def= diag{Cq,:}.
2 Compute the Generalized Schur decompositionQT(q)Z = R(q), where R(q) are upper-triangular
3 Since QT R(q)ZT = AD(q)BT, matrices R′ def= QA and
R′′ def= BT Z are upper triangular, and can be assumed to have
a unit diagonal. Hence R′ and R′′ can be computed bysolving from the bottom to the top the triangular system, twoentries R ′
ij and R ′′ij at a time:
R(q) = R′ D(q)R′′
4 Compute A = R′ QT and BT = R′′ ZT
5 Compute matrix C from T, A and B by solving theover-determined linear system C · {(B� A)T} = TK×JI
Pierre Comon Tensor problems in EE 72
![Page 163: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/163.jpg)
Context Over. Orthog. Invertible Under. Proba. ALS Diag. dom. Joint Schur
Joint triangularization of matrix slices
1 From T, determine a collection of matrices (e.g. matrixslices), T(q), satisfying T(q) = AD(q)BT,
D(q)def= diag{Cq,:}.
2 Compute the Generalized Schur decompositionQT(q)Z = R(q), where R(q) are upper-triangular
3 Since QT R(q)ZT = AD(q)BT, matrices R′ def= QA and
R′′ def= BT Z are upper triangular, and can be assumed to have
a unit diagonal. Hence R′ and R′′ can be computed bysolving from the bottom to the top the triangular system, twoentries R ′
ij and R ′′ij at a time:
R(q) = R′ D(q)R′′
4 Compute A = R′ QT and BT = R′′ ZT
5 Compute matrix C from T, A and B by solving theover-determined linear system C · {(B� A)T} = TK×JI
Pierre Comon Tensor problems in EE 72
![Page 164: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/164.jpg)
Context Over. Orthog. Invertible Under. Proba. ALS Diag. dom. Joint Schur
Joint triangularization of matrix slices
1 From T, determine a collection of matrices (e.g. matrixslices), T(q), satisfying T(q) = AD(q)BT,
D(q)def= diag{Cq,:}.
2 Compute the Generalized Schur decompositionQT(q)Z = R(q), where R(q) are upper-triangular
3 Since QT R(q)ZT = AD(q)BT, matrices R′ def= QA and
R′′ def= BT Z are upper triangular, and can be assumed to have
a unit diagonal. Hence R′ and R′′ can be computed bysolving from the bottom to the top the triangular system, twoentries R ′
ij and R ′′ij at a time:
R(q) = R′ D(q)R′′
4 Compute A = R′ QT and BT = R′′ ZT
5 Compute matrix C from T, A and B by solving theover-determined linear system C · {(B� A)T} = TK×JI
Pierre Comon Tensor problems in EE 72
![Page 165: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/165.jpg)
Context Over. Orthog. Invertible Under. Proba. ALS Diag. dom. Joint Schur
Bibliographical comments s
For collection of symmetric matrices:
maximizes iteratively a lower bound to the decrease on aprobabilistic objective [Pham01]
alternately minimize∑
q ||T(q)− AΛ(q)AT||2 wrt A andΛ(q) [Yere02]. See also: [Li07] [VollO06]
minimizes iteratively∑
q ||T(q)− AΛ(q)AT||2 under theassumption that A is diagonally dominant [Zieh04] .
For collection of non symmetric matrices:
factor A into orthogonal and triangular parts, and perform ajoint Schur decomposition [DelaDV04].
others: algorithms applicable to underdetermined case workhere [AlbeFCC05]
Pierre Comon Tensor problems in EE 73
![Page 166: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/166.jpg)
Context Over. Orthog. Invertible Under. Proba. ALS Diag. dom. Joint Schur
Lecture 3/3
Pierre Comon Tensor problems in EE 74
![Page 167: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/167.jpg)
Context Over. Orthog. Invertible Under. Proba. ALS Diag. dom. Joint Schur
What we have seen so far
For over-determined mixtures:
Solving the invertible problem is sufficient
Orthogonal framework:
Fully use 2nd order statistics, and then decomposeapproximately the tensor under orthogonal constraint.Easier to handle singularity, but arbitrary.Several (contrast) criteria and algorithms for orthogonaldecomposition
Invertible framework:
Decompose the higher order cumulant tensor directly underinvertible constraintIll-posedSeveral algorithms, mainly working with matrix slices
Principles & algorithms dedicated to under-determined mixturesmay apply
Pierre Comon Tensor problems in EE 75
![Page 168: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/168.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Binary case
James Joseph Sylvester (1814–1897)
Pierre Comon Tensor problems in EE 76
![Page 169: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/169.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Sylvester’s theorem
Sylvester’s theorem in R (1886)
A binary quantic t(x1, x2) =∑d
i=0 c(i) γi xi1 xd−i
2 can bewritten in R[x1, x2] as a sum of dth powers of r distinct linearforms:
t(x1, x2) =∑r
j=1 λj (αj x1 + βj x2)d if and only if:
1 there exists a vector g of dimension r + 1 such thatγ0 γ1 · · · γr
γ1 γ2 · · · γr+1
......
γd−r · · · γd
g0
g1
...gr
= 0. (7)
2 q(x1, x2)def=∑r
`=0 g` x`1 x r−`
2 has r distinct real roots
Then q(x1, x2)def=∏r
j=1(βj x1 − αj x2) yields the r forms
Valid even in non generic cases.
Pierre Comon Tensor problems in EE 77
![Page 170: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/170.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Sylvester’s theorem
Sylvester’s theorem in R (1886)
A binary quantic t(x1, x2) =∑d
i=0 c(i) γi xi1 xd−i
2 can bewritten in R[x1, x2] as a sum of dth powers of r distinct linearforms:
t(x1, x2) =∑r
j=1 λj (αj x1 + βj x2)d if and only if:
1 there exists a vector g of dimension r + 1 such thatγ0 γ1 · · · γr
γ1 γ2 · · · γr+1
......
γd−r · · · γd
g0
g1
...gr
= 0. (7)
2 q(x1, x2)def=∑r
`=0 g` x`1 x r−`
2 has r distinct real roots
Then q(x1, x2)def=∏r
j=1(βj x1 − αj x2) yields the r forms
Valid even in non generic cases.
Pierre Comon Tensor problems in EE 77
![Page 171: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/171.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Sylvester’s theorem
Sylvester’s theorem in R (1886)
A binary quantic t(x1, x2) =∑d
i=0 c(i) γi xi1 xd−i
2 can bewritten in R[x1, x2] as a sum of dth powers of r distinct linearforms:
t(x1, x2) =∑r
j=1 λj (αj x1 + βj x2)d if and only if:
1 there exists a vector g of dimension r + 1 such thatγ0 γ1 · · · γr
γ1 γ2 · · · γr+1
......
γd−r · · · γd
g0
g1
...gr
= 0. (7)
2 q(x1, x2)def=∑r
`=0 g` x`1 x r−`
2 has r distinct real roots
Then q(x1, x2)def=∏r
j=1(βj x1 − αj x2) yields the r forms
Valid even in non generic cases.
Pierre Comon Tensor problems in EE 77
![Page 172: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/172.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Sylvester’s theorem
Sylvester’s theorem in R (1886)
A binary quantic t(x1, x2) =∑d
i=0 c(i) γi xi1 xd−i
2 can bewritten in R[x1, x2] as a sum of dth powers of r distinct linearforms:
t(x1, x2) =∑r
j=1 λj (αj x1 + βj x2)d if and only if:
1 there exists a vector g of dimension r + 1 such thatγ0 γ1 · · · γr
γ1 γ2 · · · γr+1
......
γd−r · · · γd
g0
g1
...gr
= 0. (7)
2 q(x1, x2)def=∑r
`=0 g` x`1 x r−`
2 has r distinct real roots
Then q(x1, x2)def=∏r
j=1(βj x1 − αj x2) yields the r forms
Valid even in non generic cases.
Pierre Comon Tensor problems in EE 77
![Page 173: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/173.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Sylvester’s theorem
Sylvester’s theorem in R (1886)
A binary quantic t(x1, x2) =∑d
i=0 c(i) γi xi1 xd−i
2 can bewritten in R[x1, x2] as a sum of dth powers of r distinct linearforms:
t(x1, x2) =∑r
j=1 λj (αj x1 + βj x2)d if and only if:
1 there exists a vector g of dimension r + 1 such thatγ0 γ1 · · · γr
γ1 γ2 · · · γr+1
......
γd−r · · · γd
g0
g1
...gr
= 0. (7)
2 q(x1, x2)def=∑r
`=0 g` x`1 x r−`
2 has r distinct real roots
Then q(x1, x2)def=∏r
j=1(βj x1 − αj x2) yields the r forms
Valid even in non generic cases.
Pierre Comon Tensor problems in EE 77
![Page 174: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/174.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Sylvester’s theorem
Sylvester’s theorem in R (1886)
A binary quantic t(x1, x2) =∑d
i=0 c(i) γi xi1 xd−i
2 can bewritten in R[x1, x2] as a sum of dth powers of r distinct linearforms:
t(x1, x2) =∑r
j=1 λj (αj x1 + βj x2)d if and only if:
1 there exists a vector g of dimension r + 1 such thatγ0 γ1 · · · γr
γ1 γ2 · · · γr+1
......
γd−r · · · γd
g0
g1
...gr
= 0. (7)
2 q(x1, x2)def=∑r
`=0 g` x`1 x r−`
2 has r distinct real roots
Then q(x1, x2)def=∏r
j=1(βj x1 − αj x2) yields the r forms
Valid even in non generic cases.
Pierre Comon Tensor problems in EE 77
![Page 175: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/175.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Proof of Sylvester’s theorem (1)
Lemma
For homogeneous polynomials of degree d parameterized as
p(x)def=∑
|i|=d c(i) γ(i; p) xi, define the apolar scalar product:
〈p, q〉 =∑|i|=d
c(i) γ(i; p) γ(i; q)
Then L(x)def= aTx ⇒ 〈p, Ld〉 =
∑|i|=d c(i) γ(i; p) ai = p(a)
Pierre Comon Tensor problems in EE 78
![Page 176: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/176.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Proof of Sylvester’s theorem (1)
Lemma
For homogeneous polynomials of degree d parameterized as
p(x)def=∑
|i|=d c(i) γ(i; p) xi, define the apolar scalar product:
〈p, q〉 =∑|i|=d
c(i) γ(i; p) γ(i; q)
Then L(x)def= aTx ⇒ 〈p, Ld〉 =
∑|i|=d c(i) γ(i; p) ai = p(a)
Pierre Comon Tensor problems in EE 78
![Page 177: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/177.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Proof of Sylvester’s theorem (2)
1 Assume the r distinct linear forms Lj = αjx1 + βjx2 are given.
Let q(x)def=∏r
j=1(βjx1 − αjx2). Then q(αj , βj) = 0, ∀j .
2 Hence from lemma, ∀m(x) of degree d − r ,〈mq, Ld
j 〉 = mq(aj) = 0, and 〈mq, t〉 = 0.
3 Take for instance polynomials mµ(x) = xµ1 xd−r−µ
2 ,1 ≤ µ ≤ d − r , and denote g` coefficients of q:
〈mµq, t〉 = 0 ⇒r∑
`=0
g` γ`+µ = 0
This is exactly (7) expressed in canonical basis
4 Roots of q(x1, x2) are distinct real since forms Lj are.
5 Reasoning goes also backwards
Pierre Comon Tensor problems in EE 79
![Page 178: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/178.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Proof of Sylvester’s theorem (2)
1 Assume the r distinct linear forms Lj = αjx1 + βjx2 are given.
Let q(x)def=∏r
j=1(βjx1 − αjx2). Then q(αj , βj) = 0, ∀j .2 Hence from lemma, ∀m(x) of degree d − r ,〈mq, Ld
j 〉 = mq(aj) = 0, and 〈mq, t〉 = 0.
3 Take for instance polynomials mµ(x) = xµ1 xd−r−µ
2 ,1 ≤ µ ≤ d − r , and denote g` coefficients of q:
〈mµq, t〉 = 0 ⇒r∑
`=0
g` γ`+µ = 0
This is exactly (7) expressed in canonical basis
4 Roots of q(x1, x2) are distinct real since forms Lj are.
5 Reasoning goes also backwards
Pierre Comon Tensor problems in EE 79
![Page 179: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/179.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Proof of Sylvester’s theorem (2)
1 Assume the r distinct linear forms Lj = αjx1 + βjx2 are given.
Let q(x)def=∏r
j=1(βjx1 − αjx2). Then q(αj , βj) = 0, ∀j .2 Hence from lemma, ∀m(x) of degree d − r ,〈mq, Ld
j 〉 = mq(aj) = 0, and 〈mq, t〉 = 0.
3 Take for instance polynomials mµ(x) = xµ1 xd−r−µ
2 ,1 ≤ µ ≤ d − r , and denote g` coefficients of q:
〈mµq, t〉 = 0 ⇒r∑
`=0
g` γ`+µ = 0
This is exactly (7) expressed in canonical basis
4 Roots of q(x1, x2) are distinct real since forms Lj are.
5 Reasoning goes also backwards
Pierre Comon Tensor problems in EE 79
![Page 180: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/180.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Proof of Sylvester’s theorem (2)
1 Assume the r distinct linear forms Lj = αjx1 + βjx2 are given.
Let q(x)def=∏r
j=1(βjx1 − αjx2). Then q(αj , βj) = 0, ∀j .2 Hence from lemma, ∀m(x) of degree d − r ,〈mq, Ld
j 〉 = mq(aj) = 0, and 〈mq, t〉 = 0.
3 Take for instance polynomials mµ(x) = xµ1 xd−r−µ
2 ,1 ≤ µ ≤ d − r , and denote g` coefficients of q:
〈mµq, t〉 = 0 ⇒r∑
`=0
g` γ`+µ = 0
This is exactly (7) expressed in canonical basis
4 Roots of q(x1, x2) are distinct real since forms Lj are.
5 Reasoning goes also backwards
Pierre Comon Tensor problems in EE 79
![Page 181: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/181.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Proof of Sylvester’s theorem (2)
1 Assume the r distinct linear forms Lj = αjx1 + βjx2 are given.
Let q(x)def=∏r
j=1(βjx1 − αjx2). Then q(αj , βj) = 0, ∀j .2 Hence from lemma, ∀m(x) of degree d − r ,〈mq, Ld
j 〉 = mq(aj) = 0, and 〈mq, t〉 = 0.
3 Take for instance polynomials mµ(x) = xµ1 xd−r−µ
2 ,1 ≤ µ ≤ d − r , and denote g` coefficients of q:
〈mµq, t〉 = 0 ⇒r∑
`=0
g` γ`+µ = 0
This is exactly (7) expressed in canonical basis
4 Roots of q(x1, x2) are distinct real since forms Lj are.
5 Reasoning goes also backwards
Pierre Comon Tensor problems in EE 79
![Page 182: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/182.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Algorithm for r th order symmetric tensors of dimension 2
Start with r = 1 (d × 2 matrix) and increase r until it looses itscolumn rank
1 2
2 3
3 4
4 5
5 6
6 7
7 8
−→
1 2 3
2 3 4
3 4 5
4 5 6
5 6 7
6 7 8
−→
1 2 3 4
2 3 4 5
3 4 5 6
4 5 6 7
5 6 7 8
Pierre Comon Tensor problems in EE 80
![Page 183: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/183.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Decomposition of maximal rank: x1xd−12
1 Maximal rank r = d when (7) reduces to a 1-row matrix:
[0, 0, . . . 0, 1, 0] g = 0
2 Find (αi , βi ) such that q(x1, x1) =∏d
j=1(βjx1 − αjx2)def=∑d
`=0 g` x`1 x r−`
2 has d distinct roots
3 Take αj = 1. Then gd−1 = 0 just means∑βj = 0
Choose arbitrarily such distinct βj ’s
4 Compute λj ’s by solving the Van der Monde linear system:1 . . . 1β1 . . . βd
β21 . . . β2
d
: : :βd
1 . . . βdd
λ =
0:010
Pierre Comon Tensor problems in EE 81
![Page 184: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/184.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Decomposition of maximal rank: x1xd−12
1 Maximal rank r = d when (7) reduces to a 1-row matrix:
[0, 0, . . . 0, 1, 0] g = 0
2 Find (αi , βi ) such that q(x1, x1) =∏d
j=1(βjx1 − αjx2)def=∑d
`=0 g` x`1 x r−`
2 has d distinct roots
3 Take αj = 1. Then gd−1 = 0 just means∑βj = 0
Choose arbitrarily such distinct βj ’s
4 Compute λj ’s by solving the Van der Monde linear system:1 . . . 1β1 . . . βd
β21 . . . β2
d
: : :βd
1 . . . βdd
λ =
0:010
Pierre Comon Tensor problems in EE 81
![Page 185: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/185.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Decomposition of maximal rank: x1xd−12
1 Maximal rank r = d when (7) reduces to a 1-row matrix:
[0, 0, . . . 0, 1, 0] g = 0
2 Find (αi , βi ) such that q(x1, x1) =∏d
j=1(βjx1 − αjx2)def=∑d
`=0 g` x`1 x r−`
2 has d distinct roots
3 Take αj = 1. Then gd−1 = 0 just means∑βj = 0
Choose arbitrarily such distinct βj ’s
4 Compute λj ’s by solving the Van der Monde linear system:1 . . . 1β1 . . . βd
β21 . . . β2
d
: : :βd
1 . . . βdd
λ =
0:010
Pierre Comon Tensor problems in EE 81
![Page 186: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/186.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Decomposition of maximal rank: x1xd−12
1 Maximal rank r = d when (7) reduces to a 1-row matrix:
[0, 0, . . . 0, 1, 0] g = 0
2 Find (αi , βi ) such that q(x1, x1) =∏d
j=1(βjx1 − αjx2)def=∑d
`=0 g` x`1 x r−`
2 has d distinct roots
3 Take αj = 1. Then gd−1 = 0 just means∑βj = 0
Choose arbitrarily such distinct βj ’s
4 Compute λj ’s by solving the Van der Monde linear system:1 . . . 1β1 . . . βd
β21 . . . β2
d
: : :βd
1 . . . βdd
λ =
0:010
Pierre Comon Tensor problems in EE 81
![Page 187: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/187.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Rank of binary quantics
A binary quantic of odd degree 2n + 1 has generic rank n + 1
A binary quantic of even degree 2n generic rank n + 1
A binary quantic of degree d may reach maximal rank d .Orbit of maximal rank is x1 xd−1
2 .
Sylvester’s theorem allows to compute the decomposition,even in non generic cases
Pierre Comon Tensor problems in EE 82
![Page 188: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/188.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Rank of binary quantics
A binary quantic of odd degree 2n + 1 has generic rank n + 1
A binary quantic of even degree 2n generic rank n + 1
A binary quantic of degree d may reach maximal rank d .Orbit of maximal rank is x1 xd−1
2 .
Sylvester’s theorem allows to compute the decomposition,even in non generic cases
Pierre Comon Tensor problems in EE 82
![Page 189: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/189.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Rank of binary quantics
A binary quantic of odd degree 2n + 1 has generic rank n + 1
A binary quantic of even degree 2n generic rank n + 1
A binary quantic of degree d may reach maximal rank d .Orbit of maximal rank is x1 xd−1
2 .
Sylvester’s theorem allows to compute the decomposition,even in non generic cases
Pierre Comon Tensor problems in EE 82
![Page 190: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/190.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Rank of binary quantics
A binary quantic of odd degree 2n + 1 has generic rank n + 1
A binary quantic of even degree 2n generic rank n + 1
A binary quantic of degree d may reach maximal rank d .Orbit of maximal rank is x1 xd−1
2 .
Sylvester’s theorem allows to compute the decomposition,even in non generic cases
Pierre Comon Tensor problems in EE 82
![Page 191: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/191.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Problem P16
In the super generic case, there are infinitely manydecompositions
In dimension larger than 2: simple ways to compute one suchdecomposition, as in binary case?
Pierre Comon Tensor problems in EE 83
![Page 192: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/192.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Alexander-Hirschowitz theorem
Theorem (1995) For d > 2, the generic rank of a dth ordersymmetric tensor of dimension K is always equal to the lowerbound
Rs =
⌈(K+d−1d
)K
⌉(8)
except for the following cases:(d ,K ) ∈ {(3, 5), (4, 3), (4, 4), (4, 5)}, for which it should beincreased by 1 (i.e. only a finite number of exceptions, also calleddefective cases)
Pierre Comon Tensor problems in EE 84
![Page 193: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/193.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Values of the Generic Rank (1)
Symmetric tensors of order d and dimension K
dK 2 3 4 5 6 7 8
3 2 4 5 8 10 12 154 3 6 10 15 21 30 42
Rs ≥1
K
(K + d − 1
d
)
Bold: exceptions to the ceil rule: Rs = d 1K
(K+d−1d
)e, sometimes
called defective cases.Green: lower bound 1
K
(K+d−1d
)is integer and nondefective,
hence finite number of solutions with proba 1
Pierre Comon Tensor problems in EE 85
![Page 194: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/194.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Values of the Generic Rank (2)
Warning: for unsymmetric tensors of order d and dimension K ,the generic rank is different
dK 2 3 4 5 6 7
3 2 5 7 10 14 194 4 9 20 37 62 97
R ≥ Kd
Kd − d + 1
Bold: exceptions to the ceil rule: R = d Kd
Kd−d+1e.Green: lower bound Kd
Kd−d+1 is integer and nondefective
Pierre Comon Tensor problems in EE 86
![Page 195: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/195.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Numerical computation of the Generic Rank
Mapping (for unsymmetric tensors):
{u(`), v(`), . . . ,w(`), 1 ≤ ` ≤ r} ϕ−→r∑
`=1
u(`) ⊗ v(`) ⊗ . . . ⊗w(`)
{Cn1 ⊗ . . . ⊗Cnd}r ϕ−→ A
ý The smallest r for wich rank(Jacobian(ϕ)) =∏
i ni is thegeneric rank, R.
ý Example of use of Terracini’s lemma
Pierre Comon Tensor problems in EE 87
![Page 196: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/196.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Example of computation of Generic Rank
{a(`),b(`), c(`)} ϕ−→ T =r∑
`=1
a(`) ⊗b(`) ⊗ c(`)
T has coordinate vector:∑r
`=1 a(`)⊗ b(`)⊗ c(`). Hence theJacobian of ϕ is the r(n1 + n2 + n3)× n1n2n3 matrix:
J =
In1 ⊗ bT(1) ⊗ cT(1): ⊗ : ⊗ :
In1 ⊗ bT(r) ⊗ cT(r)a(1)T ⊗ In2 ⊗ cT(1)
: ⊗ : ⊗ :a(r)T ⊗ In2 ⊗ cT(r)a(1)T ⊗ b(1)T ⊗ In3
: ⊗ : ⊗ :a(r)T ⊗ b(r)T ⊗ In3
and
{rank{J} = dim(Im(ϕ))R = Min{r : Im{ϕ} = A}
Pierre Comon Tensor problems in EE 88
![Page 197: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/197.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Problem P5
Similar theorem as AH for unsymmetric tensors?
that is, decomposition of homogeneous polynomials of degreed but partial degree 1 into sum of products of linear forms
Lectures of Giorgio Ottaviani today...
Pierre Comon Tensor problems in EE 89
![Page 198: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/198.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Genericity in R vs. C
Define Zr = {tensors of rank r}A rank r is typical if Zr is Zariski-dense
In the complex field, there is only one typical rank, called thegeneric rank.
In the real field, there can be several typical ranks(smallest equals generic rank in C)
Pierre Comon Tensor problems in EE 90
![Page 199: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/199.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Genericity in R vs. C
Define Zr = {tensors of rank r}A rank r is typical if Zr is Zariski-dense
In the complex field, there is only one typical rank, called thegeneric rank.
In the real field, there can be several typical ranks(smallest equals generic rank in C)
Pierre Comon Tensor problems in EE 90
![Page 200: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/200.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Genericity in R vs. C
Define Zr = {tensors of rank r}A rank r is typical if Zr is Zariski-dense
In the complex field, there is only one typical rank, called thegeneric rank.
In the real field, there can be several typical ranks(smallest equals generic rank in C)
Pierre Comon Tensor problems in EE 90
![Page 201: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/201.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Problem P10
Case of the Real field
Similar result as AH theorem in the real field?
i.e. values of typical ranks for any order and dimensions
Pierre Comon Tensor problems in EE 91
![Page 202: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/202.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Low rank approximation
We need
1 to know the exact rankand
2 the rank to be sub-generic
Hence we make suboptimal rank reduction
by a 2-stage rank reductionor
by HOSVD truncation
Pierre Comon Tensor problems in EE 92
![Page 203: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/203.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Two-stage suboptimal rank reduction (1)
1 Associate one linear operator with tensor T, defined by amatrix M
2 Compute the best rank r approximate of M, e.g. viatruncated SVD (not always possible)
3 Unfold each of the r singular vectors into a matrix, andcompute its rank-1 approximate
4 From this starting point, run a few iterations of a descent on
||T−r∑
p=1
up⊗ vp
⊗ . . . ⊗wp||2
Pierre Comon Tensor problems in EE 93
![Page 204: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/204.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Two-stage suboptimal rank reduction (1)
1 Associate one linear operator with tensor T, defined by amatrix M
2 Compute the best rank r approximate of M, e.g. viatruncated SVD (not always possible)
3 Unfold each of the r singular vectors into a matrix, andcompute its rank-1 approximate
4 From this starting point, run a few iterations of a descent on
||T−r∑
p=1
up⊗ vp
⊗ . . . ⊗wp||2
Pierre Comon Tensor problems in EE 93
![Page 205: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/205.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Two-stage suboptimal rank reduction (1)
1 Associate one linear operator with tensor T, defined by amatrix M
2 Compute the best rank r approximate of M, e.g. viatruncated SVD (not always possible)
3 Unfold each of the r singular vectors into a matrix, andcompute its rank-1 approximate
4 From this starting point, run a few iterations of a descent on
||T−r∑
p=1
up⊗ vp
⊗ . . . ⊗wp||2
Pierre Comon Tensor problems in EE 93
![Page 206: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/206.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Two-stage suboptimal rank reduction (1)
1 Associate one linear operator with tensor T, defined by amatrix M
2 Compute the best rank r approximate of M, e.g. viatruncated SVD (not always possible)
3 Unfold each of the r singular vectors into a matrix, andcompute its rank-1 approximate
4 From this starting point, run a few iterations of a descent on
||T−r∑
p=1
up⊗ vp
⊗ . . . ⊗wp||2
Pierre Comon Tensor problems in EE 93
![Page 207: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/207.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Two-stage suboptimal rank reduction (2)
Example: If T is K × K × K × K symmetric
1 build the symmetric matrix M of size K 2 × K 2.
2 Compute the r dominant eigenvectors ep of M.
3 We wish each e to be of the form u ⊗u.
Hence minimize ||UnvecK (ep)− up uTp ||2 via rank-1
approximatesEventually:
T ≈r∑
p=1
up⊗up
⊗up⊗up
Pierre Comon Tensor problems in EE 94
![Page 208: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/208.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Two-stage suboptimal rank reduction (2)
Example: If T is K × K × K × K symmetric
1 build the symmetric matrix M of size K 2 × K 2.
2 Compute the r dominant eigenvectors ep of M.
3 We wish each e to be of the form u ⊗u.
Hence minimize ||UnvecK (ep)− up uTp ||2 via rank-1
approximatesEventually:
T ≈r∑
p=1
up⊗up
⊗up⊗up
Pierre Comon Tensor problems in EE 94
![Page 209: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/209.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Two-stage suboptimal rank reduction (2)
Example: If T is K × K × K × K symmetric
1 build the symmetric matrix M of size K 2 × K 2.
2 Compute the r dominant eigenvectors ep of M.
3 We wish each e to be of the form u ⊗u.
Hence minimize ||UnvecK (ep)− up uTp ||2 via rank-1
approximatesEventually:
T ≈r∑
p=1
up⊗up
⊗up⊗up
Pierre Comon Tensor problems in EE 94
![Page 210: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/210.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Two-stage suboptimal rank reduction (2)
Example: If T is K × K × K × K symmetric
1 build the symmetric matrix M of size K 2 × K 2.
2 Compute the r dominant eigenvectors ep of M.
3 We wish each e to be of the form u ⊗u.
Hence minimize ||UnvecK (ep)− up uTp ||2 via rank-1
approximatesEventually:
T ≈r∑
p=1
up⊗up
⊗up⊗up
Pierre Comon Tensor problems in EE 94
![Page 211: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/211.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Two-stage suboptimal rank reduction (2)
Example: If T is K × K × K × K symmetric
1 build the symmetric matrix M of size K 2 × K 2.
2 Compute the r dominant eigenvectors ep of M.
3 We wish each e to be of the form u ⊗u.Hence minimize ||UnvecK (ep)− up uT
p ||2 via rank-1approximates
Eventually:
T ≈r∑
p=1
up⊗up
⊗up⊗up
Pierre Comon Tensor problems in EE 94
![Page 212: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/212.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Two-stage suboptimal rank reduction (2)
Example: If T is K × K × K × K symmetric
1 build the symmetric matrix M of size K 2 × K 2.
2 Compute the r dominant eigenvectors ep of M.
3 We wish each e to be of the form u ⊗u.Hence minimize ||UnvecK (ep)− up uT
p ||2 via rank-1approximatesEventually:
T ≈r∑
p=1
up⊗up
⊗up⊗up
Pierre Comon Tensor problems in EE 94
![Page 213: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/213.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
HOSVD s
For a tensor of order d , compute first a rank-(R1,R2, . . . ,Rd)approximate:
1 Associate d linear operators with tensor T, defined byunfolding matrices Mi , 1 ≤ i ≤ d
2 Compute the rank-Ri approximation of each matrix Mi
(reduction of the multilinear mode-i ranks)
3 Use truncated left singular marices U(i) to compute theR1 × R2 × . . .Rd core tensor T0
4 Run an iterative descent on ||T0 −∑r
p=1 up⊗ vp
⊗ . . . ⊗wp||2
Pierre Comon Tensor problems in EE 95
![Page 214: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/214.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Problem P4
Approximation of a tensor by another of lower rank
1 ill-posed in general for free real/complex entries
2 other problem statement (e.g. border rank)?
3 case of positive entries
4 case of semi-definite positive quantic
5 other cases?
Pierre Comon Tensor problems in EE 96
![Page 215: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/215.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Problem P4
Approximation of a tensor by another of lower rank
1 ill-posed in general for free real/complex entries
2 other problem statement (e.g. border rank)?
3 case of positive entries
4 case of semi-definite positive quantic
5 other cases?
Pierre Comon Tensor problems in EE 96
![Page 216: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/216.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Problem P4
Approximation of a tensor by another of lower rank
1 ill-posed in general for free real/complex entries
2 other problem statement (e.g. border rank)?
3 case of positive entries
4 case of semi-definite positive quantic
5 other cases?
Pierre Comon Tensor problems in EE 96
![Page 217: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/217.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Problem P4
Approximation of a tensor by another of lower rank
1 ill-posed in general for free real/complex entries
2 other problem statement (e.g. border rank)?
3 case of positive entries
4 case of semi-definite positive quantic
5 other cases?
Pierre Comon Tensor problems in EE 96
![Page 218: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/218.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Problem P4
Approximation of a tensor by another of lower rank
1 ill-posed in general for free real/complex entries
2 other problem statement (e.g. border rank)?
3 case of positive entries
4 case of semi-definite positive quantic
5 other cases?
Pierre Comon Tensor problems in EE 96
![Page 219: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/219.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Algorithms when K > 2
Now survey some numerical algorithms to compute thedecomposition:
when rank is strictly smaller than generic
when dimension is larger than 2
suboptimality (symmetry not fully imposed - link with P15)
Pierre Comon Tensor problems in EE 97
![Page 220: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/220.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
BIOME algorithms
These algorithms work with a cumulant tensor of even order2r > 4
Related to symmetric flattening introduced in previous lectures
We take the case 2r = 6 for the presentation, and denote
C`mnijk
def= Cum{xi , xj , xk , x∗l , x∗m, x∗n} (9)
In that case, we have
C`mnx , ijk =
P∑p=1
Hip Hjp Hkp H∗`p H∗
mp H∗np ∆p
where ∆pdef= Cum{sp, sp, sp, s∗p , s∗p , s∗p} denote the diagonal
entries of a P × P diagonal matrix, ∆(6)
Pierre Comon Tensor problems in EE 98
![Page 221: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/221.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
BIOME algorithms
These algorithms work with a cumulant tensor of even order2r > 4
Related to symmetric flattening introduced in previous lectures
We take the case 2r = 6 for the presentation, and denote
C`mnijk
def= Cum{xi , xj , xk , x∗l , x∗m, x∗n} (9)
In that case, we have
C`mnx , ijk =
P∑p=1
Hip Hjp Hkp H∗`p H∗
mp H∗np ∆p
where ∆pdef= Cum{sp, sp, sp, s∗p , s∗p , s∗p} denote the diagonal
entries of a P × P diagonal matrix, ∆(6)
Pierre Comon Tensor problems in EE 98
![Page 222: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/222.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
BIOME algorithms
These algorithms work with a cumulant tensor of even order2r > 4
Related to symmetric flattening introduced in previous lectures
We take the case 2r = 6 for the presentation, and denote
C`mnijk
def= Cum{xi , xj , xk , x∗l , x∗m, x∗n} (9)
In that case, we have
C`mnx , ijk =
P∑p=1
Hip Hjp Hkp H∗`p H∗
mp H∗np ∆p
where ∆pdef= Cum{sp, sp, sp, s∗p , s∗p , s∗p} denote the diagonal
entries of a P × P diagonal matrix, ∆(6)
Pierre Comon Tensor problems in EE 98
![Page 223: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/223.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
BIOME algorithms
These algorithms work with a cumulant tensor of even order2r > 4
Related to symmetric flattening introduced in previous lectures
We take the case 2r = 6 for the presentation, and denote
C`mnijk
def= Cum{xi , xj , xk , x∗l , x∗m, x∗n} (9)
In that case, we have
C`mnx , ijk =
P∑p=1
Hip Hjp Hkp H∗`p H∗
mp H∗np ∆p
where ∆pdef= Cum{sp, sp, sp, s∗p , s∗p , s∗p} denote the diagonal
entries of a P × P diagonal matrix, ∆(6)
Pierre Comon Tensor problems in EE 98
![Page 224: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/224.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Writing in matrix form
Tensor Cx is of dimensions K × K × K × K × K × K andenjoys symmetries and Hermitian symmetries.
Tensor Cx can be stored in a K 3 × K 3 Hermitian matrix, C(6)x ,
called the hexacovariance. With an appropriate storage of thetensor entries, we have
C(6)x = H�3 ∆(6) H�3H (10)
Because C(6)x is Hermitian, ∃V unitary, such that
(C(6)x )1/2 = H�3 (∆(6))1/2 V (11)
Idea: Use redundancy existing between blocks of (C(6)x )1/2.
Pierre Comon Tensor problems in EE 99
![Page 225: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/225.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Writing in matrix form
Tensor Cx is of dimensions K × K × K × K × K × K andenjoys symmetries and Hermitian symmetries.
Tensor Cx can be stored in a K 3 × K 3 Hermitian matrix, C(6)x ,
called the hexacovariance. With an appropriate storage of thetensor entries, we have
C(6)x = H�3 ∆(6) H�3H (10)
Because C(6)x is Hermitian, ∃V unitary, such that
(C(6)x )1/2 = H�3 (∆(6))1/2 V (11)
Idea: Use redundancy existing between blocks of (C(6)x )1/2.
Pierre Comon Tensor problems in EE 99
![Page 226: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/226.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Writing in matrix form
Tensor Cx is of dimensions K × K × K × K × K × K andenjoys symmetries and Hermitian symmetries.
Tensor Cx can be stored in a K 3 × K 3 Hermitian matrix, C(6)x ,
called the hexacovariance. With an appropriate storage of thetensor entries, we have
C(6)x = H�3 ∆(6) H�3H (10)
Because C(6)x is Hermitian, ∃V unitary, such that
(C(6)x )1/2 = H�3 (∆(6))1/2 V (11)
Idea: Use redundancy existing between blocks of (C(6)x )1/2.
Pierre Comon Tensor problems in EE 99
![Page 227: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/227.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Writing in matrix form
Tensor Cx is of dimensions K × K × K × K × K × K andenjoys symmetries and Hermitian symmetries.
Tensor Cx can be stored in a K 3 × K 3 Hermitian matrix, C(6)x ,
called the hexacovariance. With an appropriate storage of thetensor entries, we have
C(6)x = H�3 ∆(6) H�3H (10)
Because C(6)x is Hermitian, ∃V unitary, such that
(C(6)x )1/2 = H�3 (∆(6))1/2 V (11)
Idea: Use redundancy existing between blocks of (C(6)x )1/2.
Pierre Comon Tensor problems in EE 99
![Page 228: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/228.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Using the invariance to estimate V
1 Cut the K 3×P matrix (C(6)x )1/2 into K blocks of size K 2×P.
Each of these blocks, Γ[n], satisfies:
Γ[n] = (H�HH)D[n] (∆(6))1/2 V
where D[n] is the P × P diagonal matrix containing the nthrow of H, 1 ≤ n ≤ K .Hence matrices Γ[n] share the same common right singularspace
2 Compute the joint EVD of the K (K − 1) matrices
Θ[m, n]def= Γ[m]−Γ[n]
as: Θ[m, n] = VΛ[m, n]VH.
Pierre Comon Tensor problems in EE 100
![Page 229: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/229.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Using the invariance to estimate V
1 Cut the K 3×P matrix (C(6)x )1/2 into K blocks of size K 2×P.
Each of these blocks, Γ[n], satisfies:
Γ[n] = (H�HH)D[n] (∆(6))1/2 V
where D[n] is the P × P diagonal matrix containing the nthrow of H, 1 ≤ n ≤ K .Hence matrices Γ[n] share the same common right singularspace
2 Compute the joint EVD of the K (K − 1) matrices
Θ[m, n]def= Γ[m]−Γ[n]
as: Θ[m, n] = VΛ[m, n]VH.
Pierre Comon Tensor problems in EE 100
![Page 230: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/230.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Estimation of H
Matrices Λ[m, n] cannot be used directly because (∆(6))1/2 isunknown. But we use V to obtain the estimate of H�3 up to ascale factor:
H�3 = (C(6)x )1/2 V (12)
One possibility to get H from H�3 is as follows:
3 Build K 2 matrices Ξ[m] of size K × P form rows of H�3
4 From Ξ[m] find H and diagonal matrices D[m], in the LSsense:
Ξ[m]D[m] ≈ H, 1 ≤ m ≤ K 2
Pierre Comon Tensor problems in EE 101
![Page 231: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/231.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Estimation of H
Matrices Λ[m, n] cannot be used directly because (∆(6))1/2 isunknown. But we use V to obtain the estimate of H�3 up to ascale factor:
H�3 = (C(6)x )1/2 V (12)
One possibility to get H from H�3 is as follows:
3 Build K 2 matrices Ξ[m] of size K × P form rows of H�3
4 From Ξ[m] find H and diagonal matrices D[m], in the LSsense:
Ξ[m]D[m] ≈ H, 1 ≤ m ≤ K 2
Pierre Comon Tensor problems in EE 101
![Page 232: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/232.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Estimation of H
Matrices Λ[m, n] cannot be used directly because (∆(6))1/2 isunknown. But we use V to obtain the estimate of H�3 up to ascale factor:
H�3 = (C(6)x )1/2 V (12)
One possibility to get H from H�3 is as follows:
3 Build K 2 matrices Ξ[m] of size K × P form rows of H�3
4 From Ξ[m] find H and diagonal matrices D[m], in the LSsense:
Ξ[m]D[m] ≈ H, 1 ≤ m ≤ K 2
Pierre Comon Tensor problems in EE 101
![Page 233: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/233.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Estimation of H
Matrices Λ[m, n] cannot be used directly because (∆(6))1/2 isunknown. But we use V to obtain the estimate of H�3 up to ascale factor:
H�3 = (C(6)x )1/2 V (12)
One possibility to get H from H�3 is as follows:
3 Build K 2 matrices Ξ[m] of size K × P form rows of H�3
4 From Ξ[m] find H and diagonal matrices D[m], in the LSsense:
Ξ[m]D[m] ≈ H, 1 ≤ m ≤ K 2
Pierre Comon Tensor problems in EE 101
![Page 234: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/234.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Estimation of H (details) s
Stationary values of criterion∑M
m=1 ‖ΞmDm −H‖2F , Mdef= K 2,
yield the solution below
Obtain vectors dpdef= [D1 (p, p) , D2 (p, p) , · · · DM (p, p)]T,
by solving the linear systems:
Fp dp = 0
where matrices Fp are defined as
Fp (m1,m2) =
{(M − 1)
{ΞH
m1Ξm1
}(p, p) if m1 = m2
−{ΞH
m1Ξm2
}(p, p) otherwise
Deduce the estimate H = 1M
∑Mm=1 ΞmDm
Pierre Comon Tensor problems in EE 102
![Page 235: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/235.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Conditions of identifiability of BIOME(2r) s
Source cumulants of order 2r > 4 are nonzero and have thesame sign
Columns vectors of mixing matrix H are not collinear
Matrix H�(r−1) is full column rank.
This last condition implies that tensor rank must be at mostK r−1 (e.g. P ≤ K 2 for order 2r = 6).
Pierre Comon Tensor problems in EE 103
![Page 236: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/236.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
FOOBI algorithms
Again same problem: Given a K 2 × P matrix H�2, find a realorthogonal matrix Q such that the P columns of H�2 Q are of theform h[p]⊗ h[p]∗
FOOBI: use the K 4 determinantal equations characterizingrank-1 matrices h[p]h[p]H of the form:φ(X,Y)ijk` = xijy`k − xiky`j + yijx`k − yikx`j
FOOBI2: use the K 2 equations of the form:Φ(X,Y) = XY + Y X + trace{X}Y + trace{Y}Xwhere matrices X and Y are K × K Hermitean.
Pierre Comon Tensor problems in EE 104
![Page 237: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/237.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
FOOBI algorithms
Again same problem: Given a K 2 × P matrix H�2, find a realorthogonal matrix Q such that the P columns of H�2 Q are of theform h[p]⊗ h[p]∗
FOOBI: use the K 4 determinantal equations characterizingrank-1 matrices h[p]h[p]H of the form:φ(X,Y)ijk` = xijy`k − xiky`j + yijx`k − yikx`j
FOOBI2: use the K 2 equations of the form:Φ(X,Y) = XY + Y X + trace{X}Y + trace{Y}Xwhere matrices X and Y are K × K Hermitean.
Pierre Comon Tensor problems in EE 104
![Page 238: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/238.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
FOOBI algorithms
Again same problem: Given a K 2 × P matrix H�2, find a realorthogonal matrix Q such that the P columns of H�2 Q are of theform h[p]⊗ h[p]∗
FOOBI: use the K 4 determinantal equations characterizingrank-1 matrices h[p]h[p]H of the form:φ(X,Y)ijk` = xijy`k − xiky`j + yijx`k − yikx`j
FOOBI2: use the K 2 equations of the form:Φ(X,Y) = XY + Y X + trace{X}Y + trace{Y}Xwhere matrices X and Y are K × K Hermitean.
Pierre Comon Tensor problems in EE 104
![Page 239: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/239.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
FOOBI s
1 Normalize the columns of the K 2 × P matrix H�2 such that
matrices H[r ]def= UnvecK
(h�2[r ]
)are Hermitean
2 Compute the K 2(K − 1)2 × P(P − 1)/2 matrix P defined byφ(H[r ],H[s]), 1 ≤ r ≤ s ≤ P.
3 Compute the P weakest right singular vectors of P, Unvecthem and store them in P matrices W[r ]
4 Jointly diagonalize W[r ] by a real orthogonal matrix Q
5 Then compute Fdef= (C
(4)x )1/2∆Q and deduce h[r ] as the
dominant left singular vectors of Unvec (f[r ]).
Pierre Comon Tensor problems in EE 105
![Page 240: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/240.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
FOOBI2 s
1 Normalize the columns of the K 2 × P matrix H�2 such that
matrices H[r ]def= UnvecK
(h�2[r ]
)are Hermitean
2 Compute the K (K + 1)/2 Hermitean matrix B[r , s] of sizeP × P defined by:
Φ(H[r ],H[s])|ijdef= B[i , j ]|rs
3 Jointly cancel diagonal entries of matrices B[i , j ] by a realcongruent orthogonal transform Q
4 Then compute Fdef= (C
(4)x )1/2∆Q and deduce h[r ] as the
dominant left singular vectors of Unvec (f[r ]).
NB: Better bound than FOOBI and BIOME(4), but iterativealgorithm sensitive to initialization
Pierre Comon Tensor problems in EE 106
![Page 241: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/241.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Algorithms based on characteristic functions
Fit with a model of exact rank
1 Back to the core equation (3):
Ψx(u) =∑p
Ψsp
(∑q
uq Aqp
)
2 Goal: Find a matrix H such that the K−variate functionΨx(u) decomposes into a sum of P univariate functions
ψpdef= Ψsp .
3 Idea: Fit both sides on a grid of values u[`] ∈ G
Pierre Comon Tensor problems in EE 107
![Page 242: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/242.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Algorithms based on characteristic functions
Fit with a model of exact rank
1 Back to the core equation (3):
Ψx(u) =∑p
Ψsp
(∑q
uq Aqp
)
2 Goal: Find a matrix H such that the K−variate functionΨx(u) decomposes into a sum of P univariate functions
ψpdef= Ψsp .
3 Idea: Fit both sides on a grid of values u[`] ∈ G
Pierre Comon Tensor problems in EE 107
![Page 243: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/243.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Algorithms based on characteristic functions
Fit with a model of exact rank
1 Back to the core equation (3):
Ψx(u) =∑p
Ψsp
(∑q
uq Aqp
)
2 Goal: Find a matrix H such that the K−variate functionΨx(u) decomposes into a sum of P univariate functions
ψpdef= Ψsp .
3 Idea: Fit both sides on a grid of values u[`] ∈ G
Pierre Comon Tensor problems in EE 107
![Page 244: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/244.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Equations derived from the CAF
Assumption: functions ψp, 1 ≤ p ≤ P admit finite derivativesup to order r in a neighborhood of the origin, containing G.
Then, Taking r = 3 as a working example:
∂3Ψx
∂ui∂uj∂uk(u) =
P∑p=1
Hip Hjp Hkp ψ(3)p (
K∑q=1
uq Hqp)
If L > 1 point in grid G, then yields another mode in tensor
Pierre Comon Tensor problems in EE 108
![Page 245: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/245.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Equations derived from the CAF
Assumption: functions ψp, 1 ≤ p ≤ P admit finite derivativesup to order r in a neighborhood of the origin, containing G.Then, Taking r = 3 as a working example:
∂3Ψx
∂ui∂uj∂uk(u) =
P∑p=1
Hip Hjp Hkp ψ(3)p (
K∑q=1
uq Hqp)
If L > 1 point in grid G, then yields another mode in tensor
Pierre Comon Tensor problems in EE 108
![Page 246: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/246.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Equations derived from the CAF
Assumption: functions ψp, 1 ≤ p ≤ P admit finite derivativesup to order r in a neighborhood of the origin, containing G.Then, Taking r = 3 as a working example:
∂3Ψx
∂ui∂uj∂uk(u) =
P∑p=1
Hip Hjp Hkp ψ(3)p (
K∑q=1
uq Hqp)
If L > 1 point in grid G, then yields another mode in tensor
Pierre Comon Tensor problems in EE 108
![Page 247: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/247.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Putting the problem in tensor form
A decomposition into a sum of rank-1 terms:
Tijk` =∑p
Hip Hjp Hkp B`p
or equivalently
T =∑p
h(p) ⊗h(p) ⊗h(p) ⊗b(p)
Tensor T is K × K × K × L,symmetric in all modes except the last.
Pierre Comon Tensor problems in EE 109
![Page 248: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/248.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Putting the problem in tensor form
A decomposition into a sum of rank-1 terms:
Tijk` =∑p
Hip Hjp Hkp B`p
or equivalently
T =∑p
h(p) ⊗h(p) ⊗h(p) ⊗b(p)
Tensor T is K × K × K × L,symmetric in all modes except the last.
Pierre Comon Tensor problems in EE 109
![Page 249: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/249.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Joint use of different derivative orders
Example
Derivatives of order 3:
T(3)ijk` =
∑p
Hip Hjp Hkp B`p
Derivatives of order 4:
T(4)ijkm` =
∑p
Hip Hjp Hkp Hmp C`p
Derivatives of orders 3 and 4:
Tijk`[m] =∑p
Hip Hjp Hkp D`p[m]
with D`p[m] = Hmp C`p and D`p[0] = B`p.
Pierre Comon Tensor problems in EE 110
![Page 250: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/250.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Joint use of different derivative orders
Example
Derivatives of order 3:
T(3)ijk` =
∑p
Hip Hjp Hkp B`p
Derivatives of order 4:
T(4)ijkm` =
∑p
Hip Hjp Hkp Hmp C`p
Derivatives of orders 3 and 4:
Tijk`[m] =∑p
Hip Hjp Hkp D`p[m]
with D`p[m] = Hmp C`p and D`p[0] = B`p.
Pierre Comon Tensor problems in EE 110
![Page 251: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/251.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Joint use of different derivative orders
Example
Derivatives of order 3:
T(3)ijk` =
∑p
Hip Hjp Hkp B`p
Derivatives of order 4:
T(4)ijkm` =
∑p
Hip Hjp Hkp Hmp C`p
Derivatives of orders 3 and 4:
Tijk`[m] =∑p
Hip Hjp Hkp D`p[m]
with D`p[m] = Hmp C`p and D`p[0] = B`p.
Pierre Comon Tensor problems in EE 110
![Page 252: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/252.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Problem P13
Results for tensors enjoying partial symmetries
e.g., symmetric in the 3 first modes and not not in others
e.g. tensors with symmetries in some modes and Hermiteansymmetries in others
Generic/typical ranks?
Existence/well-poseness?
Uniqueness?
Pierre Comon Tensor problems in EE 111
![Page 253: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/253.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Problem P18
What is best to do when several symmetric tensors, possibly ofdifferent orders, are supposed to be decomposed with same matrixH?
Pierre Comon Tensor problems in EE 112
![Page 254: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/254.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Iterative algorithms
Many practitioners execute more or less brute force minimizationsof ||T−
∑rp=1 up
⊗ vp⊗ . . . ⊗wp||2
Gradient with fixed or variable (ad-hoc) stepsize
Alternate Least Squares (ALS)
Levenberg-Marquardt
Newton
Conjugate gradient
...
Remarks
Hessian is generally huge, but sparse
Problem of local minima: ELS variants for all of the above
Pierre Comon Tensor problems in EE 113
![Page 255: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/255.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
What we have seen so far
In dimension 2, we are able to handle decompositions ofsymmetric tensors algebraically
Suitable to extend this to higher dimensions (now partly donefor sub-generic cases)
Work remains for generic case or above
A lot to do for unsymmetric tensors
Problem of tensors enjoying some symmetries
Collection of tensors sharing common terms is theirdecompositions
Pierre Comon Tensor problems in EE 114
![Page 256: Tensor problems in Engineering - Gipsa-labpierre.comon/FichiersPdf/...Warning: tensor should not have proportional matrix slices (degeneration) Pierre Comon Tensor problems in EE 11](https://reader031.fdocuments.in/reader031/viewer/2022013020/5e5df0997fec771fb407a858/html5/thumbnails/256.jpg)
Context Over. Orthog. Invertible Under. Binary Ranks Biome Foobi CAF Iterative End
Summary of the lectures
We want to know when the decomposition of a tensor isunique
In the latter cases, we want to be able to compute thedecomposition
We have suboptimal ways of doing it in some cases
Pierre Comon Tensor problems in EE 115