G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed...
-
Upload
beatrix-west -
Category
Documents
-
view
214 -
download
1
Transcript of G lobal O ptimality of the S uccessive M ax B et A lgorithm USC ENITIAA de NANTES France Mohamed...
Global Optimality of the Successive MaxBet Algorithm
USC ENITIAA de NANTES
FranceMohamed HANAFI
and
Jos M.F. TEN BERGE
Department of psychology University of Groningen
The Netherlands
Global Optimality of the Successive MaxBet Algorithm
Summary. 1. The Successive MaxBet Problem (SMP). 2. The MaxBet Algorithm. 3. Global Optimality : Motivation/Problems. 4. Conclusions and Open questions.
1. The Successive MaxBet Problem (S.M.P)
jk pp ,kjA
kjAA Kjk ,,2,1,
KK , Blocks Matrix
s.p.s.d
KKKK
K
K
AAA
AAA
AAA
A
21
22221
11211
Auuu '
order 1
K
jkjkjk
1,
' uAu
Maximize
Subject to
11
''
K
kkkuuuu
1. The Successive MaxBet Problem (S.M.P)
Kk ,,2,1 1' kkuu
order s
Kk ,,2,1
Auuuuu '21 ,...,, K
Maximize
Subject to1' kkuu
0uU kk'{
121 skkkk uuuU Kk ,,2,1
1. The Successive MaxBet Problem (S.M.P)
2. The Successive MaxBet AlgorithmTen Berge (1986,1988)
Order 1
K
jjkjk
1
uAv
1. Take arbitrary initial unit length vectors ku
2. Compute :
3. rescale vk to unit length, and set uk= vk
4. Repeat steps 2 and 3 till convergence
Kk ,,2,1
Kk ,,2,1
Kk ,,2,1
Order s
2. The Successive MaxBet AlgorithmTen Berge (1986,1988)
''jjpkjkkpkj jk
UUIAUUIA
K
jjkjk
1
uAv
1. Take arbitrary initial unit length vectors ku
2. Compute :
3. rescale vk to unit length, and set uk= vk
4. Repeat steps 2 and 3 till convergence
Kk ,,2,1
Kk ,,2,1
Kk ,,2,1
Property 1 : Convergence of the MaxBet Algorithm
u
u
Property 2 : Necessary Condition of Convergence
1' kkuu
Kk ,,2,1
KKKmmmm
m
m
u
u
u
u
u
u
AAA
AAA
AAA
22
11
2
1
21
22221
11211
K ....21u
3. Motivation and results
1. MaxBet Algorithm depends on the starting vector
2. MaxBet algorithm does not guarantee the computation of the global solution of SMP
43 23 -13 0 -7
23 31 10 1 0
-13 10 64 -19 -2
0 1 -19 24 18
-7 0 -2 18 58
A
11A 12A
22A21A
3. Motivation and results : an example
2K 21 p 32 p
42185 64023
(u)= 10621Function value{ 3846.7 5978.4
(v)= 9825.1
0.67 0.36 0.20 0.53 0.30
{Starting Vector *u
0.64 0.31 0.64 0.24 0.10
*v
0.69 0.72 0.58 -0.43 -0.68
v{Solution Vector
0.94 0.31 -0.92 0.35 0.11
u
3. Motivation and results: Two Questions
Q1. How can we know that the solution computed by the Maxbet algorithm is global or not ?
Q2. When the solution is not global, how can we reach using this solution the global solution ?
K
K
pKKKKK
Kp
Kp
IAAA
AIAA
AAIA
A
21
222221
112111
,...,,2
1
21
3. Motivation and Results : Proceeding
Global solution of SMP
Spectral properties (eigenvalues and eigenvectors) of K ,...,, 21
A
When, for a solution, {u, 1,
2, …,
K} satisfies
KKKKKKK
K
K
u
u
u
u
u
u
AAA
AAA
AAA
22
11
2
1
21
22221
11211
RESULT 1
then u is the global solution of SMP.
is negative semidefinite, K ,...,, 21A
Result 1
we have :vAv= vAv 1v1
v1
2v2v2
… KvK
vK
= (v) 1
2 …
K,
hence(v)1+2+…+K= (u)
the matrix A is negative semidefinite,
ELEMENTS OF PROOF (Result 1)
To what extent the previous sufficient condition (Result 1):
is necessary ?
K ,...,, 21A(matrix is negative semi definite)
3. Motivation and Results
matrix blocks 2,2 is A
RESULT 2
22
11
2
1
2221
1211
u
u
u
u
AA
AA
When u is the global maximum of S.M.P it verifies :
2
1
2122221
12111
,p
p
IAA
AIAA
21 ,Athen matrix is negative semi definite
Result 2
Suppose has a positive eigenvalue 21 ,A
021 , wwA
1. w is block-normed vector
2. w is not block-normed vector
2.1. w is not block orthogonal to u
2.2. w is block orthogonal to u
ELEMENTS OF PROOF (Result 2)
1. w is block-normed vector
021 ,
' wAw
uAww 21'
w is better solution than u
2. w is not block-normed vector2.1. w is not block orthogonal to u
222
22211
222112
)'(8)''(
)''(
wuwwww
wwww
222
22211
2222
)'(8)''(
)'(16
wuwwww
wu
wuv
v is better solution than u
*wuv
v is better solution than u
211
122
)'(
)'(
dud
dudq
0'
0'
22
11
ud
ud
qww t*
w is not block-normed vector2.2. w is block orthogonal to u
RESULT 2
Result 3
positive are elements allwith
matrix blocks , is KKA
K
kkpp
1
ghaA phg ....,2,1,
then matrix is negative semi definite K ,...,, 21A
When u is the global maximum of S.M.P
positive are elements allwith
matrix blocks , is KKA
Suppose has a positive eigenvalue
0
K ,...,, 21A
wwA K,...,, 21
ELEMENTS OF PROOF (Result 3)
p
hgghhg
p
hgghhg auuauuu
,,
||
u has all elements of the same sign
ELEMENTS OF PROOF (Result 3)
k
K
kkkK
1
'',...,,
'
21wwAwwwAw
w has all elements of the same sign
2K
sign same theof elements allnot with
matrix blocks , is KKA
The sufficient condition (Result 1) :
is not necessary
K ,...,, 21A(matrix is negative semi definite)
Result 4
45 -20 5 6 16 3
-20 77 -20 -25 -8 -21
5 -20 74 47 18 -32
6 -25 47 54 7 -11
16 -8 18 7 21 -7
3 -21 -32 -11 -7 70
ELEMENTS OF PROOF (Result 4)
A
3K
21 p
22 p
23 p
0.49-0.87 0.80 0.59 0.56-0.82
(u) =378.96
Random research with 10.000.000 starting vectors
=0.48 u =
ELEMENTS OF PROOF (Result 4)
- Possible Application in statistics :
Multivariate Methods (Analysis of K sets of data )
4. General Conclusions
1. Generalized canonical correlationAnalysis: Horst (1961)
3. Soft Modeling Approach :Estimation of latent variables under mode B
Wold (1984); Hanafi (2001)
2. Rotation methods : MaxDiff, MaxBet, generalized Procrustes Analysis
Gower(1975); Van de Geer(1984);Ten Berge (1986,1988)
-
- Necessary condition for the case K=3 when matrix A has not all
elements of the same sign?
4. Perspective and Little Open Question
2X
0
0
2'
1'
aXa
aXa
nn,
??0a
nn,1X
uf
Ku
u
u
u2
11k
'kuu
K,...,,k 21
Motivation: Illustration 1 MaxBet Algorithm depends on the starting vector
The Successive MaxBet Problem (S.M.P)and
Multivariate Methods
kpn, K,,,k 21kX
K
kkpn
1
, KXXXX 21
Some multivarite methods
Generalized canonical correlation methods
Rotation methods(Agreement methods)
SOFT MODELING APPRAOCH(Approch)
n
XXA
'
kjAA
Rotation methods
njk
kj
XXA
'
S M P = MaxBet method Van de Geer (1984) Ten Berge (1986,1988)
Kjk ,,2,1,
S M P = MaxBet method Van de Geer (1984) Ten Berge (1986,1988)
Kjk ,,2,1,
S M P = MaxDiff method Van de Geer (1984) Ten Berge (1986,1988)
Kjk ,,2,1,
jk
jkn
jk
kj
0
' XXA
S M P = MaxBet method Van de Geer (1984) Ten Berge (1986,1988)
Kjk ,,2,1, Kjk ,,2,1,
jkn
K
jkn
jk
jk
kj XX
XX
A '
'
2
S M P = Generalized Procrustes Analysis Gower(1975), Ten Berge (1986,1988)
kX 'kkkk WPX
Generalized canonical correlation methods
SVD
kjAA n
jkkj
PPA
'
SMP = Horst method(1961)
kjAA njk
kjkj
PPA
'
S M P = Soft Modeling Appraoch (Hanafi 2001)
1,1,0 jkkj
Mode B soft modeling approach
uuAuuuIAu ''' mm mK Auu '
uIAu m' MaximizeK,...,,k 211 Subject to k
'kuu
1k'kuu K,...,,k 21
Auu ' MaximizeK,...,,k 211 Subject to k
'kuu
Auuuuu '21 ,...,, K
Kk ,,2,1 1' kkuu
Maximize
Subject to
KKKmmmm
m
m
u
u
u
u
u
u
AAA
AAA
AAA
22
11
2
1
21
22221
11211
0u
Multivariate Eigenvalue Problem Watterson and Chu(1993)
kppp ....2 solutions ofnumber 21
1' kkuuKk ,,2,1
KKKmmmm
m
m
u
u
u
u
u
u
AAA
AAA
AAA
22
11
2
1
21
22221
11211