Post on 14-Dec-2015
بنام خدا
1
An Introduction to multi-way analysis
Mohsen Kompany-ZarehIASBS, Nov 1-3, 2010
2
Session one
3
The main source:
4
Kronecker productKhatri-Rao product
Multi-way dataMatricizing the dataInteraction triadGPARAFACPanel performance
Matricizing and subarrayRankDimensionality vectorRank-deficiency in three-way arrays
Tucker3 rotational freedomUnique solution
Tucker2 modelTucker1 model
>> A=[2 3 4; 2 3 4]>> B=[3 4; 3 5]
>> krnAB=[A(1,1)*B A(1,2)*B A(1,3)*B ; A(2,1)*B A(2,2)*B A(2,3)*B]
krnAB =
6 8 9 12 12 16 6 10 9 15 12 20 6 8 9 12 12 16 6 10 9 15 12 20
>>
kronecker product (A B)
5
>> A=[2 3 4; 2 3 4]>>B=[3 4; 3 5]
>> p=kron(A,B)
>>p= 6 8 9 12 12 16 6 10 9 15 12 20 6 8 9 12 12 16 6 10 9 15 12 20
>>
6
All columns in A see all columns in B.
kronecker product
>> A=[2 3 4; 2 3 4]>>C=[3 4 5; 3 5 2]>>krnAC=[kron(A(:, ),C(:, ))... column 1 kron(A(:,1),C(:,2))... column 2 kron(A(:,1),C(:,3))... .. kron(A(:,2),C(:,1))... .. kron(A(:, ),C(:, ))... .. kron(A(:,2),C(:,3))... kron(A(:,3),C(:,1))... kron(A(:,3),C(:,2))... kron(A(:, ),C(:, ))] column 9
krnAC =
6 8 10 9 12 15 12 16 20 6 10 4 9 15 6 12 20 8 6 8 10 9 12 15 12 16 20 6 10 4 9 15 6 12 20 8
>>
Khatri-Rao Product
kronecker product
7
1 1
2 2
3 3
>> A=[2 3 4; 2 3 4]>>C=[3 4 5; 3 5 2]
krnAC =
6 8 10 9 12 15 12 16 20 6 10 4 9 15 6 12 20 8 6 8 10 9 12 15 12 16 20 6 10 4 9 15 6 12 20 8
kronecker product
8
vec(a1 b1) vec(a2 b2) vec(a3 b3)
vec(a1 b2)
vec(a1 b3) vec(a2 b3) vec(a2 b1)
vec(a3 b1)
vec(a3 b2)
Interaction terms
>> A=[2 3 4; 2 3 4]>> B=[3 4 5; 3 5 2]
khtrAB=
6 12 20 6 15 8 6 12 20 6 15 8
>>
9
No of columns in A should be the same as the number of columns in B.
Khatri-Rao Product
10
Kronecker productKhatri-Rao product
Multi-way dataMatricizing the dataInteraction triadGPARAFACPanel performance
Matricizing and subarrayRankDimensionality vectorRank-deficiency in three-way arrays
Tucker3 rotational freedomUnique solution
Tucker2 modelTucker1 model
(generalization of matrix algebra) A zero-order tensor: a scalar;a first-order tensor : a vector; a second-order tensor (a matrix) for a sample => 3 way data, for analysisa third-order tensor (three-way array) for a sample => 4 way data, for analysis a fourth-order tensor : a four-way array and so on.
11
Multi-way Data
12
45 50 55 600
0.5
1
1.5
2Elution prof
450 500 550 6000
0.5
1
1.5
2Vis spectrum
24
6
450500
550600
0
1
2
3
Spect.s at diff ret times
24
645
5055
60
0
12
3
Elut prof.s at diff wavel.s
2 4 6
2
4
6
4550
5560
450500
550600
0
1
2
3
One component, HPLC-DAD
a1 b1
13
45 50 55 60
450500
550600
0
10
20
45 50 55 60
450500
550600
0
10
20
45 50 55 60
450500
550600
0
10
20
45 50 55 60
450500
550600
0
10
20
45 50 55 60
450500
550600
0
10
20
45 50 55 60
450500
550600
0
10
20
45 50 55 60
450500
550600
0
10
20
45 50 55 60
450500
550600
0
10
20
45 50 55 60
450500
550600
0
10
20
45 50 55 60
450500
550600
0
10
20
45 50 55 60
450500
550600
0
10
20
45 50 55 60
450500
550600
0
10
20
One component, HPLC-DAD, different concentrations (elution profile)
Only the intensities are changed...These 9 matrices form a TRIAD, the simplest trilinear data
14
>> a1'
0.0033 0.0971 0.8131 1.9506 1.3406 0.2640 0.0149
>> b1'
0.0222 1.7650 0.4060 0.8826 0.0111 0.0000 0.0000
>> c1'
1 2 3 4 5 6 7 8 9 10 11 12
A triad : XA cube of data 12x7x7 3rd order data for one sample
Obtained from Tensor product of 3 vectors
a1 b1 c1
% A triad by outer product % X111=a1 b1 c1 ...for l=1:length(a1) for m=1:length(b1) for n=1:length(c1) disp([l m n]) Xtriad(l,m,n)=a1(l)*b1(m)*c1(n); end end end X=Xtriad;....
200 300 400 500 600 7000
0.5
1Ex spectrum
200 300 400 500 600 7000
0.5
1em spectrum
0 2 4 6 8 100
0.5
1Concentration profile
a1
b1
c1
Matricizing the data
16
X111= Unfold3D(X111, 1) (in three directions) The first chemical component
17
...and for the 2nd and the next chemical components:
X111 = a1 b1 c1
X222 = a2 b2 c2
X333 = a3 b3 c3
Each component in a separate triad (no interaction)
+
+
X = X111 + X222 + X333 Trilinear
PARAFAC
18
X111 = a1 b1 c1
X222 = a2 b2 c2
X121 = a1 b2 c1
+
+
X = X111 + X222 + X121 NonTrilinear!!
Tucker
In the presence of Interaction :Interaction triad
19
How many interaction triads?
For two components in three modes:
X111 = a1 b1 c1
X112 = a1 b1 c2
X121 = a1 b2 c1
X122 = a1 b2 c2
X211 = a2 b1 c1
X212 = a2 b1 c2
X221 = a2 b2 c1
X222 = a2 b2 c2
G(111)= 2
G(112)= 0
G(121)= 1
G(122)= 0
G(211)= 0
G(212)= 0
G(221)= 0
G(222)=-36 possible interaction triads 1 interaction triads
G
A(11x2)
G(2x2x2)
C(3x2)
B(1002)G(111)= 2
G(222)=-3
G(121)= 1
21
For three components in three modes:
(3 3 3) – 3 = 24 possible interactions
A(15x4)
G(?x?x?)
C(20x2)
B(1003)
How many G elements?
23
% Tucker3 outer productG=rand(4,3,2);
for p=1:size(G,1) for q=1:size(G,2) for r=1:size(G,3) for i=1:size(A,2) for k=1:size(C,2) for m=1:size(B,2) disp([p q r i j k]) Xtriad(l,m,n)=A(i,l)*B(j,m)*C(k,n)*G(i,j,k); end end end X=X+Xtriad; end endend
One triad
24
What about Tucker4?
25
26
% PARAFAC outer productG=zeros(3,3,3);G(1,1,1)=1;G(2,2,2)=1;G(3,3,3)=1;for p=1:size(G,1) for q=1:size(G,2) for r=1:size(G,3) for i=1:size(A,2) for k=1:size(C,2) for m=1:size(B,2) disp([p q r i j k]) Xtriad(l,m,n)=A(i,l)*B(j,m)*C(k,n)*G(i,j,k); end end end X=X+Xtriad; end endend
One triad
A(15x3)
C(20x3)
B(1003)
PARAFACSimple interpretation
Monitoring panel performance within and between experiments by multi-way models
Rosaria Romano and Mohsen Kompany-Zareh
Copenhagen Univ, 2007
Organic Milk of high Quality Sensory studies 2007- University of Copenhagen
- Spring experiment (May, week 21 & 22)- Autumn experiment (September, week 36 & 37)
Two different experiments were conducted in 2007:
The objective is to establish knowledge about production of high quality organic milk with a composition and flavour different from conventionally produced milk.
Spring experiment dataData description:
• 7 varieties of milk with respect to: - 2 cow races: Holstein-Fries (HF), Jersey (JE); - 7 farms: WB, EMC, UGJ, JP, HM, OA, KI.
• panel: - 9 assessors, 2 sessions (focus on the second!), 3 replicates for each session.
• 12 descriptors: odor (green), appearance (yellow), flavor (creamy, boiled-milk, sweet,
bitter, metallic, sourness, stald-feed) after taste (astringent0, fatness, astringent20).
• measurement scale: continuous scale anchored at 0 and 15.
Parafac on the spring experiment(1)Model: Parafac with two components (27.9% ExpVar), on data averaged across the samples mode
-30 -20 -10 0 10 20 30-25
-20
-15
-10
-5
0
5
10
15
20
1
2
EMC-HF-2 HM-HF-1 HM-HF-2 HM-HF-3
JP-JE-2
JP-JE-3
KI-HF-1
KI-HF-2
KI-HF-3
OA-JE-1
OA-JE-2 OA-JE-3
UGJ-JE-1
UGJ-JE-2
UGJ-JE-3
WB-JE-1
WB-JE-2
WB-JE-3
Decluttered
2 4 6 8 10 12-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Variable1,
2
OGreen
APYellow
FCreamy
FBoiledMilk
FSweet
FBitter
FMetallic
FSourness
FStaldFeedRelat
ATAstringent0 ATFatness
ATAstringent20
HF
JE
high reproducibility of the replicates in both groups;
big variation in the JE group: - WB is the less yellow JE milk;
- UGJ seems have something in common with HF group.
Parafac on the spring experiment(2)Model: Parafac with two components (27.9% ExpVar), on data averaged across the samples mode
0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6
0.2
0.25
0.3
0.35
0.4
0.45
0.5
0.55
0.6
0.65
1
2
Arne Julie
Katrine
Line
Lisbeth
Magda
Maria
Nina
Sandra
Arne Julie Katrine Line Lisbeth Magda Maria Nina Sandra 0
10
20
30
40
50
60
70
80
90
100Assessor Performance
ijk
F
fkfjfifijk ecbaxParafac
1
: 100*
max
1
2
1
2
1
2
1
2
F
f
jf
F
f
jf
F
f
jf
F
f
jf
e
b
e
b
BROMA
Best Reliability on Multi-way Assessment (Bro and Romano, 2008)
33
Kronecker productKhatri-Rao product
Multi-way dataMatricizing the dataInteraction triadGPARAFACPanel performance
Matricizing and subarrayRankDimensionality vectorRank-deficiency in three-way arrays
Tucker3 rotational freedomUnique solution
Tucker2 modelTucker1 model
A has full rank (if and only if ) : r(A) = min(I,J).
If r(A )= R, [Schott 1997]
Þ A = t1p1 + ·· ·+tRpR
R rank one matrices (tr pr , components).
34
Bases are not unique: rotational freedom intensity (or scale) indeterminacy. sign indeterminacy.
Rank
If X (I × J ) : generated with I × J random numbers =>probability of (X has less than full rank) =0 .. => measured data sets in chemistry: always full rank (mathematical rank) <= measurment noise Ex: UV spectra (100 wavelengths) ; ten different samples, each: same absorbing species at different concentrations.
Þ X (10 ×100) if Lambert–Beer law holds : rank one.
35
+ measurement errors => mathem rank = ten.
X = cs’ + E = Xhat + E (model of X) vector c : concns, s : pure UV spectrum of the abs species E : noise part.
1. systematic variation 2. Noise (undesirable)
Þpseudo-rank =Math rank (Xhat) = one < math rank (X).
‘chemical rank’ : number of chemical sources of variation in data.
36
Rank deficiency pseudo-rank < chemical rank. ( linear relations in or restrictions on the data).
Ex;X = c1s1 + c2s2 + c3s3 + E , s1 = s2 (linear relation)
=> X = (c1 + c2)s1 + c3s3 + E
Chem rank (X)= 3
pseudo-rank (X)= 2, rank deficient
37
38
A randomly generated 2 × 2 × 2 array to have a rank lower than three : a positive probability [Kruskal 1989]. a probability of 0.79 of obtaining a rank two array a probability of 0.21 of obtaining a rank three . probability of obtaining rank one or lower is zero.
generalized to : 2 × n × n arrays [Ten Berge 1991].
39
2 × 2 × 2 array:the maximum rank: three typical rank: {2, 3}, (almost all individual rank: very hard to establish.
Three way rank : important in second-order calibration and curve resolution. for degrees of freedom ?? for significance testing.
40
X(4 × 3 × 2)
Boldfaces : in the foremost frontal slice
41
Matricizing and Sub-arraysMatricizing
42
sub-arrays
Row-rank, column-rank, tube-rank
two-way X : rank(X) = rank(X’) column rank= row rank
:not hold for three-way arrays.
three-way array X(I × J × K) : matricized in three different ways (i) row-wise, giving X(J ×IK), a two-way array(ii) column-wise, giving X(I×JK) ,(iii) tube-wise, giving X(K×IJ). and three more with the same ranks,not mentioned
ranks of the arrays X(J×IK),X(I×JK) and X(K×IJ), = (P, Q, R): dimensionality vector of X.
43
Dimensionality vector
44
P, Q and R: not necessarily equal. In contrast with two-way P = Q = r(X).
dimensionality vector (P, Q, R) of a three-way array X with rank S Obeys certain inequalities [Kruskal 1989]:
(i) P ≤ QR ; Q ≤ PR; R ≤ PQ (ii) max(P, Q, R) ≤ S ≤ min(PQ, QR, PR)
These arrays have rank 4, 3, and 2.Dimensionality vector is [4 3 2] P, Q and R can be unequal.45
Three matricized forms:
Pseudo-rank, rank deficiency and chemical sources of variation
pseudo-rank of three-way arrays: straight generalization of the two-way definit.
X = Xhat + E E : array of residuals.
pseudo-rank of X = minimum # PARAFAC components necessary to exactly fit Xhat.
46
Spectrophometric acid-base titration of mixtures of three weak mono-protic acids (or Flow injection analysis + pH gradient) HA2 H+ + A2- HA3 H+ + A3-
HA4 H+ + A4- six components
models of separate titration of the three analytes (HA2, HA3, HA4), XHA2 = ca,2sa,2 + cb,2sb,2 + EHA2
XHA3 = ca,3sa,3 + cb,3sb,3 + EHA3
XHA4 = ca,4sa,4 + cb,4sb,4 + EHA4
10 samples, 15 titn points, and 20 wavel.s => X(10×15×20),47
Rank-deficiency in three-way arrays
X = Xhat + E ca,2 + cb,2 = α(ca,3 + cb,3) = β(ca,4 + cb,4)Þonly four independently varying concn profiles. Pseudo-rank (X(IJK)) = four. pseudo-rank (X(3 × JK)) =three.
six different ultraviolet spectra form, pseudo-rank (X(6 × KI)) =six
==>> a Tucker3 (6,4,3) model is needed to fit X.
48
49
2 4 6
0.5
1
1.5
2
2.5
3
3.52 4 6
0.5
1
1.5
2
2.5
3
3.52 4 6
0.5
1
1.5
2
2.5
3
3.5
2 4 6
0.5
1
1.5
2
2.5
3
3.5
5 10 15
-0.4
-0.2
0
0.2
0.4
0 20 40 60 80
-0.2
-0.1
0
0.1
0.2
0 50 100-0.3
-0.2
-0.1
0
0.1
0.2
3 6 4 = 72 nonzero elements !!
Inequality laws:(i) P ≤ QR ; Q ≤ PR; R ≤ PQ(ii) max(3, 6, 4) ≤ S ≤ min(PQ, QR, PR) 6 ≤ S ≤ 12
50
three-way rank of X is ≥ 6 (six PARAFAC components fit the data) Pseudo rank (S=6) is not less than chemical rank(6) => no three-way rank deficiency.
rank deficiencies in one loading matrix of a three-way array are not the same as a three-way rank deficiency.
51
How it is possible to have a rank deficient three-way data?
52
Kronecker productKhatri-Rao product
Multi-way dataMatricizing the dataInteraction triadGPARAFACPanel performance
Matricizing and subarrayRankDimensionality vectorRank-deficiency in three-way arrays
Tucker3 rotational freedomUnique solution
Tucker2 modelTucker1 model
Tucker component models
Ledyard Tucker was one of the pioneers in multi-way analysis.
He proposed a series of models nowadays called N-mode PCA or Tucker models [Tucker 1964- 1966]
53
54
TUCKER3 MODELS
: nonzero off-diagonal elements in its core.
In Kronecker product notation the Tucker3 model
55
PROPERTIES OF THE TUCKER3 MODEL
TA : arbitrary nonsingular matrix
Such a transformation of the loading matrix A can be defined similarly for B and C, using TB and TC, respectively
56
Tucker3 rotational freedom
Tucker3 model has rotational freedom, But: it is not possible to rotate Tucker3 core-array to a superdiagonal form (and to obtain a PARAFAC model.!
57
The Tucker3 model : not give unique component matrices it has rotational freedom.
rotational freedom ÞOrthogonal component matrices (at no cost in fit by defining proper matrices TA, TB and TC)
convenient : to make the component matrices orthogonal
easy interpretation of the elements of the core-array and of the loadings by the loading plots
58
59
SS of elements of core-array
Þ amount of variation explained by combination of factors in different modes.
variation in X: unexplained and explained by model
Using a proper rotation all the variance of explained part can be gathered in core.
60
The rotational freedom of Tucker3 models can also be used to rotate the core-array to a simple structure as is also common in two-way analysis (will be explained).
Imposing the restrictions A’A = B’B = C’C = I : not sufficient for obtaining a unique solution
To obtain uniqe estimates of parameters, 1. loading matrices should be orthogonal, 2. A should also contain eigenvectors of X(CC’ ⊗ BB’)X’ corresp. to decreasing eigenvalues of that same matrix; similar restrictions should be put on B and C
[De Lathauwer 1997, Kroonenberg et al. 1989].
61
Unique solution
62
Unique Tucker
2 4 6 80
0.5
1
5 10 150
0.5
1
0 5 10 15 200
0.5
1Simulated data:
Two components,PARAFAC model
63
1 2 3
0.5
1
1.5
2
2.5
3
3.5
1 2 3
0.5
1
1.5
2
2.5
3
3.5
1 2 3
0.5
1
1.5
2
2.5
3
3.5 0
1
2
3
4
5
6x 10
-15
0
0.2
0.4
0.6
0.8
1
-15
-10
-5
0
1 2 3 4 5 6 7 8-1
0
1
0 2 4 6 8 10 12 14 16-0.5
0
0.5
1
0 2 4 6 8 10 12 14 16 18 20-0.5
0
0.5
1
UniqueTucker3 component model
P=Q=R=3
Only two significant elements in core
64
1 2 3
0.5
1
1.5
2
2.5
3
3.5
1 2 3
0.5
1
1.5
2
2.5
3
3.5
1 2 3
0.5
1
1.5
2
2.5
3
3.5 -15
-10
-5
0
x 10-15
-1
-0.8
-0.6
-0.4
-0.2
0
-15
-10
-5
0
1 2 3 4 5 6 7 8-1
0
1
0 2 4 6 8 10 12 14 16-0.5
0
0.5
1
0 2 4 6 8 10 12 14 16 18 20-1
-0.5
0
0.5
Not exactly unique!
65
1 2 3 4 5 6 7 8-1
0
10 2 4 6 8 10 12 14 16
-0.5
0
0.50 5 10 15 20
-1
0
1
0.5 1 1.5 2 2.5
0.5
1
1.5
2
2.50.5 1 1.5 2 2.5
0.5
1
1.5
2
2.5
0.5 1 1.5 2 2.5
0.5
1
1.5
2
2.50.5 1 1.5 2 2.5
0.5
1
1.5
2
2.5
1 2 3 4 5 6 7 8-1
0
1
0 2 4 6 8 10 12 14 16-0.5
0
0.5
0 2 4 6 8 10 12 14 16 18 20-0.5
0
0.5
Not exactly unique!
But very similar
66
Kronecker productKhatri-Rao product
Multi-way dataMatricizing the dataInteraction triadGPARAFACPanel performance
Matricizing and subarrayRankDimensionality vectorRank-deficiency in three-way arrays
Tucker3 rotational freedomUnique solution
Tucker2 modelTucker1 model
all three modes are reduced
In tucker 3
67
68
Data reduction only in two dimensions...
Tucker2 model
Tucker1 models : reduce only one of the modes.
+ X (and accordingly G) are matricized :
69
Tucker1 model
70
different models [Kiers 1991, Smilde 1997].
Threeway component models for X (I × J × K), A : the (I × P) component matrix (of first (reduced) mode,
X(I×JK) : matricized X; A,B,C : component matrices; G : different matricized core-arrays ; I :superdiagonal array (ones on superdiagonal. (compon matrices, core-arrays and residual error arrays : differ for each model
=> PARAFAC model is a special case of Tucker3 model.
PARAFAC: X(IxJK)= A G(RxRR)(CB)’
Tucker3: X(IxJK)= A G(PxQR)(CB)’
Tucker2: X(IxJK)= A G(PxQK)(I B)’
Tucker1: X(IxJK)= A G(PxJK)(I I)’
71
Thanks andSee you in the next session...