Multivariate Functional Data Analysis

ORI ROSEN

Department of Mathematical Sciences, University of Texas at El Paso,

El Paso, Texas, USA

and

WESLEY THOMPSON

Department of Statistics, University of Pittsburgh, Pittsburgh,

Pennsylvania, USA

Outline

• Motivating example

• The model

• Estimation

• Example

Motivating Example

• Psychiatric study to compare psychotherapy to pharmacotherapyat the University of Pisa, Italy. Total of 103 acutely depressedsubjects.

• Response variables: two subscales of the clinician-administered Hamilton Depression rating Scale.

• HRSD-17 – measures anhedonia, sleep disturbance,agitation, anxiety, weight loss

• HRSD-RV – measures reverse vegetative symptomssuch as weight gain and hyperphagia

• Covariates: treatment group, lifetime depression spectrum

0 4 8 12 16−2

−1

0

1

2

3

4

weeks

stan

dard

ized

scor

e

HRSD−17

0 4 8 12 16−2

−1

0

1

2

3

4

5

weeks

stan

dard

ized

scor

e

HRSD−RV

The Model

yi1(tij) = xxx′ijµ1µ1µ1(tij) + zzz′ijgi1gi1gi1(tij) + δi1(tij)

yi2(tij) = xxx′ijµ2µ2µ2(tij) + zzz′ijgi2gi2gi2(tij) + δi2(tij)... = ...

yip(tij) = xxx′ijµpµpµp(tij) + zzz′ijgipgipgip(tij) + δip(tij)

The Model

yi1(tij) = xxx′ijµ1µ1µ1(tij) + zzz′ijgi1gi1gi1(tij) + δi1(tij)

yi2(tij) = xxx′ijµ2µ2µ2(tij) + zzz′ijgi2gi2gi2(tij) + δi2(tij)... = ...

yip(tij) = xxx′ijµpµpµp(tij) + zzz′ijgipgipgip(tij) + δip(tij)

• µµµk(t) = (µk1(t), . . . , µkr(t))′ is an r-vector of fixed functions,k = 1, . . . , p

The Model

yi1(tij) = xxx′ijµ1µ1µ1(tij) + zzz′ijgi1gi1gi1(tij) + δi1(tij)

yi2(tij) = xxx′ijµ2µ2µ2(tij) + zzz′ijgi2gi2gi2(tij) + δi2(tij)... = ...

yip(tij) = xxx′ijµpµpµp(tij) + zzz′ijgipgipgip(tij) + δip(tij)

• µµµk(t) = (µk1(t), . . . , µkr(t))′ is an r-vector of fixed functions,k = 1, . . . , p

• gggik(t) = (gik1(t), . . . , giks(t))′ is an s-vector of randomfunctions, k = 1, . . . , p

The Model (Cont.)

yyyi(tij) = Xijµµµ(tij) + Zijgigigi(tij) + δδδi(tij) ,

• yyyi(tij) = (yi1(tij), . . . , yip(tij))′, i = 1, . . . , n, j = 1, . . . ,mi.

The Model (Cont.)

yyyi(tij) = Xijµµµ(tij) + Zijgigigi(tij) + δδδi(tij) ,

• yyyi(tij) = (yi1(tij), . . . , yip(tij))′, i = 1, . . . , n, j = 1, . . . ,mi.

• µµµ(t) = (µµµ′1(t), . . . , µµµ′p(t))

′

The Model (Cont.)

yyyi(tij) = Xijµµµ(tij) + Zijgigigi(tij) + δδδi(tij) ,

• yyyi(tij) = (yi1(tij), . . . , yip(tij))′, i = 1, . . . , n, j = 1, . . . ,mi.

• µµµ(t) = (µµµ′1(t), . . . , µµµ′p(t))

′

• gggi(t) = (ggg′i1(t), . . . , ggg′ip(t))

′

The Model (Cont.)

yyyi(tij) = Xijµµµ(tij) + Zijgigigi(tij) + δδδi(tij) ,

• yyyi(tij) = (yi1(tij), . . . , yip(tij))′, i = 1, . . . , n, j = 1, . . . ,mi.

• µµµ(t) = (µµµ′1(t), . . . , µµµ′p(t))

′

• gggi(t) = (ggg′i1(t), . . . , ggg′ip(t))

′

• Xij = Ip ⊗ xxx′ij, Zij = Ip ⊗ zzz′ij.

The Model (Cont.)

yyyi(tij) = Xijµµµ(tij) + Zijgigigi(tij) + δδδi(tij) ,

• yyyi(tij) = (yi1(tij), . . . , yip(tij))′, i = 1, . . . , n, j = 1, . . . ,mi.

• µµµ(t) = (µµµ′1(t), . . . , µµµ′p(t))

′

• gggi(t) = (ggg′i1(t), . . . , ggg′ip(t))

′

• Xij = Ip ⊗ xxx′ij, Zij = Ip ⊗ zzz′ij.

• δδδi(tij) = (δi1(tij), . . . , δip(tij))′

Example

Assume 2 groups of subjects and a bivariate response (p = 2):

xxxij = (xij1, xij2)′, where xij1 = 1, xij2 =

1 if treatment0 if control

Example

Assume 2 groups of subjects and a bivariate response (p = 2):

xxxij = (xij1, xij2)′, where xij1 = 1, xij2 =

1 if treatment0 if control

yi1(tij) = (1 xij2)(µ11(tij)µ12(tij)

)+ gi1(tij) + δi1(tij)

yi2(tij) = (1 xij2)(µ21(tij)µ22(tij)

)+ gi2(tij) + δi2(tij)

Example

Assume 2 groups of subjects and a bivariate response (p = 2):

xxxij = (xij1, xij2)′, where xij1 = 1, xij2 =

1 if treatment0 if control

yi1(tij) = (1 xij2)(µ11(tij)µ12(tij)

)+ gi1(tij) + δi1(tij)

yi2(tij) = (1 xij2)(µ21(tij)µ22(tij)

)+ gi2(tij) + δi2(tij)

soyi1(tij) = µ11(tij) + xij2 · µ12(tij) + gi1(tij) + δi1(tij)

yi2(tij) = µ21(tij) + xij2 · µ22(tij) + gi2(tij) + δi2(tij)

The Error Term

δδδi(t) follows a multivariate Ornstein-Uhlenbeck (O-U) process, i.e.,

dδδδi(t) = −Aδδδi(t) +BdWWW i(t) .

• A and B are p × p matrices of full rank common to all i =1, . . . , n,

• WWW i(t) is the p-dimensional Wiener process.

• Stationary if the EV of A have positive real parts.

• The stationary variance-covariance matrix Σ is the solution to

AΣ + ΣA′ = BB′

The Error Term (Cont.)

In the stationary state, for s < t

Cov(δδδi(t), δδδi(s)) = exp−A(t− s)Σ ,

so A controls how rapidly the influence of time s dies off by time t > s.

The Transition Density

Let ∆tij = tij − ti,j−1 j = 1, . . . ,mi, ti0 = 0.

The transition density of the O-U process is given by

p(δδδi(tij)|δδδi(ti,j−1),∆tij) ∝ |Ω∆tij|−1/2 exp

−1

2γγγ′tijΩ

−1∆tij

γγγtij

,

where

γγγtij = δδδi(tij)− exp(−A∆tij)δδδi(ti,j−1)

Ω∆tij = Σ− exp(−A∆tij)Σ exp(−A′∆tij).

Nonparametric Function Estimation– Simple Example

200 observations were generated from

yi = sin(3πxi) + εi, , 0 ≤ xi ≤ 1, εi ∼ N(0, 0.42)

Model:

yi = β0 + β1xi +30∑k=1

uk(xi − κk)+ + εi, εi ∼ N(0, σ2ε )

• Low rank truncated line basis functions.

• Fixed effects: βs and us fixed.

• Mixed effects: βs fixed, u1, . . . , u30iid∼ N(0, σ2

u).

0 0.2 0.4 0.6 0.8 1−2

−1

0

1

2

x

yFixed Effects Model

0 0.2 0.4 0.6 0.8 1−2

−1

0

1

2

x

y

Mixed Model

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

x0 0.2 0.4 0.6 0.8 1

0

0.2

0.4

0.6

0.8

1

x

Nonparametric Function Estimation (Cont.)

We estimate

µkl(t) and gikm(t), k = 1, . . . , p, l = 1, . . . , r,m = 1, . . . , s, i = 1, . . . , n

by cubic splines using low-rank radial basis functions (French et al.,2001, Ruppert et al., 2003):

Write µkl(tij) = φφφ′ijβββkl + ψψψ′ijvvvkl and gikm(tij) = φφφ′ijwwwikm + ψψψ′ijuuuikm.

Nonparametric Function Estimation (Cont.)

We estimate

µkl(t) and gikm(t), k = 1, . . . , p, l = 1, . . . , r,m = 1, . . . , s, i = 1, . . . , n

by cubic splines using low-rank radial basis functions (French et al.,2001, Ruppert et al., 2003):

Write µkl(tij) = φφφ′ijβββkl + ψψψ′ijvvvkl and gikm(tij) = φφφ′ijwwwikm + ψψψ′ijuuuikm.

• Place K knots κ1, . . . , κK at sample quantiles of tij.

Nonparametric Function Estimation (Cont.)

We estimate

µkl(t) and gikm(t), k = 1, . . . , p, l = 1, . . . , r,m = 1, . . . , s, i = 1, . . . , n

by cubic splines using low-rank radial basis functions (French et al.,2001, Ruppert et al., 2003):

Write µkl(tij) = φφφ′ijβββkl + ψψψ′ijvvvkl and gikm(tij) = φφφ′ijwwwikm + ψψψ′ijuuuikm.

• Place K knots κ1, . . . , κK at sample quantiles of tij.

• Let φφφ′ij = (1 tij), ξξξ′ij = (|tij − κ1|3, . . . , |tij − κK|3),

• Compute ΛK = [|κk − κk′|3]1≤k,k′≤K and ψψψ′ij = ξξξ′ijΛ−1/2K (via

SVD)

Prior Distributions

Priors on the basis function coefficients:

1. Flat priors on βββkl.

2. vvvklind∼ N(000, σ2

vklIK)

3. wwwikmind∼ N(000,diag(σ2

wkm0, σ2wkm1

)).

4. uuuikmind∼ N(000, σ2

ukmIK).

Priors on the variance components:independent IG(ai, bi), e.g., p(σ2

vkl) ∝ (σ2

vkl)−(a1+1) exp(−b1/σ2

vkl)

Prior Distributions (Cont.)

Prior on A:

• Let A = SΨS−1, where

S =

1 s12 s13 . . . s1ps21 1 s23 . . . s2p... ... ... . . . ...sp1 sp2 sp3 . . . 1

and Ψ = diag(ψ1, . . . , ψp), ψi > 0, i = 1, . . . , p.

• sij ∼ N(0, σ2s), logψi ∼ N(0, σ2

ψ)

Prior Distributions (Cont.)

Prior on C = BB′:

• Let C = LDL′ (modified Cholesky factorization), where

L =

1 0 0 . . . 0l21 1 0 . . . 0l31 l32 1 . . . 0... ... ... . . . ...lp1 lp2 lp3 . . . 1

and D = diag(d1, . . . , dp), di > 0, i = 1, . . . , p

• lij ∼ N(0, σ2L), log(di) ∼ N(0, σ2

D)

0 4 8 12 16−2

−1

0

1

2

3

4

weeks

stan

dard

ized

scor

e

HRSD−17

0 4 8 12 16−2

−1

0

1

2

3

4

5

weeks

stan

dard

ized

scor

e

HRSD−RV

Example• Psychiatric study to compare psychotherapy to pharmacotherapy

at the University of Pisa, Italy. Total of 103 acutely depressedsubjects.

• Response variables: 2 subscales of the clinician-administeredHamilton Depression rating Scale (measured at baseline andweekly):

HRSD-17 (0–31)HRSD-RV (0–13).

Higher values indicate more depressive symptoms.

• Covariates:treatment grouplifetime depression spectrum (LDS), assessed at baseline.

Example (Cont.)

yyyi(tij) = Xijµµµ(tij) + Zijgigigi(tij) + δδδi(tij) ,where Xij = Ip ⊗ xxx′ij, Zij = Ip ⊗ zzz′ij.

In the example yyyi(tij) = (HRSD-17,HRSD-RV)′

zzzij = 1xxxij = (xij1, xij2, xij3, xij4)′

where xij1 = 1

xij2 =

1 if subject i received psychotherapy0 if subject i received pharmacotherapy,

xij3 = LDS scorexij4 = xij2xij3.

0 4 8 12 16−2

−1.5

−1

−0.5

0

0.5

1

1.5

2

t (weeks)

Intercept

0 4 8 12 16−1

−0.5

0

0.5

1

1.5

t (weeks)

Treatment

0 4 8 12 16−0.05

0

0.05

0.1

0.15

t (weeks)

LDS Score

0 4 8 12 16−0.15

−0.1

−0.05

0

0.05

t (weeks)

Treatment x LDS

HRSD−17

0 4 8 12 16−2

−1.5

−1

−0.5

0

0.5

1

1.5

2

t (weeks)

Intercept

0 4 8 12 16−2

−1.5

−1

−0.5

0

0.5

1

1.5

2

t (weeks)

Treatment Group

0 4 8 12 16−0.05

0

0.05

0.1

0.15

t (weeks)

LDS Score

0 4 8 12 16−0.15

−0.1

−0.05

0

0.05

0.1

t (weeks)

Treament x LDS

HRSD−RV

Conclusions from the Analysis• Little evidence of a treatment effect

• Significant effect of LDS score – higher LDS predicts worseoutcomes

• LDS effect is more pronounced in the middle period of HRSD-17, and towards the end of the 16 weeks for HRSD-RV.

• Interaction is significant in the last 8 weeks of HRSD-17, andmarginally significant in the same time period for HRSD-RV.

• Interaction effect – LDS is less predictive of poor response inpsychotherapy group than in pharmacotherapy group.

• Univariate fitting – interaction is not significant; wider pointwisecredible intervals.