Symmetric semi-parametric models with applications using Rgiapaula/slides_exemplos_semi.pdf ·...

Symmetric semi-parametric models withapplications using R

Gilberto A. Paula

Instituto de Matemática e EstatísticaUniversidade de São Paulo, Brasil

[email protected]

2o Semestre 2015

G. A. Paula (IME-USP) Symmetric semi-parametric models in R 2o Semestre 2015 1 / 105

Examples

Outline

1 Examples

2 Defining f (x)

3 Additive normal model

4 Semi-parametric normal model

5 Packages in R

6 Voltage drop data

7 Boston housing data

8 Symmetric distributions


10 Extensions available in the library ssym

11 Comparison of snacks

12 Bibliography


Examples

Voltage drop data

Description

As a 1st example we will consider the voltage drop data (Montgomeryand Peck, 2001) in which a battery voltage drop in a guided missilemotor is observed over the time of missile flight. It was intended avoltage drop model for using a digital-analog simulation model of themissile. Altogether there are 41 observations.


Examples

Scatter plot of voltage drop data

0 5 10 15 20

810

1214

Time

Volta

ge


Examples


0 5 10 15 20

810

1214

Time

Volta

ge


Examples

Possible model

Description

The data suggest a nonparametric model such as:

Voltagei = α+ f (Timei) + ǫi ,


Examples

Possible model

Description

The data suggest a nonparametric model such as:


where ǫi∼ N(0, σ2) for i = 1, . . . , 41, with f (·) being a continuous,smooth and nonparametric function.


Examples

Boston housing data

Description

As a 2nd example we will consider the Boston housing data that havebeen analyzed by various authors (see, for instance, Belsley et al.1980). The aim of the study is to assess the association of houseprices with the air quality of the neighborhood by using regressionmodels. The outcome variable


Examples

Boston housing data

Description


LMEDV (logarithm of the median house price in USD 1000)


Examples

Boston housing data

Description


LMEDV (logarithm of the median house price in USD 1000)

is related with 13 explanatory variables. Altogether there are 506observations.


Examples

Boston housing data

Illustration

We will work, for the purpose of motivating the semi-parametricmodels, with three explanatory variables:


Examples

Boston housing data

Illustration


NOX (annual average nitric oxide concentration, p.p. 10 million);


Examples

Boston housing data

Illustration



LSTAT (% lower status of the population);


Examples

Boston housing data

Illustration



LSTAT (% lower status of the population);

DIS (weighted distances to five Boston employment centers).


Examples

Plot of LMEDV versus NOX

0.4 0.5 0.6 0.7 0.8

2.0

2.5

3.0

3.5

4.0

NOX

LME

DV


Examples


0.4 0.5 0.6 0.7 0.8

2.0

2.5

3.0

3.5

4.0

NOX

LME

DV


Examples

Plot of LMEDV versus LSTAT

10 20 30

2.0

2.5

3.0

3.5

4.0

LSTAT

LME

DV


Examples


10 20 30

2.0

2.5

3.0

3.5

4.0

LSTAT

LME

DV


Examples

Plot of LMEDV versus DIS

2 4 6 8 10 12

2.0

2.5

3.0

3.5

4.0

DIS

LME

DV


Examples

Possible model

Description

We may try to fit initially the following semi-parametric model:

LMEDVi = α+ βNOXi + f1(LSTATi) + f2(DISi) + ǫi ,


Examples

Possible model

Description


LMEDVi = α+ βNOXi + f1(LSTATi) + f2(DISi) + ǫi ,

where ǫiiid∼ N(0, σ2) for i = 1, . . . , 506, with f1(·) and f2(·) being

continuous, smooth and nonparametric functions.


Examples

Comparison of snacks

Description

As a 3rd example, we will consider a data set from an experimentdeveloped in School of Public Health - Universidade de São Paulo, inwhich 4 different forms of light snacks (B, C, D and E) were comparedacross 20 weeks with a traditional snack (A).


Examples


Experiment description

The hydrogenated vegetable fat (hvt) was replaced by canola oil underdifferent proportions:


Examples




A: 22% hvf, 0% canola oil


Examples





B: 0% hvf, 22% canola oil


Examples






C: 17% hvf, 5% canola oil


Examples







D: 11% hvf, 11% canola oil


Examples








E: 5% hvf, 17% canola oil.


Examples









In this analysis we will only consider the variable TEXTURE that will becompared across time among the 5 snack types.


Examples

Mean profiles

5 10 15 20

4050

6070

80

Weeks

Text

ure

ABCDE


Examples

Variation coefficient profiles

5 10 15 20

0.05

0.10

0.15

0.20

0.25

0.30

0.35

Weeks

VC

of T

extu

reABCDE


Examples

Double gamma model

Description

Similarly to Paula (2013) we may consider a semi-parametric doublegamma model:


Examples

Double gamma model

Description


yijkind∼ G(µij , φij);


Examples

Double gamma model

Description



log(µij) = β0 + βi + f (Weeksj);


Examples

Double gamma model

Description




log(φ−1ij ) = γ0 + γi ,


Examples

Double gamma model

Description




log(φ−1ij ) = γ0 + γi ,

for i = 1(A), 2(B), 3(C), 4(D), 5(E), j = 2, 4, . . . , 20 and k = 1, . . . , 15,where φ−1

ij is the dispersion parameter, β0 + βi and γ0 + γi denote thesnack effects whereas f (·) is continuous, smooth and nonparametricfunction.


Defining f (x)

Outline

1 Examples

2 Defining f (x)



5 Packages in R

6 Voltage drop data






12 Bibliography


Defining f (x)

Defining f (x)

How to define f (x)?


Defining f (x)

Defining f (x)


Piecewise-cubic splines


Defining f (x)

Defining f (x)


Piecewise-cubic splinesB-splines


Defining f (x)

Defining f (x)



Natural cubic splines


Defining f (x)

Defining f (x)



Natural cubic splinesP-splines


Defining f (x)

Defining f (x)



Natural cubic splinesP-splinesThin-plate splines


Defining f (x)

Defining f (x)



Natural cubic splinesP-splinesThin-plate splines· · ·


Defining f (x)

Defining f (x)




Kernel


Defining f (x)

Defining f (x)




Kernel

Loess


Defining f (x)

Defining f (x)




Kernel

Loess

Wavelets


Defining f (x)

Defining f (x)




Kernel

Loess

Wavelets

· · ·


Defining f (x)


Definition

Suppose the explanatory variable values are in the interval [a, b], fori = 1, . . . , n, with m internal knots, namely a < t1 < · · · < tm < b,where m ≤ n − 2.


Defining f (x)


Definition

Suppose the explanatory variable values are in the interval [a, b], fori = 1, . . . , n, with m internal knots, namely a < t1 < · · · < tm < b,where m ≤ n − 2.

A simple choice for the nonparametric function f (x) could be thepiecewise-cubic spline, described as

f (x) = β0 + β1x + β2x2 +

m∑

j=1

γj(x − tj)3+,

where (x − tj)+ = max[0, (x − tj)].


Defining f (x)

Voltage drop data

Suppose m = 2 internal knots at t1 = 6.5 and t2 = 13.


Defining f (x)

Voltage drop data

Suppose m = 2 internal knots at t1 = 6.5 and t2 = 13.

0 5 10 15 20

810

1214

Time

Volta

ge


Defining f (x)

Voltage drop data

Fitting on the interval [0;6.5]


Defining f (x)

Voltage drop data


yi = β0 + β1xi + β2x2i + β3x3

i + ǫi .


Defining f (x)

Voltage drop data


yi = β0 + β1xi + β2x2i + β3x3

i + ǫi .

Fitting on the interval (6.5;13]


Defining f (x)

Voltage drop data


yi = β0 + β1xi + β2x2i + β3x3

i + ǫi .


yi = β0 + β1xi + β2x2i + β3x3

i + γ1(xi − 6.5)3 + ǫi .


Defining f (x)

Voltage drop data


yi = β0 + β1xi + β2x2i + β3x3

i + ǫi .


yi = β0 + β1xi + β2x2i + β3x3

i + γ1(xi − 6.5)3 + ǫi .

Fitting on the interval (13;20]


Defining f (x)

Voltage drop data


yi = β0 + β1xi + β2x2i + β3x3

i + ǫi .


yi = β0 + β1xi + β2x2i + β3x3

i + γ1(xi − 6.5)3 + ǫi .


yi = β0 + β1xi + β2x2i + β3x3

i + γ1(xi − 6.5)3 + γ2(xi − 13)3 + ǫi .


Defining f (x)

Voltage drop data


yi = β0 + β1xi + β2x2i + β3x3

i + ǫi .


yi = β0 + β1xi + β2x2i + β3x3

i + γ1(xi − 6.5)3 + ǫi .


yi = β0 + β1xi + β2x2i + β3x3

i + γ1(xi − 6.5)3 + γ2(xi − 13)3 + ǫi .

The parameter vector β = (β0, β1, β2, β3, γ1, γ2)⊤ may be estimated by

least-squares.


Defining f (x)

B-splines

Definition

A more flexible class that contains candidates for f (x) is the B-splinesclass, defined as


Defining f (x)

B-splines

Definition


f (x) =q

∑

j=1

Nj(x)τj , x ∈ [a, b],


Defining f (x)

B-splines

Definition


f (x) =q

∑

j=1

Nj(x)τj , x ∈ [a, b],

where Nj(x) are the B-spline basis functions and τj are coefficients.


Defining f (x)


Definition

NCS (see, for instance, Green and Silverman, 1994) may beexpressed as B-splines and have the following properties:


Defining f (x)


Definition


the explanatory variable values have distinct values, namelya ≤ t1 < · · · < tq ≤ b,


Defining f (x)


Definition



f (x) is a cubic spline in the intervals [t1, t2], . . . , [tq−1, tq],


Defining f (x)


Definition




f (x) is linear in the intervals [a, t1] and [tq, b],


Defining f (x)


Definition





f (x), f ′(x) and f ′′(x) are continuous.


Defining f (x)


Definition






Therefore, for NCS one has m = q − 2 internal knots.


Defining f (x)


Definition






Therefore, for NCS one has m = q − 2 internal knots.

NCS may also be defined for arbitrary m internal knots.


Defining f (x)

P-splines

Definition

P-splines (Eilers and Marx, 1996) form a flexible class of B-splinesdefined as


Defining f (x)

P-splines

Definition


f (x) =q

∑

j=1

Nj,k (x)τj , x ∈ [a, b],


Defining f (x)

P-splines

Definition


f (x) =q

∑

j=1

Nj,k (x)τj , x ∈ [a, b],

where Nj,k (x) are the B-spline basis functions of degree k (de Boor,1978), for k = 0, 1, 2, . . ., τj are coefficients, m is the number of internalknots, namely a < t1 < · · · < tm < b, and m = q + k + 1.


Defining f (x)

P-splines

Basis function

De Boor’s B-splines basis functions are expressed as


Defining f (x)

P-splines

Basis function


Nj,0(x) ={

1 tj ≤ x ≤ tj+1

0 otherwise


Defining f (x)

P-splines

Basis function


Nj,0(x) ={

1 tj ≤ x ≤ tj+1

0 otherwise

and

Nj,k (x) =(x − tj)(tj+k − tj)

Nj,k−1(x) +(tj+k+1 − x)(tj+k+1 − tj+1)

Nj+1,k−1(x),

for j = 1, . . . , q and k = 1, 2, 3, . . . .


Defining f (x)

Penalization

Why to penalize?

The aim of penalization is to reduce the parametric space solution inorder to avoid overfitting.


Additive normal model

Outline

1 Examples

2 Defining f (x)



5 Packages in R

6 Voltage drop data






12 Bibliography




Description

First, we will assume the following nonparametric model:




Description


yi = f (ti) + ǫi ,




Description


yi = f (ti) + ǫi ,

where f (t) is a continuous, smooth and nonparametric function and

ǫiiid∼ N(0, σ2), for i = 1, . . . , n.




Penalization

A suggestion is to use the second derivative penalization.




Penalization

A suggestion is to use the second derivative penalization. So, theobjective function to be minimized is given by




Penalization


SP(f, λ) =n

∑

i=1

{yi − f (ti)}2 + λ

∫ b

a[f ′′(x)]2dx ,




Penalization


SP(f, λ) =n

∑

i=1

{yi − f (ti)}2 + λ

∫ b

a[f ′′(x)]2dx ,

where f = (f (t1), . . . , f (tq))⊤, [a, b] denotes the data interval and λ > 0is the smoothing parameter.




Penalization


SP(f, λ) =n

∑

i=1

{yi − f (ti)}2 + λ

∫ b

a[f ′′(x)]2dx ,

where f = (f (t1), . . . , f (tq))⊤, [a, b] denotes the data interval and λ > 0is the smoothing parameter.

The solution is a natural cubic spline with knots at the distinct valuesa ≤ t1 < · · · < tq ≤ b.




Smoothing parameter

One has the following λ interpretation:




Smoothing parameter


when λ → 0 minimizing SP(f, λ) leads to a data interpolation;




Smoothing parameter



when λ → ∞ one has to impose f ′′(x) = 0 so the solution leads toa linear function for f (x);




Smoothing parameter



when λ → ∞ one has to impose f ′′(x) = 0 so the solution leads toa linear function for f (x);

then 0 < λ < ∞.



Semi-parametric normal model

Penalization

One has for B-splines the following solution (see, for instance, Wood,2006):




Penalization


∫ b

a[f ′′(x)]2dx = τ⊤Kτ ,




Penalization


∫ b

a[f ′′(x)]2dx = τ⊤Kτ ,

where K is a (q × q) non-negative definite smoothing matrix that doesnot depend on τ .



Outline

1 Examples

2 Defining f (x)



5 Packages in R

6 Voltage drop data






12 Bibliography




Description

We will assume now the following partially linear model:




Description


yi = x⊤

i β + f (ti) + ǫi ,




Description


yi = x⊤

i β + f (ti) + ǫi ,

where x i = (xi1, . . . , xip)⊤ contains values of explanatory variables,

β = (β1, . . . , βp)⊤, f (ti) = N⊤

i τ is a B-spline and ǫiiid∼ N(0, σ2), for

i = 1, . . . , n.




Description


yi = x⊤

i β + f (ti) + ǫi ,


β = (β1, . . . , βp)⊤, f (ti) = N⊤


i = 1, . . . , n.

Objective function

The penalized least-squares function becomes




Description


yi = x⊤

i β + f (ti) + ǫi ,


β = (β1, . . . , βp)⊤, f (ti) = N⊤


i = 1, . . . , n.

Objective function

The penalized least-squares function becomes

SP(θ, λ) = (y − Xβ − Nτ )⊤(y − Xβ − Nτ ) + λτ⊤Kτ ,

where θ = (β⊤, τ⊤)⊤.




Iterative process

One has the following iterative process:




Iterative process


starting with β(0) as the parametric least-squares solution;




Iterative process



τ (0) = (N⊤N + λK)−1N⊤(y − Xβ(0));




Iterative process



τ (0) = (N⊤N + λK)−1N⊤(y − Xβ(0));back-fitting (Gauss-Seidel) algorithm:




Iterative process




β(m+1) = (X⊤X)−1X⊤{y − Nτ (m)}




Iterative process




β(m+1) = (X⊤X)−1X⊤{y − Nτ (m)}τ (m+1) = (N⊤N + λK)−1N⊤{y − Xβ(m+1)},




Iterative process




β(m+1) = (X⊤X)−1X⊤{y − Nτ (m)}τ (m+1) = (N⊤N + λK)−1N⊤{y − Xβ(m+1)},

for m = 0, 1, 2, . . . and λ fixed.




Effective degrees of freedom

From the iterative process at the convergence one has that






f = Nτ






f = Nτ

= N(N⊤N + λK)−1N⊤{y − Xβ}






f = Nτ

= N(N⊤N + λK)−1N⊤{y − Xβ}= H(λ){y − Xβ}.






f = Nτ

= N(N⊤N + λK)−1N⊤{y − Xβ}= H(λ){y − Xβ}.

So, as suggested by Hastie and Tibshirani (1990) one may take






f = Nτ

= N(N⊤N + λK)−1N⊤{y − Xβ}= H(λ){y − Xβ}.


df(λ) = tr{H(λ)}






f = Nτ

= N(N⊤N + λK)−1N⊤{y − Xβ}= H(λ){y − Xβ}.


df(λ) = tr{H(λ)}= tr{N(N⊤N + λK)−1N⊤}






f = Nτ

= N(N⊤N + λK)−1N⊤{y − Xβ}= H(λ){y − Xβ}.


df(λ) = tr{H(λ)}= tr{N(N⊤N + λK)−1N⊤}= tr{N⊤N(N⊤N + λK)−1}.




Model selection

The Akaike Information Criterion (AIC) and the Bayesian InformationCriterion (BIC) are, respectively, defined as




Model selection


AIC(λ) = −2L(θ, σ2) + 2{p + df(λ) + 1};




Model selection


AIC(λ) = −2L(θ, σ2) + 2{p + df(λ) + 1};

BIC(λ) = −2L(θ, σ2) + log(n){p + df(λ) + 1},




Model selection


AIC(λ) = −2L(θ, σ2) + 2{p + df(λ) + 1};

BIC(λ) = −2L(θ, σ2) + log(n){p + df(λ) + 1},

for given λ.




Estimator of the variance

For σ2 one has (given λ) the following estimator:






σ2 =

∑ni=1(yi − yi)

2

{n − p − df(λ)} .






σ2 =

∑ni=1(yi − yi)

2

{n − p − df(λ)} .

Choosing the smoothing parameter

Minimizing the generalized cross-validation score






σ2 =

∑ni=1(yi − yi)

2

{n − p − df(λ)} .



GCV(λ) =n∑n

i=1(yi − yi)2

{n − df(λ)}2 ,






σ2 =

∑ni=1(yi − yi)

2

{n − p − df(λ)} .



GCV(λ) =n∑n

i=1(yi − yi)2

{n − df(λ)}2 ,

or minimizing (jointly) AIC(λ) and df(λ) for a grid of λ values.



Alternative penalization

P-splines

Eilers and Marx (1996) proposes the alternative penalization




P-splines


SP(θ, λ) = (y − Xβ − Nτ )⊤(y − Xβ − Nτ ) + λ

q∑

j=d+1

[∆dτj ]2,




P-splines



q∑

j=d+1

[∆dτj ]2,

where N is the de Boor’s basis and ∆dτj is the penalty difference termof order d .




P-splines



q∑

j=d+1

[∆dτj ]2,


In matrix notation

SP(θ, λ) = (y − Xβ − Nτ )⊤(y − Xβ − Nτ ) + λτ⊤D⊤

d Ddτ ,




P-splines



q∑

j=d+1

[∆dτj ]2,


In matrix notation

SP(θ, λ) = (y − Xβ − Nτ )⊤(y − Xβ − Nτ ) + λτ⊤D⊤

d Ddτ ,

where Dd is the penalty difference matrix of order d .



P-splines

Penalization examples



P-splines


∆τj = τj − τj−1

D1 =

−1 1 0 00 −1 1 00 0 −1 1

.



P-splines



D1 =

−1 1 0 00 −1 1 00 0 −1 1

.

∆2τj = τj − 2τj−1 + τj−2

D2 =

1 −2 1 0 00 1 −2 1 00 0 1 −2 1

.



P-splines



D1 =

−1 1 0 00 −1 1 00 0 −1 1

.

∆2τj = τj − 2τj−1 + τj−2

D2 =

1 −2 1 0 00 1 −2 1 00 0 1 −2 1

.

∆3τj = τj − 3τj−1 + 3τj−2 − τj−3

D3 =

−1 3 −3 1 0 00 −1 3 −3 1 00 0 −1 3 −3 1

.


Packages in R

Outline

1 Examples

2 Defining f (x)



5 Packages in R

6 Voltage drop data






12 Bibliography


Packages in R

Packages in R

Packages in R

Some packages for fitting semi-parametric regression models availablefrom CRAN at http://CRAN.R-project.org:


Packages in R

Packages in R

Packages in R


gamlss (Rigby and Stasinopoulos, 2015)


Packages in R

Packages in R

Packages in R



mgcv (Wood, 2015)


Packages in R

Packages in R

Packages in R



mgcv (Wood, 2015)

ssym (Vanegas and Paula, 2015a, 2015b)


Voltage drop data

Outline

1 Examples

2 Defining f (x)



5 Packages in R

6 Voltage drop data






12 Bibliography


Voltage drop data


0 5 10 15 20

810

1214

Time

Volta

ge


Voltage drop data

Fitted model

Description

We will fit by the package ssym the following model:



Voltage drop data

Fitted model

Description



where α is an intercept, f (·) is a continuous, smooth and

nonparametric function and ǫiiid∼ N(0, σ2) for i = 1, . . . , 41.


Voltage drop data

Fitted model

Description



where α is an intercept, f (·) is a continuous, smooth and

nonparametric function and ǫiiid∼ N(0, σ2) for i = 1, . . . , 41.

Suggestion: (n13 + 3) knots.


Voltage drop data

> require(ssym)> fit1.battery = ssym.l(voltage ~ ncs(time), data=battery,family="Normal")> summary(fit1.battery)

Family: NormalSample size: 41Quantile of the Weights0% 25% 50% 75% 100%1 1 1 1 1

************************** Median/Location submodel ********************************** Parametric component

Estimate Std.Err z-value Pr(>|z|)(Intercept) 10.904 0.0542 201.3309 < 2.2e-16 *********** Nonparametric component

Smooth.param Basis.dimen d.f. Statistic p-valuencs(time) 4.243 5.000 4.931 2709 <2e-16 ***

**** Deviance: 41


Voltage drop data

************************* Skewness/Dispersion submodel ******************************* Parametric component

Estimate Std.Err z-value Pr(>|z|)(Intercept) -2.3484 0.2209 -10.6329 < 2.2e-16 ***

**** Deviance: 42.2

*******************************************************************Overall goodness-of-fit statistic: 0.152165

-2*log-likelihood: 20.068AIC: 33.931BIC: 45.808

> np.graph(fit1.battery,which=1,xlab="Time", ylab="Voltage")> np.graph(fit1.battery,which=1,xlab="Time", ylab="Voltage",obs=TRUE)


Voltage drop data

Voltage 95% confidence band

0 5 10 15 20

−4−2

02

4

Voltage

Non

para

met

ric e

stim

ate

0 5 10 15 20

−4−2

02

4

Voltage

Non

para

met

ric e

stim

ate


Voltage drop data

Voltage 95% confidence band

0 5 10 15 20

−4−2

02

4

Voltage

Non

para

met

ric e

stim

ate

0 5 10 15 20

−4−2

02

4

0 5 10 15 20

−4−2

02

4

Voltage

Non

para

met

ric e

stim

ate


Boston housing data

Outline

1 Examples

2 Defining f (x)



5 Packages in R

6 Voltage drop data






12 Bibliography


Boston housing data


0.4 0.5 0.6 0.7 0.8

2.0

2.5

3.0

3.5

4.0

NOX

LME

DV


Boston housing data


10 20 30

2.0

2.5

3.0

3.5

4.0

LSTAT

LME

DV


Boston housing data

Possible model

Description


LMEDVi = α+ βNOXi + f (LSTATi) + ǫi ,


Boston housing data

Possible model

Description



where ǫiiid∼ N(0, σ2) for i = 1, . . . , 506, with f (·) being a continuous,

smooth and nonparametric function.


Boston housing data

> require(ssym)> require(MASS)> fit1.boston= ssym.l(log(medv) ~ nox + psp(lstat), data=Boston,family="Normal")

> summary(fit1.boston)

Family: NormalSample size: 506Quantile of the Weights0% 25% 50% 75% 100%1 1 1 1 1


Estimate Std.Err z-value Pr(>|z|)(Intercept) 3.1251 0.0650 48.0810 <2e-16 ***nox -0.1543 0.1106 -1.3954 0.1629

******** Nonparametric component

Smooth.param Basis.dimen d.f. Statistic p-valuepsp(lstat) 17.1 11.000 7.282 731.9 <2e-16 ***


Boston housing data

**** Deviance: 506



**** Deviance: 762.68


-2*log-likelihood: -74.654AIC: -54.09BIC: -10.632

> np.graph(fit1.boston, which=1, xlab="Lstat",ylab="Estimate of f(Lstat)")> envelope(fit1.boston)


Boston housing data

f(Lstat) 95% confidence band

10 20 30

−1.0

−0.5

0.0

0.5

1.0

Lstat

Non

para

met

ric e

stim

ate

10 20 30

−1.0

−0.5

0.0

0.5

1.0

Lstat

Non

para

met

ric e

stim

ate


Boston housing data

Normal probability plot

−3 −2 −1 0 1 2 3

−4−2

02

Quantile N(0,1)

Mea

n de

vian

ce r

esid

ual

−3 −2 −1 0 1 2 3

−4−2

02

4Quantile N(0,1)

Dis

pers

ion

devi

ance

res

idua

l


Symmetric distributions

Outline

1 Examples

2 Defining f (x)



5 Packages in R

6 Voltage drop data






12 Bibliography




DefinitionLet y be a continuous random variable whose distribution belongs tothe symmetric class (Fang et al.,1990; Osorio et al., 2007; Cysneiroset al., 2005), with location parameter −∞ < µ < ∞, scale parameterφ > 0 and density generator g(·).




DefinitionLet y be a continuous random variable whose distribution belongs tothe symmetric class (Fang et al.,1990; Osorio et al., 2007; Cysneiroset al., 2005), with location parameter −∞ < µ < ∞, scale parameterφ > 0 and density generator g(·). We will denote y ∼ S(µ, φ) withprobability density function





fy (y ;µ, φ) =g[(y − µ)2/φ]√

φ,

for −∞ < y < ∞, provided that g(u)>0 for u>0 and∫

∞

0 u−12 g(u)du = 1.





fy (y ;µ, φ) =g[(y − µ)2/φ]√

φ,

for −∞ < y < ∞, provided that g(u)>0 for u>0 and∫

∞

0 u−12 g(u)du = 1.When they exist E(y) = µ and Var(y) = ξφ.




Examples




Examples

Normal(µ, φ)

g(u) ∝ exp[

−12

u]

.




Examples

Normal(µ, φ)

g(u) ∝ exp[

−12

u]

.

Student-t(µ, φ, ζ) (Lange et al., 1989)

g(u) ∝[

1 +uζ

]−ζ+1

2

,

where ζ > 0 denotes the degrees of freedom.



Normal and Student-t distributions0

.00

.10

.20

.30

.4

y

f(y)

N(0,1)

t(0,1,1)

0.0

0.1

0.2

0.3

0.4

y

f(y)

N(0,1)

t(0,1,4)




Examples




Examples

Power-exponential(µ, φ, ζ) (Gómes et al., 1998)

g(u) ∝ exp[

−12

u1

1+ζ

]

,

where −1 < ζ ≤ 1 denotes the shape parameter (normal ζ = 0).




Examples

Power-exponential(µ, φ, ζ) (Gómes et al., 1998)

g(u) ∝ exp[

−12

u1

1+ζ

]

,

where −1 < ζ ≤ 1 denotes the shape parameter (normal ζ = 0).

Contaminated-normal(µ, φ, ζ1, ζ2) (Little, 1998)

g(u) ∝√

ζ2 exp[

−12ζ2u

]

+(1 − ζ1)

ζ1exp

[

−12

u]

,

for ζ1, ζ2 ∈ (0, 1).



Normal and power exponential distributions0

.00

.10

.20

.30

.40

.5

y

f(y)

N(0,1)

EP(0,1,−0.3)

0.0

0.1

0.2

0.3

0.4

y

f(y)

N(0,1)

EP(0,1,0.5)




Examples




Examples

Slash(µ, φ, ζ) (Kafadar, 1988)

g(u) ∝ IGF(

ζ+12,u2

)

,

where ζ > 0 is the shape parameter, IGF(a, x) is the incompletegamma function for a > 0 and x ≥ 0.



Normal and slash distributions0

.00

.10

.20

.30

.4

y

f(y)

N(0,1)

Slash(0,1,1)

0.0

0.1

0.2

0.3

0.4

y

f(y)

N(0,1)

Slash(0,1,3)




Examples




Examples

Sinh-normal(µ, φ, ζ)(Rieck and Nedelman, 1991)

g(u) ∝ cosh(u12 ) exp

[

− 2ζ2 sinh2(u

12 )

]

,

where ζ > 0 is the shape parameter. The log-BS (Leiva et al.2007) is a particular case for φ = 4.




Examples




Examples

Sinh-t(µ, φ, ζ1, ζ2)(Días-Garcia and Leiva, 2005)

g(u) ∝ cosh(u12 )[

ζ2ζ21 + 4 sinh2(u

12 )]−

ζ2+12

,

where ζ1 > 0 is the shape parameter and ζ2 denotes the degreesof freedom. The log-BS-t (Barros et al. 2008) is a particular casefor φ = 4.



Normal and sinh-normal distributions0

.00

.20

.40

.60

.8

y

f(y)

N(0,1)

SN(0,1,1)

0.0

0.1

0.2

0.3

0.4

y

f(y)

N(0,1)

SN(0,1,3)



Semi-parametric symmetric models

Description

We will now consider the following partially linear model:




Description


yi = x⊤

i β + f (ti) + ǫi




Description


yi = x⊤

i β + f (ti) + ǫi

= x⊤

i β + N⊤

i τ + ǫi ,




Description


yi = x⊤

i β + f (ti) + ǫi

= x⊤

i β + N⊤

i τ + ǫi ,


β = (β1, . . . , βp)⊤, f (ti) = N⊤

i τ is a B-spline, τ = (τ1, . . . , τq)⊤ and ǫi

iid∼S(0, φ), for i = 1, . . . , n.




Objective function

The penalized log-likelihood function is given by




Objective function


Lp(θ, φ, λ) = L(θ, φ)− 12λτ⊤Kτ ,




Objective function


Lp(θ, φ, λ) = L(θ, φ)− 12λτ⊤Kτ ,

where

L(θ, φ) = −n2

logφ+n

∑

i=1

log{g(ui)},




Objective function


Lp(θ, φ, λ) = L(θ, φ)− 12λτ⊤Kτ ,

where

L(θ, φ) = −n2

logφ+n

∑

i=1

log{g(ui)},

θ = (β⊤, τ⊤)⊤, ui = (yi − µi)2/φ, λ is the smoothing parameter and K

is a positive definite matrix.




Iterative process

For λ fixed one has the iterative process:




Iterative process


given starting values β(0), τ (0) and φ(0);




Iterative process


given starting values β(0), τ (0) and φ(0);back-fitting algorithm (Ibacache-Pulgar el al.,2013; Vanegas andPaula, 2015c,d):




Iterative process



β(r+1) = (X⊤D(r)v X)−1X⊤D(r)

v {y − Nτ (r)}




Iterative process



β(r+1) = (X⊤D(r)v X)−1X⊤D(r)

v {y − Nτ (r)}τ (r+1) = (N⊤D(r)

v N + φ(r)λK)−1N⊤D(r)v {y − Xβ(r+1)}




Iterative process



β(r+1) = (X⊤D(r)v X)−1X⊤D(r)

v {y − Nτ (r)}τ (r+1) = (N⊤D(r)

v N + φ(r)λK)−1N⊤D(r)v {y − Xβ(r+1)}

φ(r+1) = 1n{y − Xβ(r+1) − Nτ (r+1)}⊤D(r)

v {y − Xβ(r+1) − Nτ (r+1)},




Iterative process



β(r+1) = (X⊤D(r)v X)−1X⊤D(r)

v {y − Nτ (r)}τ (r+1) = (N⊤D(r)

v N + φ(r)λK)−1N⊤D(r)v {y − Xβ(r+1)}

φ(r+1) = 1n{y − Xβ(r+1) − Nτ (r+1)}⊤D(r)

v {y − Xβ(r+1) − Nτ (r+1)},

for r = 0, 1, 2, . . ., where Dv = diag{v1, . . . , vn} with vi > 0 beingweights.




Estimation of the extra parameters

The extra parameters ζ1 and ζ2 are estimated by minimizing thefunction






Υ = n−1n

∑

i=1

∣

∣

∣Φ−1[Fz(z

(i))]− υ

(i)∣

∣

∣,






Υ = n−1n

∑

i=1

∣

∣

∣Φ−1[Fz(z

(i))]− υ

(i)∣

∣

∣,

where Fz(·) is the cumulative distribution function of the S(0, 1), z(i)

isthe i-th order statistic of z1, . . . , zn with zi = (yi − µi)/

√φ, i = 1, . . . , n,

and υ(i)

is the expectation of the i-th order statistic in a sample of size nof the standard normal distribution.





From the convergence of the back-fitting algorithm one has that






f = Nτ






f = Nτ

= N(N⊤Dv N + φλK)−1N⊤Dv{y − Xβ}






f = Nτ

= N(N⊤Dv N + φλK)−1N⊤Dv{y − Xβ}= H(λ){y − Xβ}.






f = Nτ


Then, from Hastie and Tibshirani (1990) one may take






f = Nτ



df(λ) = tr{H(λ)}






f = Nτ



df(λ) = tr{H(λ)}= tr{N⊤Dv N(N⊤Dv N + φλK)−1}.




Model selection


AIC(λ) = −2L(θ, φ) + 2{p + df(λ) + 1};




Model selection


AIC(λ) = −2L(θ, φ) + 2{p + df(λ) + 1};

BIC(λ) = −2L(θ φ) + log(n){p + df(λ) + 1},




Model selection


AIC(λ) = −2L(θ, φ) + 2{p + df(λ) + 1};

BIC(λ) = −2L(θ φ) + log(n){p + df(λ) + 1},

for given λ.




Inference

One has for large-sample the following variance-covariance matrix:




Inference


Var(θ) =φ

4dg

{

Z⊤Z +φλ

4dgM}−1

(Z⊤Z){

Z⊤Z +φλ

4dgM}−1

,




Inference


Var(θ) =φ

4dg

{

Z⊤Z +φλ

4dgM}−1

(Z⊤Z){

Z⊤Z +φλ

4dgM}−1

,

and

Var(φ) =4φ2

n(4fg − 1),




Inference


Var(θ) =φ

4dg

{

Z⊤Z +φλ

4dgM}−1

(Z⊤Z){

Z⊤Z +φλ

4dgM}−1

,

and

Var(φ) =4φ2

n(4fg − 1),

where Z = [X N], M = diag{0,K}, dg = E{W 2g (υ

2)υ2} andfg = E{W 2

g (υ2)υ4} with υ ∼ S(0, 1).




Inference


Var(θ) =φ

4dg

{

Z⊤Z +φλ

4dgM}−1

(Z⊤Z){

Z⊤Z +φλ

4dgM}−1

,

and

Var(φ) =4φ2

n(4fg − 1),

where Z = [X N], M = diag{0,K}, dg = E{W 2g (υ

2)υ2} andfg = E{W 2

g (υ2)υ4} with υ ∼ S(0, 1).

Additional assumption: supt

|f (t)− f (t)| P−−−→n→∞

0.




Residuals




Residualsquantile residual

qi = Φ−1[Fz(z(i))].





qi = Φ−1[Fz(z(i))].

mean deviance residual

tµ(zi) = sign(zi)[

di(µ|φ)]

12,

where di(µ|φ) is the the i-th log-likelihood difference given φ.





qi = Φ−1[Fz(z(i))].

mean deviance residual

tµ(zi) = sign(zi)[

di(µ|φ)]

12,

where di(µ|φ) is the the i-th log-likelihood difference given φ.

dispersion deviance residual

tφ(zi) = sign(zi)[

di(φ|µ)]

12,

where di(φ|µ) is the the i-th log-likelihood difference given µ.




Sensitivity analysis

In order to assess the influence of small perturbations in the model ordata on the parameter estimates we may apply the Local InfluenceApproach (Cook, 1986; Poon and Poon, 1999).





In order to assess the influence of small perturbations in the model ordata on the parameter estimates we may apply the Local InfluenceApproach (Cook, 1986; Poon and Poon, 1999).For example, one maystudy the conformal curvature

Bℓ(θ) =|ℓ⊤∆⊤(−Lθθ

p )−1∆ℓ|

√

tr(∆⊤(−Lθθp )−1∆)2

,





In order to assess the influence of small perturbations in the model ordata on the parameter estimates we may apply the Local InfluenceApproach (Cook, 1986; Poon and Poon, 1999).For example, one maystudy the conformal curvature

Bℓ(θ) =|ℓ⊤∆⊤(−Lθθ

p )−1∆ℓ|

√

tr(∆⊤(−Lθθp )−1∆)2

,

in the unitary direction ℓ, where 0 ≤ Bℓ(θ) ≤ 1, −Lθθp denotes the

observed information matrix and ∆ depends on the perturbationscheme.


Boston housing data

Outline

1 Examples

2 Defining f (x)



5 Packages in R

6 Voltage drop data






12 Bibliography


Boston housing data

Alternative models

Description




Boston housing data

Alternative models

Description



where ǫiiid∼ Slash(0, φ, ζ) for i = 1, . . . , 506, with f (·) being a

continuous, smooth and nonparametric function.


Boston housing data

Choosing the Slash extra parameter

1.0 1.2 1.4 1.6 1.8 2.0

0.04

00.

045

0.05

00.

055

0.06

0

η

Υ(η)

1.0 1.2 1.4 1.6 1.8 2.0

−107

−106

−105

−104

−103

η

−2*lo

g−Lik

eliho

od


Boston housing data

> require(ssym)> require(MASS)> fit2.boston= ssym.l(log(medv) ~ nox + psp(lstat), xi=2,data=Boston, family="Slash")> extra.parameter(fit2.boston,1,2)> summary(fit3.boston)

Family: Slash ( 1.35 )Sample size: 506Quantile of the Weights0% 25% 50% 75% 100%

0.17 1.23 1.36 1.41 1.42


Estimate Std.Err z-value Pr(>|z|)(Intercept) 3.14287 0.0576 54.5243 < 2e-16 ***nox -0.16499 0.0978 -1.6865 0.09169.

******** Nonparametric component


Boston housing data

Smooth.param Basis.dimen d.f. Statistic p-valuepsp(lstat) 10.1 11.00 8.17 882.2 <2e-16 ***

**** Deviance: 675.62



**** Deviance: 669.41



> envelope(fit3.boston)> plot(residuals(fit3.boston)$mu, fit3.boston$weights,xlab="Mean deviance residual",ylab="Weight")> np.graph(fit3.boston,which=1,xlab="Lstat",ylab="Nonparametric estimate")


Boston housing data


−3 −2 −1 0 1 2 3

−4−2

02

4

Quantile N(0,1)

Mea

n de

vian

ce r

esid

ual

−3 −2 −1 0 1 2 3

−4−2

02

4Quantile N(0,1)

Dis

pers

ion

devi

ance

res

idua

l


Boston housing data

Weight versus residual

−3 −2 −1 0 1 2 3

0.2

0.4

0.6

0.8

1.0

1.2

1.4

Mean deviance residual

Wei

ght


Boston housing data

f(Lstat) 95% confidence band

10 20 30

−0.5

0.0

0.5

1.0

lstat

Non

para

met

ric e

stim

ate

10 20 30

−0.5

0.0

0.5

1.0

lstat

Non

para

met

ric e

stim

ate


Extensions available in the library ssym

Outline

1 Examples

2 Defining f (x)



5 Packages in R

6 Voltage drop data






12 Bibliography



Extensions

Symmetric additive models

yi = x⊤

i β + fµ1(ti1) + · · ·+ fµr (tir ) + ǫi ,



Extensions

Symmetric additive models

yi = x⊤

i β + fµ1(ti1) + · · ·+ fµr (tir ) + ǫi ,

where fµj (t), for j = 1, . . . , r , are continuous, smooth and

nonparametric functions and ǫiiid∼ S(0, φ), for i = 1, . . . , n.



Extensions

Symmetric heteroscedastic additive models



Extensions


yi = x⊤

i β + fµ1(ai1) + · · ·+ fµr (air ) + ǫi ,



Extensions


yi = x⊤

i β + fµ1(ai1) + · · ·+ fµr (air ) + ǫi ,

ǫiind∼ S(0, φi),



Extensions


yi = x⊤

i β + fµ1(ai1) + · · ·+ fµr (air ) + ǫi ,


log(φi) = z⊤i γ + fφ1(bi1) + · · ·+ fφs(bis),



Extensions


yi = x⊤

i β + fµ1(ai1) + · · ·+ fµr (air ) + ǫi ,



where fµj (a) and fφk (b), for j = 1, . . . , r and k = 1, . . . , s, arecontinuous, smooth and nonparametric functions.



Extensions

Symmetric nonlinear additive models



Extensions


yi = η(x i ;β) + ǫi ,



Extensions


yi = η(x i ;β) + ǫi ,



where η(x i ;β) is a nonlinear function of β and fφk (b), for k = 1, . . . , s,are continuous, smooth and nonparametric functions.



Outline

1 Examples

2 Defining f (x)



5 Packages in R

6 Voltage drop data






12 Bibliography











In this analysis we will only consider the variable TEXTURE that will becompared across time among the 5 snack types.



Mean profiles

5 10 15 20

4050

6070

80

Weeks

Text

ure

ABCDE



Variation coefficient profiles

5 10 15 20

0.05

0.10

0.15

0.20

0.25

0.30

0.35

Weeks

VC

of T

extu

reABCDE



Density of texture and log(texture)

20 40 60 80 100 140

0.00

00.

010

0.02

0

Texture

Den

sity

3.5 4.0 4.5 5.00.

00.

40.

81.

2Log(texture)

Den

sity



Alternative models

Description

We will consider the following semi-parametric model:



Alternative models

Description


log(textureijk ) = β0 + βi + fµ(weekj) + ǫijk ,



Alternative models

Description



ǫijkind∼ Power-exponential(0, φij , ζ),



Alternative models

Description




log(φij) = γ0 + γi + fφ(weekj),



Alternative models

Description




log(φij) = γ0 + γi + fφ(weekj),

for i = 1(A), 2(B), 3(C), 4(D), 5(E), j = 2, 4, . . . , 20 and k = 1, . . . , 15,β0 + βi (β1 = 0) and γ0 + γi (γ1 = 0) denote the snack effects whereasfµ(·) and fφ(·) are continuous, smooth and nonparametric functions.



Choosing the power exponential extra parameter

0.0 0.2 0.4 0.6

0.03

0.04

0.05

0.06

η

Υ(η)

0.0 0.2 0.4 0.6

−235

−230

−225

−220

−215

−210

η

−2*lo

g−Lik

eliho

od



> require(ssym)> require(MASS)> par(mfrow=c(1,2))> plot(density(texture),xlab="Texture", main="")> plot(density(log(texture)),xlab="Log(texture)", main="")> fit1.snacks = ssym.l(log(texture) ~type + ncs(week)| type+ ncs(week), xi=0.4, family="Powerexp")> extra.parameter(fit1.snacks,0,1)

> fit2.snacks = ssym.l(log(texture) ~type + ncs(week)| type+ ncs(week), xi=0.11, family="Powerexp")> summary(fit2.snacks)

Family: Powerexp ( 0.11 )Sample size: 750Quantile of the Weights0% 25% 50% 75% 100%

0.82 1.04 1.16 1.32 4.28




Estimate Std.Err z-value Pr(>|z|)(Intercept) 4.155072 0.0229 181.7439 < 2.2e-16 ***type2 -0.171145 0.0279 -6.1407 8.215e-10 ***type3 -0.088709 0.0311 -2.8544 0.004312 **type4 -0.247158 0.0258 -9.5731 < 2.2e-16 ***type5 -0.258958 0.0266 -9.7515 < 2.2e-16 *********** Nonparametric component

Smooth.param Basis.dimen d.f. Statistic p-valuencs(week) 59.45 9.000 8.626 347.3 <2e-16 ***

**** Deviance: 832.5


Estimate Std.Err z-value Pr(>|z|)(Intercept) -2.71029 0.1217 -22.2784 < 2.2e-16 ***type2 -0.70798 0.1720 -4.1150 3.871e-05 ***type3 -0.15554 0.1720 -0.9041 0.366type4 -1.26983 0.1720 -7.3807 1.574e-13 ***type5 -1.03528 0.1720 -6.0174 1.772e-09 *********** Nonparametric component



Smooth.param Basis.dimen d.f. Statistic p-valuencs(week) 12.15 9.00 6.75 25.6 0.00238 **

**** Deviance: 970.64



> envelope(fit1.snacks)> np.graph(fit1.snacks,which=1, exp=TRUE,ylab="Nonparametric estimate", xlab="Week")> np.graph(fit1.snacks,which=2, exp=TRUE,ylab="Nonparametric estimate", xlab="Week")




−3 −2 −1 0 1 2 3

−20

24

Quantile N(0,1)

Mea

n de

vian

ce r

esid

ual

−3 −2 −1 0 1 2 3

−4−2

02

4Quantile N(0,1)

DeD

ispe

rsio

n de

vian

ce r

esid

ual



exp(fµ(week)) 95% confidence band

5 10 15 20

0.8

0.9

1.0

1.1

1.2

1.3

Week

Non

para

met

ric e

stim

ate

5 10 15 20

0.8

0.9

1.0

1.1

1.2

1.3

Week

Non

para

met

ric e

stim

ate



exp(fφ(week)) 95% confidence band

5 10 15 20

0.5

1.0

1.5

Week

Non

para

met

ric e

stim

ate

5 10 15 20

0.5

1.0

1.5

Week

Non

para

met

ric e

stim

ate


Bibliography

Outline

1 Examples

2 Defining f (x)



5 Packages in R

6 Voltage drop data






12 Bibliography
