1
NO
NPA
RA
ME
TR
ICR
AN
DO
ME
FF
EC
TS
MO
DE
LSA
ND
LIKE
LIHO
OD
RA
TIO
TE
ST
S
Oct11,2002
David
Ruppert
CornellU
niversity
ww
w.orie.cornell.edu/̃
davidr
(These
transparenciesand
preprintsavailable
—
linkto
“RecentTalks”
and“R
ecentPapers”)
Work
donejointly
with
Ciprian
Crainiceanu,C
ornellUniversity
2
OU
TLIN
E�
Sm
oothingcan
bedone
usingstandard
mixed
models
software
because
�
Splines
canbe
viewed
asB
LUP
sin
mixed
models
�
This
random-effects
splinem
odelextendsto:
�
Sem
iparametric
models
(allows
parametric
submodels)
�
Longitudinaldata
�
nestedfam
iliesofcurves
3
EX
AM
PLE
�R
ickC
anfieldand
Chuck
Henderson,
Jr.at
Cornell
are
working
oneffects
oflow
-levellead
exposureon
IQof
children.
�
They
havea
mixed
model
butthe
dose-responsecurve
shouldbe
modeled
nonparametrically.
4
EX
AM
PLE
—C
ON
T�
They
askedS
AS
isa
”PR
OC
GA
MM
IXE
D”w
ouldbe
avail-
ablesom
eday.
�
shortanswer
was
”no”
�
Then,
theyfound
Matt
Wand’s
work
andthen
contacted
me.
�
Now
theyknow
thatGA
MM
IXE
D�
GLM
MIX
ED
.
�
SA
Shas
GA
MM
IXE
Dand
doesnotknow
it!
5
TE
ST
ING
INT
HIS
FR
AM
EW
OR
K�
Inprinciple,likelihood
ratiotests
(LRT
s)could
beused
to
testforeffects
ofinterest
�
E.g.,hypothesis
thatacurve
islinearorthatan
effectis
zero��
avariance
component
(andpossibly
afixed
effect)is
zero
�
allows
anelegant,unified
theory
6
TE
ST
ING
—C
ON
T�
How
ever,thedistribution
theoryofLR
Ts
iscom
plex:�
thenullhypothesis
ison
theboundary
ofthe
parame-
terspace,
so“standard
theory”suggests
chi-squared
mixtures
asthe
asympototic
distribution.
�
butstandardasym
ptoticsdo
notapplybecause
ofcor-
relation
�
forthe
caseofone
variancecom
ponent,we
nowhave
asymptotics
thatdoapply
7
UN
IVA
RIA
TE
NO
NPA
RA
ME
TR
ICR
EG
RE
SS
ION
�m
odel
��������� ��
�
letting�
bea
spline
� � �����
� �� �
�� �� ��
���� ��
��������� ��
willbe
treatedas
“randomeffects”
�
assume
theyare
iid�! �#"$% �
�
sizeof"
$%
controlsthe
amountofshrinkage
orsm
ooth-
ing.
8
NO
NPA
RA
ME
TR
ICM
OD
ELS
FO
RLO
NG
ITU
DIN
AL
DA
TA���&
is' thobservation
on( thsubject
�
considerthe
nonparametric
model
��& �� �& � �� �& � ��&
�
modelthe
“population”curve�
asa
spline:
� � �����
� �� �
�� �� ��
���� ��
�
modelthe
“( thsubject”
curve�� asanother
spline:
�� � ��� �
� )* �+� �
�� �� � * �+�
���� ��
9
PO
PU
LAT
ION
CU
RV
E�
Recall:
� � �����
� �� �
�� �� ��
���� ��
��� ��������
willbe
treatedas
“fixedeffects”
��������� ��
willbe
treatedas
“randomeffects”
�
assume
theyare
iid�! �#"$%,.- �
(/
=“population”)
�
thisassum
ptioncan
beview
edas
aB
ayesianm
odel
�
somew
hatdifferentthatusualinterpretationofrandom
effects
10
SU
BJE
CT
CU
RV
ES
�R
ecall:
�� � ��� �
� )* �+� �
�� �� � * �+�
���� ��
�)* �+� �����
� )* �+�
willbe
treatedas
“randomeffects”
�
assume
theyare
iid�! �#"$0 �
�
thisis
atypical“random
effects”assum
ption
�� * �+� ������ � * �+�
willalso
betreated
as“random
effects”
�
assume
theyare
iid�! �#"$%,21 �
(3
=“subject”)
11
NU
LLH
YP
OT
HE
SE
SO
FIN
TE
RE
ST
�R
ecall:
�� � ��� �
� )* �+� �
�� �� � * �+�
���� ��
�"$0 �"$%,21 �
��
nosubjecteffects
�"$%,21 �
��
subjecteffectsare4 th
degreepolynom
ials
12
RE
LAT
ED
WO
RK
�B
rumback
andR
ice(1998)
�
Zhang,Lin,R
az,andS
owers
(1998)
�
Linand
Zhang
(1999)
�
Rice
andW
u(2001)
See
referencesatend.
13
BA
LAN
CE
1-WA
YA
NO
VA
�m
odel:5�& �6 �� ��& �( �7�������8
and' �7������:9 �
and
���;� �#"$% �
�
nullhypothesis:
<�>="$% �
�
14
�If8
?@
with9
fixed,then
�ABCDFEG� ?�$IH$� �$ H$� �
(Selfand
Liang,1987;Stram
andLee,1994)
�
This
isthe
iidcase
ifwe
takethe
subjectsas
“observa-
tions”
Note:
The
equivalentfixed
effectshypothesis
is�� �JJJ�
�K �
.
�
Then
theLR
testisequivalentto
theF
-test
� �ABCD EG� ?H$K�L
�
under<�
15
�If9
?@
with8
fixed,then
�ABCD EG� �8
MONPK L
� �7 �BCD NPK L
� �QORS>T
UVXWY[Z�\ �
and
�ABCD G EG� � 8
�7�] NK L� �7 �BCD NK L��^ R
S TVXWY Z �\ �
where N
K L� ;
_a`VXWYK L�
and NPK�L
� ;_a`V WYK �
(Crainiceanu
andR
uppert,2002)
16
010
2030
4050
6070
8090
1000.5
0.55
0.6
0.65
0.7
0.75
0.8
0.85
0.9
Num
ber of levels
ProbabilityM
LR
EM
L
1-Way
AN
OV
A:Bb
cdef/gh ] BCDFEG� � ^
17
Pinheiro
andB
ates(2000,p.87)
�sim
ulatedthe
LRT
�
foundsom
eem
piricalevidencethatthe�iH$� �
iH$� m
ixture
isbetter
replacedby4� H
$� 7 �4�� H $�
for4�kj �i .
These
theoreticalresultshelp
explaintheir
findings.
18
03
90 3 9
−plot for χ21 versus the distributions in equation (9), K
=3, 5, 20
Q0.99 =6.63
6.63
5.70
20
5
3
1-Way
AN
OV
A:asym
pt.nulldistofR
LR,given
RLRj
0
19
03
90 3 9
−plot for χ21 versus the distributions in equation (10), K
=3, 5, 20
Q0.99 =6.63
6.63
4.99
20
5
3
1-Way
AN
OV
A:asym
pt.nulldistofLR
,givenLRj
0
20
PE
NA
LIZE
DS
PLIN
ES�
model:
��l�m
�� �� �
�
nullhypothesis:
<�>=m
�� ��� �� ��� �� � � Ln � L
n�op �
�
alternativehypothesis:
<q =m � ��� �� ��� �� �
����� ���
���� �� �
21
�notation:
r �s �� �������� � �������� ��t u
�
penalizedleastsquares:
minim
ized� �� ] �� �m
� v r�^ $ wr uxr�
with
x �yzz{
| L� }~~� �
22
�sam
eas
BLU
Pin
alinear
mixed
modelw
ith
Cov��� �"
$% |
and
w � "$�
"$%
(Brum
back,Ruppert,and
Wand,1999)
23
�new
formofnull:
ifo �
"$% �
or,ifo j
,�� Ln � � �
JJJ��� �
and"$% �
�
24
Exam
ple:(C
rainiceanuand
Ruppert,2002)
�� ’sequally
spaced
�
20equally
spacedknots
�4 �o�
(constantm
eanversus
piecewise
constant
mean)
Then,
Bbcde
f/gh ] BCD G EG� � ^ ���i���
not.5
and
Bbcde
f/gh ] BCD EG� � ^ ���i �i�
not.5
25
010
2030
4050
6070
8090
1000.5
0.55
0.6
0.65
0.7
0.75
0.8
0.85
0.9
0.95 1
Num
ber of knots
ProbabilityM
LR
EM
L
P-splines: Bb
cdef/gh ] BCD EG� � ^
26
OR
TH
OG
ON
ALIZ
AT
ION
�one
canapply
Gram
-Schm
idttothe
“designm
atrix”�
power
functionsare
replacedby
orthogonalpolynomi-
als
�
“Plus
functions”are
replacedby
splinebasis
functions
thatareorthogonalto
polynomials
�
The
asymptotics
oftheLR
Tare
changedby
thisreparam
etriza-
tion
27
�A
symptotics
areessentially
thesam
eas
for1-way
AN
OV
A
with
�8
( �#
levels)= �
( �
#knots)
+1
�
E.g.,5
levelsis
like4
knots
28
010
2030
4050
6070
8090
1000.5
0.55
0.6
0.65
0.7
0.75
0.8
0.85
0.9
0.95 1
Num
ber of knots
Probability
ML
RE
ML
P-splines
010
2030
4050
6070
8090
1000.5
0.55
0.6
0.65
0.7
0.75
0.8
0.85
0.9
Num
ber of levels
Probability
ML
RE
ML
Orthogonalized
=1-w
ayA
NO
VA
Asym
ptoticnullprobabilities
thatlog-LRis
zero
29
Quantiles of the asym
ptotic distribution (n=∞
)
Quantiles of distributions
n=50n=100n= ∞0.5:0.5 m
ixture
q0.66
q0.95
q0.99
q0.995
1.74
4.20
5.32 0
Com
parisonoffinite-sam
pleand
asymptotic
quantiles
Hypotheses:
lineartrend
versus20-knotlinear
spline
30
Com
parisonofLR
Tw
ithothertests
Reference:
Crainiceanu,
Ruppert,
Aerts,
Claeskens,
and
Wand
(2002,inpreparation)
�
Results
innexttable
arefor
testing
�
constantmean
versus
�
generalalternative
�
piecewise
constantspline,or
�
linearspline
31
�T
hecom
parisonsare
made
with
an
�increasing,
�
concave,and
�
periodic
mean
function,chosenso
thatgoodtests
hadpow
erap-
proximately
0.8
32
�R
-testisfrom
Cantoniand
Hastie
(2002)�
F-testis
asin
Hastie
andT
ibshirani(1990)
�
“C”
means
alternativeis
apiecew
iseconstantfunction
�
“L”m
eansalternative
isa
linearspline
33
�“1”
means
estimate
underalternative
hasD
Fone
greater
thanunder
null
�
“ML”m
eanssm
oothingparam
eterunderalternativeis
cho-
senby
ML
�
“GC
V”
means
smoothing
parameter
underalternative
is
chosenby
GC
V
34
TestA
veragepow
erM
aximum
power
Minim
umP
ower
RLR
T-C0.8885
0.96600.8166
R-G
CV
-L0.8737
0.99100.7188
R-M
L-C0.8615
0.99160.7022
F-M
L-L0.8569
0.87960.8328
R-M
L-L0.8569
0.87960.8328
F-M
L-C0.8534
0.99280.6708
F-G
CV
-L0.8482
0.99460.6634
LRT-L
0.75610.8466
0.6832
F-1-C
0.70870.8442
0.4816
F-1-L
0.67750.9414
0.3012
R-1-L
0.62390.9126
0.1462
R-G
CV
-C0.6144
0.92840.3392
35
Conclusions
�S
tandardasym
ptoticsare,in
general,notsuitable
�
Better
asymptotics
forone
variancecom
ponentare
fea-
sible
�
For
more
thanone
variancecom
ponent,onem
ightneed
touse
simulation
togetp-values
36
References
Brum
back,B
.,and
Rice,
J.,(1998),
Sm
oothingspline
models
forthe
analsysiof
nestedand
crossedsam
plesofcurves,JA
SA
,93,944–961
Brum
back,B
.,R
uppert,D
.,and
Wand,
M.P.,
(1999).C
omm
enton
“Variable
selec-
tionand
functionestim
ationin
additivenonparam
etricregression
usingdata-based
prior”by
Shively,K
ohn,andW
ood,JAS
A,94,794–797.
Cantoni,E
.,and
Hastie,T.J.,2002.
Degrees
offreedom
testsfor
smoothing
splines,
Biom
etrika,89.251–263.
Crainiceanu,C
.M.,and
Ruppert,D
.,(2002),Asym
ptoticdistribution
oflikelihoodratio
testsin
linearm
ixedm
odels,submitted
Crainiceanu,
C.
M.,
Ruppert,
D.,
andV
ogelsang,T.
J.,(2002).
Probability
thatthe
mle
ofa
variancecom
ponentis
zerow
ithapplications
tolikelihood
ratioTests,
manuscript
37
Crainiceanu,
C.
M.,
Ruppert,
D.,
Aerts,
M.,
Claeskens,
G.,
andW
and,M
.,(2002),
Testsofrolynom
ialregressionagainsta
generalalternative,inpreparation.
Hastie,
T.J.,T
ibshirani,R
.,1990.
Generalized
Additive
Models,
London:C
hapman
andH
all.
Lin,X
.,and
Zhang,
D.
(1999),Inference
ingeneralized
additivem
ixedm
odelsby
usingsm
oothingsplines,JR
SS
-B,61,381–400.
Pinheiro,J.,and
Bates,D
.,(2000),Mixed-E
ffectsM
odelsin
Sand
S-P
LUS
,Springer,
New
York.
Rice,J.,and
Wu,C
.,(2001),Nonparam
etricm
ixedeffects
models
forunequally
sam-
plednoisy
curves,”B
iometrics,57,253–259.
Self,S
.,andLiang,K
.,(1987).A
symptotic
propertiesofm
aximum
likelihoodestim
a-
torsand
likelihoodratio
testsunder
non-standardconditions,JA
SA
,82,605–610.
38
Stram
,D
.,Lee,
J.,(1994).
“Variance
Com
ponentsTesting
inthe
LongitudinalMixed
Effects
Model,”
Biom
etrics,50,1171–1177.
Zhang,D
.,Lin,X.,R
az,J.,andS
owers,M
.(1998),Sem
i-parametric
stochasticm
ixed
models
forlongitudinaldata,JA
SA
,93,710–719
Top Related