2012 COURSE IN NEUROINFORMATICSMARINE BIOLOGICAL LABORATORY
WOODS HOLE, MA
GENERALIZED LINEAR MODELS
Uri EdenSridevi V. SarmaEmery N. Brown
BU Department of Mathematics and Statistics
MIT Dept. of Brain & Cognitive Sciences
MIT Division of Health, Sciences and Technology
Massachusetts General Hospital
August 15, 2012
OBJECTIVES
• Understand the theory of the Generalized Linear Model
• Understand its relation to the General Linear Model
• Understand how model analysis is conducted using the Generalized Linear Model
OUTLINE
• Motivation: The Stimulus-Response Structure of Neuroscience Experiments
• Theory of the Generalized Linear Model (GLM)
• GLM Analysis of the Retinal Neuron Spike Train
• GLM Analysis of Sub-thalamic Nucleus Spike Trains in Parkinson Patients and Healthy Primate
• Summary
Spatial Receptive Fields of Hippocampal Pyramidal Neurons
Spike Histogram
0 7035
70
35
Circular Environment
0 7035
70
35
x1 (cm)x1 (cm)
x 2 (
cm)
x 2 (
cm)
Learning in Hippocampal Neurons
• Single cell recording in monkey hippocampus• Trial and error learning of association between picture
and response
Wirth et al. Science 2003Smith et al. Neural Computation 2003Smith et al. Journal of Neuroscience, 2004
Most neuroscience experiments are stimulus-response
Need a general framework that allows us to relate the stimuli in neuroscience experiments to their responses…
Experiment Stimulus/Covariate
Response
Free-behaving Rat
Position Spikes
Monkey Learning
Association Incorrect/Correct Responses, or Spikes
Goldfish Retina Constant, Spiking History
Spikes
fMRI Visual/Motor BOLD
Actually-we want to do regression for all types of data so
that we get the formal inference structure of ML (optimality, GOF, T tests on coeffs)!
System to Study
X Y?
Stimulus(predictable)
Response
Noise (unpredictable)
How can we use numerical data to model how X ‘impacts’ Y?
From Data to Model
• Data: ),(,...),,(),,( 2211 nn YXYXYX
iscalarY
iRX
i
pi
pnn
Xn
X
X
Y
Y
Y
Y
12
11
2
1
Constant parameter vector typically estimated from data
• Notation:
s'iX
s'iY are random variables
are typically non-random variables (covariates)
From Linear Models to GLMs
• Linear regression models of the form:
are useful for relating Gaussian continuous valued observations to a set of covariates.
Count data:
eg. number of arrivals in time T (poisson)
Binary data:
eg. incorrect/correct response of trial (bernoulli)
~ (0, )N ),(~ XNY XY
• Generalized linear models extend a simple class of models to other data types. In particular,
• Many types of data cannot be described by a Gaussian additive noise model.
The Linear Model: A Different Perspective
1
( ) ( | ) exp{ ( ) ( ) ( ) ( )}K
k kk
L f y T y C H y D
1. Y is Gaussian which belongs to the exponential family of distributions:
2. The likelihood function for the exponential family is:
)()()()(exp)|( DyHCyTyf
Canonical Link function
Data and Parametersare multiplicatively separable!
The Linear Model: A Different Perspective cont…
12
2
1
2 2
1
1( ) exp{ ( ) }
2 2
1 exp{( { log(2 )}
2
K
k kk
K
k k k kk
L y
y y
Gaussian Data
The canonical link function is then
1
( ) ( | ) exp{ ( ) ( ) ( ) ( )}K
k kk
L f y T y C H y D
jk
p
jjkkk xXYEC ,
1
modellinear][)()(
3. The likelihood for the Gaussian and its canonical link for the linear model:
The Generalized Linear Model
1
( ) ( | ) exp{ ( ) ( ) ( ) ( )}K
k kk
L f y T y C H y D
2. The canonical link function is a linear function of the parameters
01
( )J
j jj
C x
1. Y belongs to the exponential family of distributions
All the probability models we have studied, Bernoulli, binomial, Poisson, Gaussian, gamma, exponential, inverse Gaussian, beta belong to the exponential family!
The Exponential Family
1
1
exp{ }( )
!
exp{ log( ) log( !) }
kyKk k
k k
K
k k k kk
Ly
y y
Poisson Data
The canonical link function is
1
( ) log( )J
k j kjj
C x
1
( ) ( | ) exp{ ( ) ( ) ( ) ( )}K
k kk
L f y T y C H y D
The Exponential Family
1
( ) ( | ) exp{ ( ) ( ) ( ) ( )}K
k kk
L f y T y C H y D
1
1
1
1
( ) (1 )
exp{ log( ) (1 ) log(1 )}
exp{ log( ) log(1 )}1
k k
Ky y
k kk
K
k k k kk
Kk
k kk k
L p p
y p y p
py p
p
Bernoulli Data
The canonical link function
1
( ) log( )1
Jk
j kjjk
pC x
p
1
log( )1
Jk
j kjjk
px
p
1
log( )J
k j kjj
x
1
J
k j kjj
x
Link EquationModel
Gaussian
Poisson
Bernoulli
Summary of Generalized Linear Models
Model Goodness-of-Fit and Analysis
- log (y ) - log ( )2 f | = 2
B. Akaike’s Information Criterion:
ˆ- log (y ) +ML2 f | 2p
For maximum likelihood estimates it measures the trade-offbetween maximizing the likelihood ( minimizing )ˆ- log (y )ML2 f |
and the numbers of parameters p in the model.
A. Deviance (Analog of the Residual Sum of Squares):
C. Standard Errors of the Coefficients and t-testst-statistic = Coefficient Estimate/SE
- log (y )2 f |
where in the Gaussian case
Properties of the GLM
• Convex likelihood surface• Estimators asymptotically have minimum MSE• All model estimation is efficient: iterative reweighted least
squares
Stochastic Models
Linear Regression
(output | input,history)p
Generalized Linear Models (GLM)
Neural Spiking Models
GLM Neural Models
• By selecting an appropriate set of basis functions we can capture arbitrary functional relations.
• Analysis of relative contributions of components to spiking
01
1
, ,1 1
log( ) (Extrinsic Covariates)
(Spiking History)
(Ensemble Activity)
I
k i ii
J
j jj
K C
k c k ck c
f
g
h
Truccolo W, Eden UT, Fellows MR, Donoghue JP, Brown EN. (2004) J. Neurophys 93:1074-1089
Summary of GLM Theory
• Generalization of the Gaussian Linear Model (McCulloch and Nelder)
• Can be used for any probability models in the exponential family.
• Is a maximum likelihood analysis and all its optimality properties.
• An efficient computational framework using iteratively reweighted least squares.
• GLM is available as a toolbox in all major statistics packages and Matlab.
Case 1: An Analysis of the Spiking Activity of Retinal Neurons in Culture
Retinal neurons are grown in culture under constant light and environmental conditions. The spontaneous spiking activity of these neurons is recorded. The objective is to develop a statistical model which accurately describes the stochastic structure of this activity.
(Iygengar and Liu, 1997)
Discrete Time Spike Train Data
dN1 dN2 dN3 dN4 dN5 dN6 dN7
0 0 1 0 0 0 1
dNk is the spike indicator function in interval k
is the intensity of spiking at time k, which in the limit is given by
k
0
Pr(Spike in ( , ) | )( | ) lim t
tt
t t t Ht H
t
Point Process is Exponential
1
(Spike Train | ) exp( log( ( | )) ( | ) )T
t u uu
L t H N u H t
Data Component
Link Function
01
exporder
k i k ii
n
Question: Can we construct a history-dependent firing rate model to describe the retinal neuron spiking activity?
How do we pick a model order?
The ISI distribution models we construct were
(ISI | ) (ISI|last spike time)tp H p
Poisson Model:
We use GLM to build history-dependent model as
0 10 20 30 40 50-0.04
-0.02
0
0.02
0.04
0.06
0.08
0.1
Model Order
Pa
rtia
l Co
rre
lati
on
Co
eff
icie
nt
Partial Correlogram-Visualization
Order=14?
Generalized Linear Model Analysis
Exploratory Analysis:
We plotted the data as a time-series.
We computed the partial autocorrelation coefficients of order up to 50.
Confirmatory Analysis:
We fit GLM models of orders varying from 1 to 120 (msec).
We computed the deviance, AIC, KS plots and the significance of the coefficients.
GLM Coefficients
Lag (msec)
Coe
ffic
ient
Val
ue
Coefficient values
Stat. significant coeffs.
Order=14?
Absolute Goodness-of-Fit
1
( | )i
i
t
i utz u H du
Time Rescale
nttt ..., 21 nzzz ..., 21
Time-Rescaling Theorem: zi’s are i.i.d. exponential rate 1
Kolmogorov-Smirnov Analysis
KS Distance is the maximum distance between an empirical and a theoretical probability model.
tan max | ( ) ( ) |dis ce nKS F u F u
Kolmogorov-Smirnov (KS) Plot:
EC
DF
(zi)
CDF(exp(1))
Graphical measure of goodness-of-fit, based on the time rescaling theorem, for comparing an empirical and model cumulative distribution function. If the model is correct, then the rescaled ISI,s are independent, identically distributed exponential 1 (uniform) random variables whose ordered quantiles should produce a 45° line.
KS distance
AIC and KS Statistics
Poisson-GLM
1 14 50
6589 5931 5892
0.2525 0.0657 0.0462
Order
AIC
KS
Exp Gamma Inv. Gauss.
0.2330 0.2171 0.1063KS Statistic
Parametric Models:
Inferences and Conclusions
• Iyengar and Liu showed that a generalized inverse Gaussian model described these data well.
• The fit of history-dependent GLM model improves appreciably on the fits of the exponential, gamma and inverse Gaussian models, most notably in terms of KS plots.
• Our analysis shows that the GLM model describes the essential stochastic features in the data. There is a significant history dependence in the retinal neural spiking data extending back 14 msec.
• There is another effect going back approximately 50 msec.
• The shorter time-scale phenomena may reflect intrinsic dynamics of the individual neuron whereas the longer time-scale effects may also include network dynamics.
Remarks
1. Only 14 parameters are used to fit ~ 30,000 data points!
2. This type of strong history dependent effect is something we have seen in neurons from a number of different brain regions, animal models and experimental protocols. It was all simply described by GLM fitting.
Truccolo W, Eden UT, Fellow M, Donoghue JD, Brown EN. A point process framework for relating neural spiking activity to spiking history, neural ensemble and covariate effects. Journal of Neurophysiology, 2005, 93:1074-1089.
Kass RE, Ventura V, Brown EN. Statistical issues in the analysis of neuronal data. Journal of Neurophysiology, 2005, 94: 8-25.
Case 2: An Analysis of the Spiking Activity of STN Neurons in Parkinson Patients and Healthy Primate
The spiking activity of sub-thalamic neurons of basal ganglia in Parkinson patients and healthy primate is recorded under identical experimental conditions. Subjects execute a center-out directed hand-movement task.
*Sarma, Cheng, Eden, Hu, Williams, Brown, Eskandar, 2008
Neurophysiological Data
Single Neuronal Recordings from STN
0
1
time
1 Primate96 Neurons868 trials
8 Patients3-15 neurons/patient24-470 trials/patient
Behavioral Task
Grey ArrayAppears
500-1000 ms
Target cue
500-1000 ms
“U”
“R”
“D”
“L”
Go cue
500-1000 ms
Fixation
500 ms
Move
“U”
“R”
“D”
“L”
Reach & Hold
100 ms
TC GC MV
t350 ms
GA
700 ms
GA TC GC MV
mkmk
mmjkjk
jj
dddk nnkI 10:)910(
14
1,)1(:)(
10
1,
4
1,, )(exp
Effect of spiking history
long term:Network
dynamics
Period-specific movement planning, execution (stimulus) effect
GLM for STN
Compute maximum likelihood estimates for parameters using glmfit.m
short term:Intrinsic
dynamics
HEALTHY PRIMATE PD PATIENT
50 Hz
50% oscillations (10-30 Hz)
40% bursting
8% directionalselectivity
35% directionalselectivity
30 Hz
15% oscillations (10-30 Hz)
12% bursting
250 msec 250 msec
rate function
spike train
0.35
“Parkinsonian” Motor Symptoms akinesia/bradykinesia
resting tremor
rigidity
“Parkinsonian” STN Neural Symptoms 10-30 Hz oscillations
bursting
directional selectivity
lower firing rate
Population Summary
Summary
• GLM provides a computationally tractable generalization of the Gaussian linear model to non-Gaussian regression models.
• Estimation is carried out using maximum likelihood. This analysis has all the properties of maximum likelihood.
• AIC, deviance and parameter standard errors provide measures of goodness-of-fit and an inference framework analogous to regression.
• Can be applied to other exponential family models.
• Non-canonical link functions can also be used.
• GLM is a standard tool in Matlab, Minitab, R, S, SAS, Splus, and SPSS.
Top Related