Top-down coordination of local cortical 1
state during selective attention 2
Jochem van Kempen1*, Marc A. Gieselmann1, Michael Boyd1, Nicholas A. Steinmetz2, Tirin 3 Moore3, Tatiana A. Engel4, Alexander Thiele1* 4
Affiliations: 5
1 Biosciences Institute, Newcastle University, Newcastle upon Tyne, United Kingdom. 6 2 Department of Biological Structure, University of Washington, Seattle, WA, United States. 7 3 Howard Hughes Medical Institute, Stanford University, CA, United States. 8 4 Cold Spring Harbor Laboratory, Cold Spring Harbor, NY, United States. 9
* [email protected], [email protected]. 10
11
Abstract: 12
Spontaneous fluctuations in cortical excitability influence sensory processing and behavior. 13
These fluctuations, long known to reflect global changes in cortical state, were recently 14
found to be modulated locally within a retinotopic map during spatially selective attention. 15
We found that periods of vigorous (On) and faint (Off) spiking activity, the signature of 16
cortical state fluctuations, were coordinated across brain areas along the visual hierarchy 17
and tightly coupled to their retinotopic alignment. During top-down attention, this 18
interareal coordination was enhanced and progressed along the reverse cortical hierarchy. 19
The extent of local state coordination between areas was predictive of behavioral 20
performance. Our results show that cortical state dynamics are shared across brain regions, 21
modulated by cognitive demands and relevant for behavior. 22
23
One Sentence Summary: 24
Interareal coordination of local cortical state is retinotopically precise and progresses in a 25
reverse hierarchical manner during selective attention. 26
27
.CC-BY-NC-ND 4.0 International license(which was not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprintthis version posted March 26, 2020. . https://doi.org/10.1101/2020.03.26.009365doi: bioRxiv preprint
Main Text: 28
Cortical activity is not solely determined by external inputs but reflects ongoing fluctuations 29
in neural excitability referred to as cortical state (Harris and Thiele, 2011; Kohn et al., 2009). 30
Endogenous variability in cortical state shapes sensory responses and influences behavioral 31
performance (Arieli et al., 1996; Gutnisky et al., 2017; McGinley et al., 2015a; Renart and 32
Machens, 2014; Scholvinck et al., 2015). Although these fluctuations were long thought to 33
be a global phenomenon that influences activity throughout the cortex (Harris and Thiele, 34
2011; Lee and Dan, 2012), recent evidence has revealed that signatures of cortical state are 35
modulated locally within the retinotopic map in Macaque V4 during selective attention 36
(Engel et al., 2016). Cortical state fluctuations manifest in periods of vigorous (On) and faint 37
(Off) spiking activity occurring synchronously across cortical laminae. Spatially selective 38
attention directed towards the receptive fields (RFs) of the neural population modulates On-39
Off dynamics by increasing the duration of On episodes (Engel et al., 2016). Thus, cognitive 40
demands that selectively affect targeted retinotopic locations can modulate local signatures 41
of global cortical state fluctuations. However, perception and cognition depend on activity 42
of many areas spanning the cortical hierarchy, which begs the question of whether cortical-43
state dynamics are coordinated across different brain regions during attention, whether this 44
coordination progresses in a top-down or bottom-up manner, and whether it is relevant for 45
behavior. 46
We recorded simultaneously from V1 and V4 using 16-contact laminar electrodes whilst 47
three rhesus macaques performed a feature-based spatial attention task (Fig. 1A). 48
Electrodes were inserted perpendicular to the cortical surface on a daily basis such that RFs 49
overlapped both across all channels within each area and between the two areas (Fig. 1B & 50
Fig. 1C). We characterized On-Off dynamics in each area individually by fitting a Hidden 51
Markov Model (HMM) to the spike counts (10 ms bins) of multiunit activity (MUA) across 52
included channels (supplementary material, Fig. 1D). In line with previous reports for V4 53
(Engel et al., 2016), we found that a 2-phase model was the most parsimonious model for 54
the majority of recordings (V1: 64 out of 77 recordings (83.1%), V4: 73 out of 79 recordings 55
(92.4%), V1 and V4: 57 out of 73 recordings (78.1%), Fig. S1A-D). During these recordings, 56
On-Off dynamics occurred without any obvious periodicity (Fig. S1E). When attention was 57
directed towards the RFs under study, firing rates were higher during both Off and On 58
.CC-BY-NC-ND 4.0 International license(which was not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprintthis version posted March 26, 2020. . https://doi.org/10.1101/2020.03.26.009365doi: bioRxiv preprint
epochs [Wilcoxon signed rank test; V1: Off P = 10-164, On P = 10-87, V4: Off P = 10-184, On P = 59
10-103] (Fig. 1E), On-epoch durations increased in both V1 and V4 [Wilcoxon signed rank test; 60
V1 P = 10-13, V4 P = 10-8] and Off epoch durations increased in V1 but not V4 [Wilcoxon 61
signed rank test; V1: P = 10-8, V4 P = 0.81] (Fig. 1F). Additionally, when attention was 62
directed towards the RFs, altogether more time was spent in an On phase (Fig. S2A) and 63
transitions to an On phase were more likely (Fig. S2B). 64
We examined whether these spontaneous transitions were coordinated across visual areas. 65
We computed cross-correlations between the V1 and V4 time series of On-Off phases (as 66
estimated by the HMMs) during passive fixation (before stimulus onset) and during directed 67
attention (after cue onset). During fixation, V1 and V4 transitions were coordinated but 68
without either area leading/lagging the other [Wilcoxon signed rank test; P = 0.13] (Fig. 2A). 69
During directed attention, the coordination between V1 and V4 was enhanced and On-Off 70
transitions more often occurred in V4 before they were followed in V1, as evident from the 71
skew towards negative values of the V4 relative to V1 transition times [Wilcoxon signed 72
rank test; P < 10-5] (Fig. 2A). The cross-correlation strength and skew was independent of 73
microsaccades (Fig. S3), and the strength was inversely related to the separation between 74
V1 and V4 RFs [r = -0.38, P = 0.004] (Fig. 2B). Thus, the strength of On-Off dynamics 75
coordination between visual areas is tightly coupled to their retinotopic alignment. To 76
further characterize this interareal coordination, we computed average firing rates in V1 77
aligned to On-Off transition times in V4 and vice-versa. In line with transitions being driven 78
in a top-down manner, V1 firing rate changes followed V4 transitions whereas V4 firing rate 79
changes preceded V1 transitions (Fig. 2C). We also analyzed spiking activity simultaneously 80
recorded with 16-contact linear electrodes inserted perpendicular to layers in V4 and 81
tangential to layers in the frontal eye field (FEF) (or with single electrodes in FEF in some 82
sessions) from two monkeys performing a selective attention task (V4 data reported 83
previously in ref. (Engel et al., 2016)). A similar analysis revealed that changes of FEF firing 84
rates precede On-Off transitions in V4 (Fig. 2D). These results suggest that On-Off transitions 85
traverse from higher to lower areas along the visual hierarchy during selective attention. 86
To investigate the relationship between V1 and V4 On-Off transitions more closely, we fit a 87
4-state HMM to V1 and V4 data simultaneously (HMMV1-V4), with the four HMM-states 88
defined as (state 1) V1off-V4off, (state 2) V1on-V4off, (state 3) V1off-V4on and (state 4) V1on-V4on 89
.CC-BY-NC-ND 4.0 International license(which was not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprintthis version posted March 26, 2020. . https://doi.org/10.1101/2020.03.26.009365doi: bioRxiv preprint
(Fig. 3A). This model allowed us to investigate two specific scenarios (Fig. 3B). In the first 90
scenario (yellow), we asked: from a situation in which both areas are in an Off phase (state 91
1), is it more likely for V1 (state 2) or V4 (state 3) to transition (first) to an On phase? The 92
second scenario (purple) addresses a related question: from a situation in which both areas 93
are in an On phase (state 4), is it more likely for V1 (state 3) or V4 (state 2) to transition 94
(first) to an Off state? The transition probabilities (Fig. 3C & Fig. 3D) revealed that when 95
both areas were in an Off phase, it was more likely for V4 to transition to an On phase first 96
[Wilcoxon signed rank test; P < 10-3]. Likewise, if both areas were in an On phase, it was 97
more likely for V4 to transition to an Off phase first [Wilcoxon signed rank test; P < 10-3]. 98
Thus, when both areas are in the same phase, it is more likely for V4 to transition away from 99
this phase first. This finding was, however, not specific to the attend RF condition, as we 100
found similar results for each individual attention condition (attend RF and attend away), as 101
well as during fixation (data not shown). Selective attention, however, modulated the 102
transition probabilities from the yellow scenario. Specifically, it decreased the probability of 103
transitioning from state 1 to state 2, and increased the probability of transitioning from 104
state 1 to state 3 [Wilcoxon signed rank test; P < 10-2] (Fig. 3E & Fig. 3F). Finally, this model 105
revealed that, although On-Off phases/transitions are correlated, each area spends a 106
substantial fraction of time in opposite phases (Fig. 3G). Selective attention specifically 107
decreases the time spent in state 1 whereas it increases the time spent in state 3 and state 108
4, i.e. the states where V4 was in an On phase [Wilcoxon signed rank test; state 1 P < 10-5, 109
state 2 P = 0.91, state 3 P < 10-2, state 4 P < 10-3] (Fig. 3H). 110
On-Off dynamics furthermore related closely to measures of (bipolar re-referenced) local 111
field potential (LFP) (de)synchronization. During On phases in either V1 or V4, low frequency 112
(< ~20 Hz) LFP power was suppressed and high frequency (> ~20 Hz) power was increased, 113
both in V1 and V4 (Fig. S4A-D). Additionally, LFP power spectra in both areas varied across 114
the 4 states of HMMV1-V4 (Fig. S4E-F). Here, we specifically investigated the difference in 115
power spectra across states where the On-Off phase within an area remained constant, but 116
differed in the other area. For example, we investigated the V1 LFP power spectra across 117
states 1 and 3, wherein V1 was in an Off phase during both states, but V4 was either Off or 118
On. This analysis revealed that the LFP power in V1 is modulated by V4 phase 119
bidirectionally. If V1 was in either an On or an Off phase, a transition to an On phase in V4 120
.CC-BY-NC-ND 4.0 International license(which was not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprintthis version posted March 26, 2020. . https://doi.org/10.1101/2020.03.26.009365doi: bioRxiv preprint
increased V1 high-frequency power. A transition to an On phase in V1, however, only 121
affected V4 high-frequency power when V4 was in an Off phase. When V4 was in an On 122
phase, V1 phase did not affect high-frequency dynamics in V4. Thus, V4 phase influenced V1 123
LFP regardless of V1 phase, whereas V1 phase affected high-frequency dynamics in V4 only 124
during Off phases in V4. 125
In addition to selective attention, On-Off dynamics were linked to global arousal levels, as 126
measured by pupil diameter (Aston-Jones and Cohen, 2005; McGinley et al., 2015a, 2015b; 127
Reimer et al., 2014; Vinck et al., 2015). For each area individually, On epoch durations were 128
longer on trials with larger baseline pupil diameter (Fig. S5A-C), in line with previous results 129
(Engel et al., 2016). Furthermore, pupil diameter was predictive of On-Off dynamics 130
coordination. Larger baseline pupil diameter was predictive of shorter epoch durations for 131
HMMV1-V4 state 1 (where both areas were Off) and longer state 4 epoch durations (where 132
both areas were On) (Fig. S5D). Central arousal, in addition to focused attention, thus 133
specifically influenced epoch durations for states where V1 and V4 phase were aligned. 134
Importantly, pupil diameter did not differ between attention conditions (Fig. S5E), while 135
cortical state did. This shows that effects of arousal and attention on On-Off dynamics are 136
independently controlled. 137
We have demonstrated that the coordination of On-Off dynamics is retinotopically 138
organized and driven in a top-down manner during selective attention. Is this organization 139
also relevant for behavior? For both V1 and V4 individually, the On/Off phase at the time of 140
target dimming was predictive of reaction time (RT) when the target was presented inside 141
the RFs. We found an interaction between attention and On/Off phase [linear mixed effects 142
model; V1 β = 0.16±0.06, P = 0.006, V4 β = 0.13±0.06, P = 0.03] with a main effect for phase 143
[V1 β = -0.27±0.09, P = 0.002, V4 β = -0.26±0.09, P = 0.004], but no main effect of attention 144
[V1 β = -0.15±0.09, P = 0.09, V4 β = -0.1±0.09, P = 0.28]. Specifically, when either area was in 145
an On phase when the target grating dimmed, RT was faster [Wilcoxon signed rank test; V1 146
P = 0.001, V4 P < 10-4] (Fig. 4A). We furthermore found that On-Off phase coordination 147
between V1 and V4, as assessed using HMMV1-V4, was also predictive of behavioral 148
performance. Again we found an interaction between attention and On/Off phase [linear 149
mixed effects model; β = 0.08±0.02, P < 10-3], with a main effect of phase [β = -0.15±0.04, P 150
< 10-4], but not of attention [β = -0.07±0.07, P = 0.24]. Performance was worst when at the 151
.CC-BY-NC-ND 4.0 International license(which was not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprintthis version posted March 26, 2020. . https://doi.org/10.1101/2020.03.26.009365doi: bioRxiv preprint
time of target dimming both V1 and V4 were in an Off phase (state 1). Performance 152
improved when either area was in an On phase, and it improved even further when both 153
areas were in an On phase at the time of target dimming (Fig. 4B). The coordination of On-154
Off dynamics across visual areas is thus more beneficial for behavioral performance than the 155
state in either area alone. 156
In line with previous results (Engel et al., 2016), we show that transitions between On and 157
Off phases in V4 are modulated locally by spatially selective attention. In addition, we found 158
that they also occur in primary visual cortex (V1). Importantly, we show that interareal 159
coordination of On-Off dynamics occurs at a local retinotopic scale, which reflects the 160
precision of anatomical connections, and is driven in a top-down manner across areas FEF, 161
V4 and V1 during selective attention. Fluctuations in cortical state have previously been 162
ascribed to neuromodulatory influences (Buzsaki et al., 1988; Constantinople and Bruno, 163
2011; Lee and Dan, 2012) and feedback projections (Zagha et al., 2013). On-Off dynamics 164
relate to both these mechanisms as pupil diameter, associated with neuromodulatory 165
regulation of network state (Aston-Jones and Cohen, 2005; de Gee et al., 2017; Eldar et al., 166
2013; Joshi et al., 2016; Murphy et al., 2014; Reimer et al., 2016; Varazzani et al., 2015), and 167
top-down retinotopic alignment, probably driven by feedback mechanisms (Zagha et al., 168
2013), are predictive of cortical state fluctuations. 169
The interareal coordination of On-Off dynamics and its relevance to behavioral performance 170
suggests that trial-by-trial coordination of activity across brain regions is beneficial for 171
information transfer and selectively modulated according to task demands. Across-area 172
oscillatory activity is correlated according to both retinotopy and stimulus selectivity (Lewis 173
et al., 2016). Selective attention modulates this interareal coherence (Bosman et al., 2012; 174
Buschman and Miller, 2007; Gregoriou et al., 2009), potentially facilitating communication 175
between hierarchically linked areas (Fries, 2005). Although attention can reduce within-area 176
spike count correlations (Cohen and Maunsell, 2009; Herrero et al., 2013; Mitchell et al., 177
2009), depending on the signal correlation between neuronal pairs (Rabinowitz et al., 2015; 178
Ruff and Cohen, 2014), it increases correlated variability across functionally related areas 179
(Oemisch et al., 2015; Ruff and Cohen, 2016). This increased coordination might be a 180
prerequisite for successful interareal information transfer (Harris and Mrsic-Flogel, 2013) 181
and might allow propagation of sensory information to other brain regions (Luczak et al., 182
.CC-BY-NC-ND 4.0 International license(which was not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprintthis version posted March 26, 2020. . https://doi.org/10.1101/2020.03.26.009365doi: bioRxiv preprint
2013). When hierarchically linked areas are simultaneously active, potentially driven by the 183
frontal cortex, global representation of information through recurrent processing could be 184
facilitated, thereby aiding conscious stimulus processing (Baars, 2002; Dehaene and 185
Changeux, 2011). Cognitive modulation of cortical state coordination could be a key 186
component of this. 187
188
References 189
Arieli A, Sterkin A, Grinvald A, Aertsen A. 1996. Dynamics of Ongoing Activity: 190 Explanation of the Large Variability in Evoked Cortical Responses. Science (80- ) 191 273:1868–1871. doi:10.1126/science.273.5283.1868 192
Aston-Jones G, Cohen JD. 2005. An integrative theory of locus coeruleus-norepinephrine 193 function: adaptive gain and optimal performance. Annu Rev Neurosci 28:403–50. 194 doi:10.1146/annurev.neuro.28.061604.135709 195
Baars BJ. 2002. The Conscious Access Hypothesis. Trends Cogn Sci 6:47–52. 196 doi:10.1016/S1364-6613(00)01819-2 197
Benjamini Y, Yekutieli D. 2001. The control of the false discovery rate in multiple testing 198 under dependency. Ann Stat 29:1165–1188. doi:10.1214/aos/1013699998 199
Bishop CM. 2006. Pattern Recognition and Machine Learning. Springer. 200
Bokil H, Andrews P, Kulkarni JE, Mehta S, Mitra PP. 2010. Chronux: A platform for 201 analyzing neural signals. J Neurosci Methods 192:146–151. 202 doi:10.1016/j.jneumeth.2010.06.020 203
Bosman CA, Schoffelen J-M, Brunet N, Oostenveld R, Bastos AM, Womelsdorf T, Rubehn 204 B, Stieglitz T, De Weerd P, Fries P. 2012. Attentional Stimulus Selection through 205 Selective Synchronization between Monkey Visual Areas. Neuron 75:875–888. 206 doi:10.1016/j.neuron.2012.06.037 207
Buschman TJ, Miller EK. 2007. Top-Down Versus Bottom-Up Control of Attention in the 208 Prefrontal and Posterior Parietal Cortices. Science (80- ) 315:1860–1862. 209 doi:10.1126/science.1138071 210
Buzsaki G, Bickford R, Ponomareff G, Thal L, Mandel R, Gage F. 1988. Nucleus basalis and 211 thalamic control of neocortical activity in the freely moving rat. J Neurosci 8:4007–212 4026. doi:10.1523/JNEUROSCI.08-11-04007.1988 213
Cohen MR, Maunsell JHR. 2009. Attention improves performance primarily by reducing 214 interneuronal correlations. Nat Neurosci 12:1594–1600. doi:10.1038/nn.2439 215
Constantinople CM, Bruno RM. 2011. Effects and mechanisms of wakefulness on local 216 cortical networks. Neuron 69:1061–1068. doi:10.1016/j.neuron.2011.02.040 217
de Gee JW, Colizoli O, Kloosterman NA, Knapen T, Nieuwenhuis S, Donner TH. 2017. 218 Dynamic modulation of decision biases by brainstem arousal systems. Elife 6:1–36. 219 doi:10.7554/eLife.23232 220
.CC-BY-NC-ND 4.0 International license(which was not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprintthis version posted March 26, 2020. . https://doi.org/10.1101/2020.03.26.009365doi: bioRxiv preprint
Dehaene S, Changeux J-P. 2011. Experimental and Theoretical Approaches to Conscious 221 Processing. Neuron 70:200–227. doi:10.1016/j.neuron.2011.03.018 222
Eldar E, Cohen JD, Niv Y. 2013. The effects of neural gain on attention and learning. Nat 223 Neurosci 16:1146–1153. doi:10.1038/nn.3428 224
Engbert R, Kliegl R. 2003. Microsaccades uncover the orientation of covert attention. Vision 225 Res. doi:10.1016/S0042-6989(03)00084-1 226
Engel TA, Steinmetz NA, Gieselmann MA, Thiele A, Moore T, Boahen K. 2016. Selective 227 modulation of cortical state during spatial attention. Science (80- ) 354:1140–1144. 228 doi:10.1126/science.aag1420 229
Fries P. 2005. A mechanism for cognitive dynamics: Neuronal communication through 230 neuronal coherence. Trends Cogn Sci 9:474–480. doi:10.1016/j.tics.2005.08.011 231
Gieselmann MA, Thiele A. 2008. Comparison of spatial integration and surround suppression 232 characteristics in spiking activity and the local field potential in macaque V1. Eur J 233 Neurosci 28:447–459. doi:10.1111/j.1460-9568.2008.06358.x 234
Gray H, Bertrand H, Mindus C, Flecknell P, Rowe C, Thiele A. 2016. Physiological, 235 Behavioral, and Scientific Impact of Different Fluid Control Protocols in the Rhesus 236 Macaque (Macaca mulatta). Eneuro 3:1–15. 237
Gregoriou GG, Gotts SJ, Zhou H, Desimone R. 2009. High-frequency, long-range coupling 238 between prefrontal and visual cortex during attention. Science 324:1207–1210. 239 doi:10.1126/science.1171402 240
Gutnisky DA, Beaman CB, Lew SE, Dragoi V. 2017. Spontaneous Fluctuations in Visual 241 Cortical Responses Influence Population Coding Accuracy. Cereb Cortex 27:1409–242 1427. doi:10.1093/cercor/bhv312 243
Harris KD, Mrsic-Flogel TD. 2013. Cortical connectivity and sensory coding. Nature 244 503:51–8. doi:10.1038/nature12654 245
Harris KD, Thiele A. 2011. Cortical state and attention. Nat Rev Neurosci 12:509–523. 246 doi:10.1038/nrn3084 247
Herrero JL, Gieselmann M a, Sanayei M, Thiele A. 2013. Attention-induced variance and 248 noise correlation reduction in macaque v1 is mediated by NMDA receptors. Neuron 249 78:729–739. doi:10.1016/j.neuron.2013.03.029 250
Joshi S, Li Y, Kalwani RM, Gold JI. 2016. Relationships between Pupil Diameter and 251 Neuronal Activity in the Locus Coeruleus, Colliculi, and Cingulate Cortex. Neuron 252 89:221–234. doi:10.1016/j.neuron.2015.11.028 253
Kohn A, Zandvakili A, Smith MA. 2009. Correlations and brain states: from 254 electrophysiology to functional imaging. Curr Opin Neurobiol 19:434–438. 255 doi:10.1016/j.conb.2009.06.007 256
Lee S-H, Dan Y. 2012. Neuromodulation of Brain States. Neuron 76:209–222. 257 doi:10.1016/j.neuron.2012.09.012 258
Lewis CM, Bosman CA, Womelsdorf T, Fries P. 2016. Stimulus-induced visual cortical 259 networks are recapitulated by spontaneous local and interareal synchronization. Proc 260 Natl Acad Sci U S A 113:E606–E615. doi:10.1073/pnas.1513773113 261
Logothetis NK, Kayser C, Oeltermann A. 2007. In Vivo Measurement of Cortical Impedance 262 Spectrum in Monkeys: Implications for Signal Propagation. Neuron 55:809–823. 263
.CC-BY-NC-ND 4.0 International license(which was not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprintthis version posted March 26, 2020. . https://doi.org/10.1101/2020.03.26.009365doi: bioRxiv preprint
doi:10.1016/j.neuron.2007.07.027 264
Luczak A, Bartho P, Harris KD. 2013. Gating of Sensory Input by Spontaneous Cortical 265 Activity. J Neurosci 33:1684–1695. doi:10.1523/jneurosci.2928-12.2013 266
McGinley MJ, David S V., McCormick DA. 2015a. Cortical Membrane Potential Signature 267 of Optimal States for Sensory Signal Detection. Neuron 87:179–192. 268 doi:10.1016/j.neuron.2015.05.038 269
McGinley MJ, Vinck M, Reimer J, Batista-Brito R, Zagha E, Cadwell CR, Tolias AS, Cardin 270 JA, McCormick DA. 2015b. Waking State: Rapid Variations Modulate Neural and 271 Behavioral Responses. Neuron 87:1143–1161. doi:10.1016/j.neuron.2015.09.012 272
Mitchell JF, Sundberg K a, Reynolds JH. 2009. Spatial attention decorrelates intrinsic 273 activity fluctuations in macaque area V4. Neuron 63:879–88. 274 doi:10.1016/j.neuron.2009.09.013 275
Mountcastle VB. 1957. Modality and topographic properties of single neurons of cat’s 276 somatic sensory cortex. J Neurophysiol 20:408–34. doi:10.1152/jn.1957.20.4.408 277
Murphy PR, O’Connell RG, O’Sullivan M, Robertson IH, Balsters JH. 2014. Pupil diameter 278 covaries with BOLD activity in human locus coeruleus. Hum Brain Mapp 35:4140–279 4154. doi:10.1002/hbm.22466 280
Oemisch M, Westendorff S, Everling S, Womelsdorf T. 2015. Interareal Spike-Train 281 Correlations of Anterior Cingulate and Dorsal Prefrontal Cortex during Attention Shifts. 282 J Neurosci 35:13076–13089. doi:10.1523/jneurosci.1262-15.2015 283
Pettersen KH, Devor A, Ulbert I, Dale AM, Einevoll GT. 2006. Current-source density 284 estimation based on inversion of electrostatic forward solution: Effects of finite extent of 285 neuronal activity and conductivity discontinuities. J Neurosci Methods 154:116–133. 286 doi:10.1016/j.jneumeth.2005.12.005 287
Rabiner LR. 1989. A tutorial on hidden Markov models and selected applications in speech 288 recognition. Proc IEEE 77:257–286. doi:10.1109/5.18626 289
Rabinowitz NC, Goris RL, Cohen M, Simoncelli EP. 2015. Attention stabilizes the shared 290 gain of V4 populations. Elife 4:1–24. doi:10.7554/eLife.08998 291
Reimer J, Froudarakis E, Cadwell CR, Yatsenko D, Denfield GH, Tolias AS. 2014. Pupil 292 Fluctuations Track Fast Switching of Cortical States during Quiet Wakefulness. Neuron 293 84:355–362. doi:10.1016/j.neuron.2014.09.033 294
Reimer J, McGinley MJ, Liu Y, Rodenkirch C, Wang Q, McCormick DA, Tolias AS. 2016. 295 Pupil fluctuations track rapid changes in adrenergic and cholinergic activity in cortex. 296 Nat Commun 7:13289. doi:10.1038/ncomms13289 297
Renart A, Machens CK. 2014. Variability in neural activity and behavior. Curr Opin 298 Neurobiol 25:211–220. doi:10.1016/j.conb.2014.02.013 299
Roelfsema PR, Tolboom M, Khayat PS. 2007. Different Processing Phases for Features, 300 Figures, and Selective Attention in the Primary Visual Cortex. Neuron 56:785–792. 301 doi:10.1016/j.neuron.2007.10.006 302
Ruff DA, Cohen MR. 2016. Attention Increases Spike Count Correlations between Visual 303 Cortical Areas. J Neurosci 36:7523–7534. doi:10.1523/JNEUROSCI.0610-16.2016 304
Ruff DA, Cohen MR. 2014. Attention can either increase or decrease spike count correlations 305 in visual cortex. Nat Neurosci 17:1591–1597. doi:10.1038/nn.3835 306
.CC-BY-NC-ND 4.0 International license(which was not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprintthis version posted March 26, 2020. . https://doi.org/10.1101/2020.03.26.009365doi: bioRxiv preprint
Scholvinck ML, Saleem AB, Benucci A, Harris KD, Carandini M. 2015. Cortical State 307 Determines Global Variability and Correlations in Visual Cortex. J Neurosci 35:170–308 178. doi:10.1523/JNEUROSCI.4994-13.2015 309
Schroeder C. 1998. A spatiotemporal profile of visual system activation revealed by current 310 source density analysis in the awake macaque. Cereb Cortex 8:575–592. 311 doi:10.1093/cercor/8.7.575 312
Schroeder CE, Tenke CE, Givre SJ, Arezzo JC, Vaughan HG. 1991. Striate cortical 313 contribution to the surface-recorded pattern-reversal vep in the alert monkey. Vision Res 314 31:1143–1157. doi:10.1016/0042-6989(91)90040-C 315
Self MW, van Kerkoerle T, Supèr H, Roelfsema PR. 2013. Distinct Roles of the Cortical 316 Layers of Area V1 in Figure-Ground Segregation. Curr Biol 23:2121–2129. 317 doi:10.1016/j.cub.2013.09.013 318
Thiele A, Delicato LS, Roberts MJ, Gieselmann MA. 2006. A novel electrode-pipette design 319 for simultaneous recording of extracellular spikes and iontophoretic drug application in 320 awake behaving monkeys. J Neurosci Methods 158:207–211. 321 doi:10.1016/j.jneumeth.2006.05.032 322
Varazzani C, San-Galli A, Gilardeau S, Bouret S. 2015. Noradrenaline and Dopamine 323 Neurons in the Reward/Effort Trade-Off: A Direct Electrophysiological Comparison in 324 Behaving Monkeys. J Neurosci 35:7866–7877. doi:10.1523/JNEUROSCI.0454-15.2015 325
Vinck M, Batista-Brito R, Knoblich U, Cardin JA. 2015. Arousal and Locomotion Make 326 Distinct Contributions to Cortical Activity Patterns and Visual Encoding. Neuron 327 86:740–754. doi:10.1016/j.neuron.2015.03.028 328
Zagha E, Casale AE, Sachdev RNS, McGinley MJ, McCormick DA. 2013. Motor cortex 329 feedback influences sensory processing by modulating network state. Neuron 79:567–330 578. doi:10.1016/j.neuron.2013.06.008 331
332
Acknowledgments: 333
We thank Demetrio Ferro for advice on data analysis. 334
Funding: 335
Funded by Wellcome trust 093104 (JvK, MAG, AT), MRC MR/P013031/1 (JvK, MAG, AT), NIH 336
grant R01 EB026949 (TAE) and the Pershing Square Foundation (TAE), NIH grant EY014924 337
(NAS and TM). 338
Author contributions: 339
Jochem van Kempen: Conceptualization, Methodology, Formal analysis, Investigation, Data 340
Curation, Writing – original draft preparation, Writing – review and editing, Visualization 341
.CC-BY-NC-ND 4.0 International license(which was not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprintthis version posted March 26, 2020. . https://doi.org/10.1101/2020.03.26.009365doi: bioRxiv preprint
Marc A. Gieselmann: Software, Writing – review and editing, Supervision 342
Michael Boyd: Investigation 343
Nicholas A. Steinmetz: Investigation, Writing – review and editing 344
Tirin Moore: Resources, Writing – review and editing, Funding acquisition 345
Tatiana A. Engel: Conceptualization, Methodology, Software, Writing – review and editing, 346
Supervision 347
Alexander Thiele: Conceptualization, Investigation, Resources, Writing – review and editing, 348
Supervision, Funding acquisition 349
Competing interests: 350
There are no competing interests. 351
Data and materials availability: 352
Data and materials are available upon request. 353
354
.CC-BY-NC-ND 4.0 International license(which was not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprintthis version posted March 26, 2020. . https://doi.org/10.1101/2020.03.26.009365doi: bioRxiv preprint
355
Fig. 1. On-Off dynamics in V1 and V4 are modulated during selective attention. (A) 356
Behavioral paradigm. The monkey held a lever to initiate the trial, hereafter a central 357
fixation spot was turned on. Upon fixation 3 colored gratings appeared, one was presented 358
inside the receptive fields (RFs) of the V1 neurons. After a variable delay a cue matching one 359
of the grating colors surrounded the fixation spot, indicating which grating was behaviorally 360
relevant (target). In pseudorandom order the stimuli decreased in luminance (dimmed). 361
Upon dimming of the target, the monkey had to release the lever. (B) Average RF center 362
.CC-BY-NC-ND 4.0 International license(which was not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprintthis version posted March 26, 2020. . https://doi.org/10.1101/2020.03.26.009365doi: bioRxiv preprint
locations (across channels) for each recording, separately for each subject (M1-M3) and 363
area. (C) RF separation between V1 and V4 plotted against their overlap, expressed as the 364
proportion of the V1 RF. The histograms along the top (right) indicate the distribution of RF 365
separation (overlap) across all recordings. (D) Raster plot of HMM fit to population activity 366
in V1 and V4. Simultaneously recorded multi-unit spiking activity on 16-contact laminar 367
electrodes in V1 and V4 for 15 example trials, aligned to stimulus (left) and cue onset 368
(middle and right). Each trial shows across laminar activity in V1 (bottom) and V4 (top), as 369
raster plots (left two columns) color coded according to HMM estimation of On and Off 370
phases (right). Middle and right columns depict the same activity. The HMM was fit from 371
400 ms after cue onset to 30 ms after the first dimming event. Cue onset and first-dimming 372
are indicated for each trial by purple and red vertical bars respectively. (E) Attention 373
increases firing rates during Off and On phases, both in V1 and V4. (F) Attention increases 374
the duration of On episodes, both in V1 and V4, whereas it increases the duration of Off 375
episodes only in V1. Statistics: Wilcoxon signed rank test.376
.CC-BY-NC-ND 4.0 International license(which was not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprintthis version posted March 26, 2020. . https://doi.org/10.1101/2020.03.26.009365doi: bioRxiv preprint
377
Fig. 2. Across area coordination of cortical state. (A) Cross-correlation between time series 378
of On-Off phases in V1 and V4 relative to V1 phase during passive fixation (left) and after 379
cue onset (right). Insets show the area under the cross-correlation curve for times smaller 380
and larger than zero. The dashed grey line depicts the shuffle predictor. (B) RF separation 381
plotted against the area under the cross-correlation curve during attention (from the right 382
panel A). The line indicates the standardized major axis regression fit. (C) Spiking activity in 383
one area aligned to state transitions in the other area, averaged across channels and 384
recordings. Only epochs without transitions preceding or following the alignment transition 385
within 100 ms were included. Thick green and pink lines indicate the times the firing rate 386
was higher (green) or lower (pink) than the average rate. Along the bottom are the 387
histograms of the crossing point of two straight lines fit (least-squares) to the transition-388
aligned multi-unit firing rate. (D) Conventions as in C, but from a different dataset in which 389
activity was recorded simultaneously from V4 and FEF. Statistics: Wilcoxon signed rank test 390
(A), Pearson correlation (B) and FDR-corrected, one-sided, Wilcoxon signed rank test (C & 391
D). Shaded regions denote ±1 SEM.392
.CC-BY-NC-ND 4.0 International license(which was not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprintthis version posted March 26, 2020. . https://doi.org/10.1101/2020.03.26.009365doi: bioRxiv preprint
393
Fig. 3. HMM with 4 states fit simultaneously to V1 and V4 data. (A) Example trial with the 394
HMM state-trajectory (bottom) and across-laminar raster plot for V1 (middle) and V4 (top). 395
(B) Schematic describing scenarios for testing two questions: (1, left yellow box) from a 396
state where both V1 and V4 are Off, is it more likely for V1 or V4 to transition to the On 397
phase first? (2, right purple box) from a state where both V1 and V4 are On, is it more likely 398
for V1 or V4 to transition to the Off phase first? (C) HMM transition probability matrix, 399
indicating the probability of staying in a state (diagonal) or transitioning from one state to 400
another. Highlighted are scenarios set out in panel B. (D) Transition probabilities indicated in 401
panels B and C. (E) Attentional influence on state-transition probabilities: shown is the 402
difference transition matrix (attend RF – attend Away). (F) Attentional influence (attend RF – 403
attend Away) on the difference between state transition probabilities (state 3 – state 2) for 404
each of the two scenarios indicated in panels B, C and D. Selective attention increases the 405
difference between the transition probabilities for state 2 and 3 for the yellow, but not the 406
purple scenario. (G) The fraction of time spent in each of the 4 states. (H) The difference in 407
.CC-BY-NC-ND 4.0 International license(which was not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprintthis version posted March 26, 2020. . https://doi.org/10.1101/2020.03.26.009365doi: bioRxiv preprint
time spent in each of the 4 states when attention is directed towards or away from the RF 408
(attend RF – attend Away). Statistics: Wilcoxon signed rank test (FDR corrected), error bars 409
denote ±1 SEM across recordings; *, **, *** indicate significance levels (p < 0.05, p < 0.01 410
and p < 0.001, respectively).411
.CC-BY-NC-ND 4.0 International license(which was not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprintthis version posted March 26, 2020. . https://doi.org/10.1101/2020.03.26.009365doi: bioRxiv preprint
412
Fig. 4. Across-area coordination of On-Off dynamics predicts behavioral performance. (A) 413
On vs. Off phase of population activity at the time of target dimming, determined 414
individually for V1 and V4, predicts behavioral performance. RT decreases when attention is 415
directed towards the RFs and either V1 or V4 is in an On phase. (B) RT decreased from when 416
both areas were Off, through V1 On - V4 Off, through V1 Off - V4 On, to V1 and V4 On when 417
attention was directed towards the RFs. Statistics: Wilcoxon signed rank test (A), and 418
multilevel linear mixed effect model (B). Error bars denote ±1 SEM, and *, ** and *** 419
indicate FDR corrected significance levels of p < 0.05, p < 0.01 and p < 0.001, respectively. 420
421
.CC-BY-NC-ND 4.0 International license(which was not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprintthis version posted March 26, 2020. . https://doi.org/10.1101/2020.03.26.009365doi: bioRxiv preprint
Materials and Methods 422
Animals and procedures 423
Subjects in our study were 3 male rhesus macaque monkeys (Macaca mulatta, age 10-12 424
years, weight 8.5-12.5 kg). All animal procedures were performed in accordance with the 425
European Communities Council Directive RL 2010/63/EC, the National Institute of Health’s 426
Guidelines for the Care and Use of Animals for Experimental Procedures, and the UK 427
Animals Scientific Procedures Act. Animals were motivated to engage in the task through 428
fluid control at levels that do not affect animal physiology and have minimal impact on 429
psychological wellbeing (Gray et al., 2016). 430
431
Surgical preparation 432
The animals were implanted with a head post and recording chambers over area V1 and V4 433
under sterile conditions and general anesthesia. Surgical procedures and postoperative care 434
conditions have been described in detail previously (Thiele et al., 2006). 435
436
Behavioral paradigm 437
Stimulus presentation and behavioral control was regulated by Remote Cortex 5.95 438
(Laboratory of Neuropsychology, National Institute for Mental Health, Bethesda, MD). 439
Stimuli were presented on a cathode ray tube (CRT) monitor at 120 Hz, 1280 × 1024 pixels, 440
at a distance of 54 cm. The location and size of receptive field (RF) were measured as 441
described previously (Gieselmann and Thiele, 2008), using a reverse correlation method. 442
Briefly, during fixation, a series of black squares (0.5-2° size, 100% contrast) were presented 443
for 100 ms at pseudorandom locations on a 9 × 12 grid (5-25 repetitions for each location) 444
on a bright background. RF eccentricity ranged from 3.4 - 7.5° in V1, and from 2.5 to 8.9° in 445
V4. 446
During the main task (Fig. 1A), the monkeys initiated a trial by holding a lever and fixating on 447
a central white fixation spot (0.1°) displayed on a gray background (1.41 cd/m2). After a 448
fixed delay (614, 424, 674 ms, for monkeys 1, 2 and 3), three colored (for color values see 449
Table S1) square wave gratings appeared equidistant from the fixation spot, one was 450
.CC-BY-NC-ND 4.0 International license(which was not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprintthis version posted March 26, 2020. . https://doi.org/10.1101/2020.03.26.009365doi: bioRxiv preprint
centered on the RF of the V1 neurons under study. The locations of colored gratings were 451
fixed for each recording session but were pseudorandomly assigned across sessions. 452
Stimulus size varied between 2 and 4° diameter, depending on RF eccentricity and size. For 453
most recordings we used drifting gratings but presented one monkey with stationary 454
gratings during 22 out of 34 recording days. The drifting gratings moved perpendicular to 455
the grating orientation, with the motion direction pseudorandomly assigned on every trial. 456
After a random delay (618-1131 ms for monkey 1, 618-948 ms for monkeys 2 and 3; 457
uniformly distributed), a central cue appeared that matched the color of one of the gratings, 458
indicating that this grating would be behaviorally relevant on the current trial. After a 459
variable delay (1162-2133 ms for monkey 1, 1162-1822 ms for monkeys 2 and 3; uniformly 460
distributed), one of the pseudorandomly selected gratings changed luminance (for color 461
values see Table S1), referred to as dimming. If the cued grating (target) dimmed, the 462
monkey had to release the lever in order to obtain a reward. If, however, a non-cued grating 463
(distractor) dimmed, the monkey had to ignore this and keep hold of the lever until the 464
target dimmed on the second or third dimming event (each after another 792-1331 ms for 465
monkey 1; 792-1164 ms for monkeys 2 and 3; uniformly distributed). 466
467
Data acquisition and analysis 468
We recorded from all cortical layers of visual areas V1 and V4 using 16-contact laminar 469
electrodes (150 µm contact spacing, Atlas silicon probes). Out of a total of 77 V1 and 79 V4 470
recording sessions, 73 recordings were conducted simultaneously in both areas. The 471
electrodes were inserted perpendicular to the cortex on a daily basis. 472
Raw data were collected using Remote Cortex 5.95 and by Cheetah data acquisition 473
interlinked with Remote Cortex 5.95. Neuronal data were acquired with Neuralynx 474
preamplifiers and a Neuralynx Digital Lynx amplifier. Unfiltered data were sampled with 24 475
bit at 32.7 kHz and stored to disc. Data were replayed offline, sampled with 16-bit and band-476
pass filtered at 0.5-300 Hz and down sampled to 1 kHz for local field potential (LFP) data, 477
and filtered at 0.6-9 kHz for spike extraction. Eye position and pupil diameter was recorded 478
at 220 Hz (ViewPoint, Arrington Research). Pupil diameter was recorded for 75 (90.4 %) of 479
recordings. 480
.CC-BY-NC-ND 4.0 International license(which was not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprintthis version posted March 26, 2020. . https://doi.org/10.1101/2020.03.26.009365doi: bioRxiv preprint
All data analyses were performed using custom written Matlab (the Mathworks) scripts. 481
482
Data preprocessing, trial selection and channel selection 483
We corrected for any noise common to all channels via common average reference, in 484
which the average of all channels is subtracted from each individual channel. We extracted 485
population activity by progressively lowering spike extraction thresholds until approximately 486
100 Hz spiking activity was detected on each channel between fixation onset and the first 487
dimming event. Next, we computed the envelope of MUA (MUAe) by low-pass filtering 488
(<300 Hz, fifth order Butterworth) the rectified 0.6-9 kHz filtered signal. Because we noticed 489
that during some recording sessions the electrode seemed to have moved (e.g. due to 490
movement of the monkey), we visually inspected the stability of each recording by 491
investigating the stimulus aligned firing rates, MUAe and their baseline (-500 to -50 ms) 492
energy across all trials and channels. With energy (𝐸) defined as: 493
𝐸 = $ 𝑉(𝑖)!"
# 494
, where 𝑡 is the number of time points in the vector (𝑉) representing the single-trial 495
histogram or MUAe. We selected the largest continuous time window that showed stable 496
activity across all V1 & V4 channels. 497
In addition to selecting trials from stable periods, we selected channels for further 498
processing that were determined to be in gray matter. Using current source density (CSD), 499
we investigated on which channels currents were entering (sinks) and exiting (sources) 500
cortical tissue, which allowed us to determine the relative recording depth compared to the 501
known cortical anatomy (Schroeder, 1998; Schroeder et al., 1991). The CSD profile can be 502
calculated according to the finite difference approximation, taking the inverse of the second 503
spatial derivative of the stimulus-evoked voltage potential 𝜑, defined by: 504
𝐶𝑆𝐷(𝑥) = 𝜑(𝑥 + ℎ) − 2𝜑(𝑥) + 𝜑(𝑥 − ℎ)
ℎ! 505
, where 𝑥 is the depth at which the CSD is calculated and ℎ the electrode spacing (150 μm). 506
We used the iCSD toolbox (Pettersen et al., 2006) to compute the CSD. With this toolbox we 507
used a spline fitting method to interpolate 𝜑 smoothly between electrode contacts. We 508
.CC-BY-NC-ND 4.0 International license(which was not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprintthis version posted March 26, 2020. . https://doi.org/10.1101/2020.03.26.009365doi: bioRxiv preprint
used a diameter of cortical columns of 500 µm (Mountcastle, 1957), and tissue conductance 509
of 0.4 Sm-1 (Logothetis et al., 2007). 510
To aid determination of recording depth, we computed the signal-to-noise ratio (SNR), the 511
response latencies to stimulus onset for each channel and the receptive field (RF) estimation 512
(see below). SNR was computed as: 513
𝑆𝑁𝑅 = 𝑆𝑖𝑔𝑛𝑎𝑙 − 𝑁𝑜𝑖𝑠𝑒
𝜎$%#&' 514
, with 𝑆𝑖𝑔𝑛𝑎𝑙 defined as the average MUAe amplitude in one of eight 50 ms time windows, 515
from 30 to 80 ms, in 10 ms steps, to 100 to 150 ms after stimulus onset, and 𝑁𝑜𝑖𝑠𝑒 as the 516
average MUAe amplitude during the baseline period (-200 to 50 ms) before stimulus onset. 517
SNR in at least one of these eight estimates was required to be higher than 3 for a channel 518
to be included for further analyses. 519
We computed the response latency to stimulus onset for each channel according to the 520
method described by Roelfsema et al. (2007). We fitted the visual response as a 521
combination of an exponentially modified Gaussian and a cumulative Gaussian using a non-522
linear least-squares fitting procedure (function lsqcurvefit) to the average MUAe time 523
course. There are two assumptions implicit in this method. First, the onset latency has a 524
Gaussian distribution across trials and across neurons that contribute to the MUAe, and 525
second, that (part of) the response dissipates exponentially. The visual response 𝑦 across 526
time 𝑡 was modelled as: 527
𝑦(𝑡) = 𝑑 ∙ 𝐸𝑥𝑝(𝜇𝛼 + 0.5𝜎!𝛼! − 𝛼𝑡) ∙ 𝐺(𝑡, 𝑢 + 𝜎!𝛼, 𝜎) + 𝑐 ∙ 𝐺(𝑡, 𝜇, 𝜎) 528
, where 𝜇 is the mean, 𝜎 is the standard deviation, 𝛼() is the time constant of the 529
dissipation, 𝐺(𝑡, 𝜇, 𝜎) is a cumulative Gaussian, and 𝑐 and𝑑 are the factors scaling the non-530
dissipating and dissipating modulation of the visual response. The response latency was 531
defined as the time point where 𝑦(𝑡) reached 33% of the maximum of the earliest peak, the 532
first Gaussian (Roelfsema et al., 2007; Self et al., 2013). Data were aligned to the earliest 533
current sink, the presumed thalamic input layer (L4); channels were excluded if they were 534
>1 mm more superficial or >0.75 mm deeper than this layer. 535
536
.CC-BY-NC-ND 4.0 International license(which was not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprintthis version posted March 26, 2020. . https://doi.org/10.1101/2020.03.26.009365doi: bioRxiv preprint
Receptive field estimation 537
Offline RFs were determined for each channel via reverse correlation of the MUAe signal to 538
stimuli (0.5 – 2 ° black squares) presented on a 9 × 12 grid (Gieselmann and Thiele, 2008). 539
The stimulus-response map was converted to z-scores, after which the RF for each channel 540
was indicated by a contour (thresholded at a z-score of 3) surrounding the peak activity. 541
These z-scored maps were averaged across all channels for each area (the population 542
average z-score was computed using Stouffer’s Z-score method according to 𝑍 =543
∑ 𝑍#/√𝑘*#+) , with 𝑘 as the number of channels, after which we determined the overlap and 544
separation between the V1 and V4 RFs (Fig 1B-C). 545
546
Bipolar re-referencing 547
To ensure that global signals, common to multiple channels, did not affect our LFP and 548
spectral analyses (see below), we re-referenced our LFP signals according to the bipolar 549
derivation. Bipolar re-referenced LFP signals (LFPb) were computed by taking the difference 550
between two neighboring channels. 551
552
Attentional modulation 553
The effect of selective attention on neural activity was computed via an attention 554
modulation index (𝑎𝑡𝑡𝑀𝐼), defined as: 555
𝑎𝑡𝑡𝑀𝐼 =𝐴,- −𝐴%."𝐴,- +𝐴%."
556
, with 𝐴,- as the neural activity when attention was directed towards the RF, and 𝐴%." the 557
activity when attention was directed away from the RF. This index ranges from -1 to 1, with 558
zero indicating no attentional modulation and with positive (negative) values indicating 559
higher (lower) activity when attention was directed towards the RF. 560
561
Hidden Markov Model 562
To quantify On-Off dynamics in V1 and V4, we fit a Hidden Markov Model (HMM) to the 563
population activity across all laminae. We fit the HMM both to activity from each individual 564
.CC-BY-NC-ND 4.0 International license(which was not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprintthis version posted March 26, 2020. . https://doi.org/10.1101/2020.03.26.009365doi: bioRxiv preprint
area, following the procedures described by Engel et al. (2016), as well as to the activity 565
from both areas simultaneously. 566
Our HMM assumes that spike counts on the recorded channels can be well characterized as 567
a doubly-stochastic process, of which the parameters can be accurately estimated (Rabiner, 568
1989). In this study, spike counts on each channel are assumed to be produced by a Poisson 569
process with different (constant) mean rates during On or Off phases of the underlying 570
‘hidden’ (latent) process 𝑠 common to all channels that we need to infer (Engel et al., 2016). 571
The mean firing rate on each channel 𝑗 in phase 𝑠 is defined by entry 𝜆/& in the emission 572
matrix Λ. The transition matrix 𝑃 gives the probabilities of transitioning between these 573
latent phases. In the transition matrix 𝑃, each entry indicates the probability of transitioning 574
between two specific phases. For instance, 𝑃)) indicates the probability of transitioning 575
from 𝑠 = 0 to 𝑠 = 0 (remaining in the Off phase), whereas 𝑃)! indicates the probability of 576
transitioning from 𝑠 = 0 to 𝑠 = 1, more formally: 𝑃)) =𝑃%00 = 𝑃(𝑠"1) = 0|𝑠" = 0), 577
𝑃)! =1 − 𝑃%00 = 𝑃(𝑠"1) = 1|𝑠" = 0). These probabilities do not depend on time: at any 578
time step 𝑡, the probability of transitioning between phases depends only on the value of 𝑠 579
at time 𝑡 (𝑠"). The latent dynamics estimated by the HMM thus follow a discrete time series 580
in which 𝑠" summarises all information before time 𝑡. For each channel, MUA was 581
discretized by determining spike counts in 10 ms bins following each time 𝑡, with the 582
probability of observing spike count 𝑛 on channel 𝑗 during phase 𝑠 defined as 583
𝑃(𝑛|𝑠) = (𝜆/&)$
𝑛! 𝑒(2!" 584
The full description of an HMM is given by the emission matrix Λ, transition matrix 𝑃 and 585
the probabilities 𝜋3 that indicate the initial values 𝑠3, in which 𝜋#3 ≡ 𝑃(𝑠3 = 𝑖). These 586
parameters were estimated using the Expectation Maximization (EM) algorithm (Bishop, 587
2006), maximizing the probability of observing the data given the model according to the 588
Baum-Welch algorithm (Rabiner, 1989). Because the EM procedure can converge to a local 589
maximum, rather than the global maximum, we repeated the EM procedure ten times with 590
random parameter initializations, and chose the model with the highest likelihood. Random 591
values were drawn from Dirichlet distributions for 𝜋3 and P, and from a uniform distribution 592
between zero and twice the channel’s mean firing rate for Λ. The EM procedure was 593
terminated if the relative change, computed as |𝑛𝑒𝑤 − 𝑜𝑟𝑖𝑔𝑖𝑛𝑎𝑙|/|𝑜𝑟𝑖𝑔𝑖𝑛𝑎𝑙|, in the log-594
.CC-BY-NC-ND 4.0 International license(which was not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprintthis version posted March 26, 2020. . https://doi.org/10.1101/2020.03.26.009365doi: bioRxiv preprint
likelihood was smaller than 10-3 and the change in the transition and emission matrix was 595
smaller than 10-5, or if it reached the maximum number of iterations (n = 500). 596
Once the optimal parameters were estimated, we used the Viterbi algorithm to determine 597
the most likely latent trajectory for each individual trial. We applied the HMM separately to 598
each attention condition. For every trial, we applied the HMM during multiple time periods 599
of the task, during fixation and during the time window from 400 ms after cue onset to 30 600
ms after the first dimming event. For the behavioral analysis, we additionally analyzed the 601
period up to 30 ms after the second dimming event for trials in which target dimming did 602
not occur on the first dimming event, and for which the first distractor dimming was not 603
inside the RFs. 604
To determine what number of latent phases best described the data, we fit HMMs with the 605
number of phases ranging from 1 to 8, and used a four-fold cross-validation procedure to 606
compute the leave-one-channel-out cross-validation error for each HMM (Engel et al., 607
2016). We fit the HMM to a randomly selected subset of 3/4 of the trials, and computed the 608
cross-validation error on the remaining 1/4 of trials. This procedure was repeated 4 times 609
using a different 3/4 of trials for training and 1/4 of trials for testing the HMM. We 610
computed the cross-validation error 𝐶𝑉456 for each channel 𝑗 across all trials 𝐾 and time 611
bins 𝑇 as the difference between the actual and expected spike count according to: 612
𝐶𝑉456[𝑛/] =` ` (𝑛"/ − 𝜆/
&#)!7
"+)
8
*+) 613
We normalized 𝐶𝑉456 to the error in the 1-phase HMM, averaged across channels, cross-614
validations and conditions, and determined the difference in 𝐶𝑉456 with each additional 615
phase in the HMM. The normalized mean cross-validation error across each of the eight 616
HMM models for all recordings is depicted in Fig. S1. For most recordings, and for both V1 617
and V4, 𝐶𝑉456 decreased with the addition of a second phase, but did not decrease much 618
further with additional phases. This allowed the identification of the elbow (kink) in this 619
error plot as the model with two phases. We included areas/recordings for further analysis 620
that revealed a reduction in cross-validation error of at least 10% with the addition of a 621
second phase, but did not decrease by more than 10% with additional phases. For a small 622
subset of recordings, a three or a four-phase model fit the data best; these recordings were 623
excluded from further analysis. In total, we found a reduction of >10% in cross-validation 624
.CC-BY-NC-ND 4.0 International license(which was not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprintthis version posted March 26, 2020. . https://doi.org/10.1101/2020.03.26.009365doi: bioRxiv preprint
error when fitting a 2-phase versus 1-phase model in 64 V1 (83.1 %), and 73 V4 (92.4 %) 625
recordings; in 57 (78.1 %) recordings we found evidence for a 2-phase model in both V1 and 626
V4 (Fig. S1). 627
To investigate the across-area coordination of On-Off dynamics, we fit a 4-state HMM to V1 628
and V4 data simultaneously. Across these four states, both V1 and V4 could be in either an 629
Off or On phase, with the states defined as: 𝑉1%00 − 𝑉4%00 (state 1), 𝑉1%$ − 𝑉4%00 (state 630
2), 𝑉1%00 − 𝑉4%$ (state 3) and 𝑉1%$ − 𝑉4%$ (state 4). This model was fit according to the 631
same steps as the HMM applied to individual areas, with one exception. For each channel 𝑗, 632
the emission rate 𝜆 was constrained to be the same across states for which this channel 633
(area) was in the same phase. For example, rates were constrained for a V1 channel across 634
state 1 and state 3, during which V1 was in an Off phase (𝜆/&+) = 𝜆/&+9, 𝑗 ∈ 𝑉1). 635
636
Testing the effect of On-Off dynamics on behavioral performance 637
To determine the effect of On-Off dynamics and their across-area coordination on 638
behavioral performance, we investigated whether the On/Off phase of population activity at 639
the time of target dimming influenced reaction times (RT). To this end, we averaged, for 640
each recording, the RT across all trials that ended in the same phase. We subsequently 641
tested for a relationship between On/Off phase and RT across recordings (Statistical 642
testing). 643
644
Cross correlation 645
The temporal relationship between On-Off time series and transitions, microsaccade onset 646
times and activity in V1, V4 and FEF were investigated using cross-correlations. The cross-647
correlations based on HMM time series (𝐶𝐶:;;) were calculated using the function xcorr in 648
Matlab, according to: 649
𝐶𝐶:;;(𝜏) =1𝑀 `
∑ 𝑥(𝑡)𝑦(𝑡 + 𝜏)7"+)
d∑ |𝑥(𝑡)|! ∙ ∑ |𝑦(𝑡)|!7"+)
7"+)
;
<+)
650
, where M is the number of trials, 𝑇 is the number of discrete time bins, 𝑥 and 𝑦 the mean 651
subtracted On-Off time series in V1 and V4 as determined by the HMM, and 𝜏 the time lag. 652
.CC-BY-NC-ND 4.0 International license(which was not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprintthis version posted March 26, 2020. . https://doi.org/10.1101/2020.03.26.009365doi: bioRxiv preprint
Here, the numerator indicates the cross-covariance, which is normalized (the denominator) 653
such that the autocorrelation for each time series at zero lag is 1. This procedure normalized 654
𝐶𝐶:;; such that correlation coefficients were obtained. We furthermore subtracted the 655
shuffle predictor 𝐶𝐶&=.00>' from 𝐶𝐶:;; to remove any task-related (event-locked) 656
correlations between 𝑥 and 𝑦. 𝐶𝐶&=.00>' was computed by shuffling 𝑦 trials. 657
Cross-correlations (𝐶𝐶) between state transitions and microsaccade onset times were 658
computed in the same way but for a different normalization (denominator) factor. Here we 659
normalized by the number of microsaccades, resulting 𝐶𝐶 to be of the order of coincidences 660
of state transitions per microsaccade. 661
To investigate the neural activity around the time of On-Off transitions, we computed the 662
transition-triggered average (TTA). The TTA was estimated by computing the cross 663
covariance (the numerator), divided by the number of transitions for each channel 664
(denominator). Again, we subtracted the shuffle predictor to remove any task-related 665
correlations. 666
667
Power estimation 668
We estimated the power spectra of the LFPb using a custom multitaper approach based on 669
the Chronux toolbox (Bokil et al., 2010). We estimated the power separately for On and Off 670
states determined by the HMM using only epochs that lasted longer than 250 ms. Because 671
epoch durations were variable, we zero-padded each segment to the next highest power of 672
2 of the longest epoch duration (2048 time points), ensuring we could extract the same 673
frequencies for each segment. This approach gave us a half bandwidth (𝑊) of approximately 674
1.95 Hz, according to 𝑊 = (𝐾 + 1)/2𝑇, with 𝐾 being the number of data tapers (𝐾 = 7) and 675
𝑇 the length of the time window in seconds. Frequencies were estimated from 4 to 200 Hz. 676
677
Microsaccade detection 678
We low-pass filtered the horizontal and vertical eye traces at 30 Hz (2nd order Butterworth 679
filter) after which we detected microsaccades by using the algorithm developed by Engbert 680
and Kliegl (2003). This algorithm converts eye position to velocity and classifies an eye 681
movement as a microsaccade if the velocity is larger than a threshold for at least three 682
.CC-BY-NC-ND 4.0 International license(which was not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprintthis version posted March 26, 2020. . https://doi.org/10.1101/2020.03.26.009365doi: bioRxiv preprint
consecutive time points. The threshold is set to 6 times the median estimator, given by: 683
d𝑚𝑒𝑑𝑖𝑎𝑛(𝑥!) − 𝑚𝑒𝑑𝑖𝑎𝑛(𝑥)!, where 𝑥 is the eye position channel. Thus, the threshold is 684
determined for each single trial. The use of the median estimator ensured that 685
microsaccade detection is relatively robust to different levels of noise. 686
687
Statistical testing 688
To determine whether there were significant differences between attention conditions or 689
HMM states (e.g. in firing rate or epoch duration) we made use of multiple statistical 690
methods. We used (paired-sample) Wilcoxon signed rank tests whenever a comparison was 691
made between two conditions (e.g. attend RF versus attend away), or to test whether a 692
distribution was significantly different from zero. When a comparison involved multiple 693
conditions, or multiple factors (e.g. attention and state), we used linear mixed effect models 694
to test for main effects of each condition/factor and interaction effects between factors. 695
These factors were defined as fixed effects and we included random intercepts for each 696
recording as random effects, accounting for the repeated measurements within each 697
recording. Specifically, we modelled RT as a linear combination of attention condition (𝐴𝑡𝑡) 698
and HMM state coefficients, as well as their interaction: 699
𝑅𝑇~𝛽3 +𝛽)𝐴𝑡𝑡 +𝛽!𝐻𝑀𝑀 + 𝛽9𝐴𝑡𝑡 ∙ 𝐻𝑀𝑀 700
We used false discovery rate (FDR) to correct for multiple comparisons (Benjamini and 701
Yekutieli, 2001). Error bars in all figures indicate the standard error of the mean (SEM). 702
703
.CC-BY-NC-ND 4.0 International license(which was not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprintthis version posted March 26, 2020. . https://doi.org/10.1101/2020.03.26.009365doi: bioRxiv preprint
704
705
Fig. S1. Determining the number of HMM phases and their epoch durations in V1 and 706
V4 MUA. (A) Cross validation (CV) error plotted against the number of phases in each 707
HMM for V1. (B) The difference in cross validation error between the 1-phase and 2-phase 708
model, plotted against the difference between the 2-phase and 3-phase model for V1. Most 709
recordings show a large reduction in cross-validation error with the addition of a second 710
phase, and only marginal changes with additional phases. Blue (red) lines and markers 711
indicate the recordings included (excluded) for further analysis. (C-D) Same conventions as 712
(A-B) but for V4. (E) Distributions of Off and On episode durations overlaid by the 713
exponential distributions with the decay constant set by the HMM transition probabilities 714
(red, 𝑁(𝑡) = 𝑁3𝑒("/@, where 𝑁3 is the normalization constant, and 𝜏 is the decay time-715
constant computed for each recording and phase). A good match for these models indicates 716
that On-Off dynamics were not driven by an oscillatory phenomenon. Grey and thick black 717
lines indicate individual recordings and their mean, respectively. 718
719
720
.CC-BY-NC-ND 4.0 International license(which was not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprintthis version posted March 26, 2020. . https://doi.org/10.1101/2020.03.26.009365doi: bioRxiv preprint
721
Fig. S2. Attentional modulation of HMM parameters. (A) The fraction of time spent in an 722
On phase is increased when attention is directed towards the RFs. (B) Attentional influence 723
on HMM transition probabilities. Shown is the difference between transition matrices (attend 724
RF – attend Away). Statistics: Wilcoxon signed rank tests; *, **, ***, indicate FDR corrected 725
significance levels of p < 0.05, p < 0.01 and p < 0.001, respectively.726
.CC-BY-NC-ND 4.0 International license(which was not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprintthis version posted March 26, 2020. . https://doi.org/10.1101/2020.03.26.009365doi: bioRxiv preprint
727
Fig. S3. Relationship between microsaccades and On-Off transitions. (A) Cross-728
correlation of On-Off transitions in V1 triggered to microsaccade onset. (B) Same as panel A, 729
but for On-Off transitions in V4. (C) Cross-correlation between time series of On-Off 730
dynamics in V1 and V4 after exclusion of trials in which microsaccades occurred. Statistics: 731
Wilcoxon signed rank test. Shaded regions denote ±1 SEM across recordings, *, ** and *** 732
indicate significance levels of p < 0.05, p < 0.01 and p < 0.001 respectively.733
.CC-BY-NC-ND 4.0 International license(which was not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprintthis version posted March 26, 2020. . https://doi.org/10.1101/2020.03.26.009365doi: bioRxiv preprint
734
Fig. S4. Bipolar re-referenced LFP power spectrum across HMM states. (A) Power in 735
V1 during On and Off phases in V1. (B) Power in V4 during On and Off phases in V1. (C) 736
Power in V1 during On and Off phases in V4. (D) Power in V4 during On and Off phases in 737
V4. Right y-axis indicates the percentage change in power during On versus Off phases (On-738
Off). (E-F) Power spectrum in V1 (E) and V4 (F) for the 4-state HMM fit across V1 and V4 739
and the within-area power difference between On phases (red, V1: state 4-2; V4: state 4-3), 740
or Off phases (blue, V1: state 3-1; V4: state 2-1). Only On/Off episodes of at least 250 ms 741
were included. Thick percentage change lines indicate significantly modulated frequencies (p 742
< 0.05, Wilcoxon signed rank test, FDR corrected). Shaded regions denote ±1 SEM. 743
744
745
.CC-BY-NC-ND 4.0 International license(which was not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprintthis version posted March 26, 2020. . https://doi.org/10.1101/2020.03.26.009365doi: bioRxiv preprint
746
Fig. S5. The relationship between baseline pupil diameter and On/Off episode 747
durations. (A) Example recording showing that baseline pupil diameter is positively 748
correlated to the average On episode duration in V1. Each dot represents one single trial, r is 749
the Pearson correlation coefficient. The purple and red dot indicate the example trials used in 750
panel C. (B) Across recordings, the average duration of On epochs in both V1 and V4 is 751
positively correlated with the size of the baseline pupil diameter. (C) Two example trials in 752
which the average On epoch duration is larger on the trial with larger (bottom) compared to 753
the trial with smaller (top) baseline pupil diameter. (D) Across recordings, baseline pupil 754
diameter is negatively (positively) correlated with the average epoch duration when both V1 755
and V4 are in an Off (On) phase. (E) The average baseline pupil diameter during attend RF 756
conditions plotted against attend away conditions. There is no difference between attention 757
conditions. Each dot represents a recording session. Statistics: Wilcoxon signed rank test 758
(FDR corrected) (B, D, E) and Pearson correlation (A). Error bars and shaded regions denote 759
±1 SEM, and *, ** and *** indicate significance levels of p < 0.05, p < 0.01 and p < 0.001 760
respectively. 761
762
763
.CC-BY-NC-ND 4.0 International license(which was not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprintthis version posted March 26, 2020. . https://doi.org/10.1101/2020.03.26.009365doi: bioRxiv preprint
764
Red Green Blue
Monkey 2 & 3,
and monkey 1 (n=4)
a. [220 0 0] – 12.8
b. [140 0 0] – 4.2
a. [0 135 0] – 12.9
b. [0 90 0] – 4.6
a. [60 60 255] – 12.2
b. [30 30 180] – 4.6
Monkey 1 (n=1) b. [170 0 0] –6.7 b. [0 105 0] – 6.4 b. [37 37 210] – 6.6
Monkey 1 (n=1) b. [175 0 0] –7.2 b. [0 105 0] – 6.4 b. [40 40 220] – 7.7
Monkey 1 (n=8) b. [180 0 0] –7.7 b. [0 110 0] – 7.3 b. [40 40 220] – 7.7
Table S1. 765
Color values used for the 3 colored gratings across recording sessions and subjects, indicated 766
as [RGB] – luminance (cd/m2). a = Undimmed values, b = Dimmed values. For monkey 1, 767
we used a variety of dimmed values across recordings. 768
769
.CC-BY-NC-ND 4.0 International license(which was not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprintthis version posted March 26, 2020. . https://doi.org/10.1101/2020.03.26.009365doi: bioRxiv preprint
Top Related