Toolbox

Editor’s Note: Toolboxes are intended to describe and evaluate methods that are becoming widely relevant to the neurosciencecommunity or to provide a critical analysis of established techniques. For more information, see http://www.jneurosci.org/misc/ifa_minireviews.dtl.

Methods for Simultaneous EEG-fMRI: An IntroductoryReview

Rene J. Huster,1 Stefan Debener,2 Tom Eichele,3,4,5 and Christoph S. Herrmann1

1Experimental Psychology Laboratory and 2Neuropsychology Laboratory, Carl von Ossietzky University, 26111 Oldenburg, Germany, 3Section for ClinicalNeurophysiology, Department of Neurology, Haukeland University Hospital, 5011 Bergen, Norway, 4Department of Biological and Medical Psychology,University of Bergen, 5011 Bergen, Norway, and 5Mind Research Network, Albuquerque, New Mexico 87131

The simultaneous recording and analysis of electroencephalography (EEG) and fMRI data in human systems, cognitive and clinicalneurosciences is rapidly evolving and has received substantial attention. The significance of multimodal brain imaging is documented bya steadily increasing number of laboratories now using simultaneous EEG-fMRI aiming to achieve both high temporal and spatialresolution of human brain function. Due to recent developments in technical and algorithmic instrumentation, the rate-limiting step inmultimodal studies has shifted from data acquisition to analytic aspects. Here, we introduce and compare different methods for dataintegration and identify the benefits that come with each approach, guiding the reader toward an understanding and informed selectionof the integration approach most suitable for addressing a particular research question.

IntroductionThe simultaneous recording and analysisof electroencephalography (EEG) andfMRI data in human systems, cognitiveand clinical neurosciences has receivedsubstantial attention (Herrmann andDebener, 2008). Its significance is docu-mented by the steadily increasing numberof laboratories now using simultaneousrecordings. Much of the motivation to ex-plore the applicability of simultaneousEEG-fMRI comes from the selective viewof brain functioning that the two record-ing modalities separately provide. fMRIsuffers from an ill-posed temporal inverseproblem, i.e., a map with regional activa-tions does not readily permit inferencesabout “when” and in which order these

activations have occurred (Logothetis,2008). This is analogous to the well knownspatial inverse problem in EEG, wherebyone cannot infer with certainty the spatiallocation of sources in the brain from elec-trical potentials on the scalp (Michel et al.,2004; Grech et al., 2008). However, be-cause the strengths and weaknesses ofEEG and fMRI are complementary, si-multaneous EEG-fMRI may achieve whatseems otherwise largely impossible, namelythe noninvasive recording of human brainactivity with both high spatial and high tem-poral resolution.

The first applications of EEG-fMRIwere born of a clinical interest in the im-proved localization of the neural sourcesof epileptogenic EEG activity for diagno-sis and presurgical planning (Ives et al.,1993). Although the onset of pathologicalbrain activity can clearly be inferred fromEEG measurements, locations in the cor-tex from which these pathogenic neuronalevents spread cannot unambiguously bederived from EEG alone. The simultane-ous measurement and concurrent analysisof EEG and fMRI for presurgical evalua-

tion of epilepsy provides insights beyondwhat is possible with separate recordingprotocols (Mulert et al., 2008; Gotmanand Pittau, 2011). Based on this clinicalline of research, then intrinsic brain statesreflecting cognitive default modes wereidentified by assessing associations be-tween spontaneous EEG oscillations andfluctuations of the fMRI signal in restingstate (Laufs et al., 2003; Laufs, 2008). Forexample, negative correlations of fMRIactivity in visual cortex with occipital al-pha EEG (8 –12 Hz) further corroboratedthe idea that these alpha oscillations cor-respond to an idling rhythm indicatingcortical inactivation (Goldman et al.,2002; Moosmann et al., 2003). In morerecent years, EEG-fMRI integration pro-cedures have been developed to addressbasic research questions in cognitive neu-roscience by applying simultaneous EEG-fMRI in the context of classical cognitiveexperiments (Mulert and Lemieux, 2009;Ullsperger and Debener, 2010).

One important challenge with simul-taneous EEG-fMRI is the decreased signalquality potentially evident in both modal-

Received Jan. 31, 2012; revised March 5, 2012; accepted March 7, 2012.This work was supported by the German Research Foundation

(Deutsche Forschungsgemeinschaft, Grants HU1729/2-1 andSFB/TR31-A09).

Correspondence should be addressed to Dr. Rene J. Huster, Experimen-tal Psychology Lab, Institute for Psychology, Carl von Ossietzky University,26111 Oldenburg, Germany. E-mail: rene.huster@uni-oldenburg.de.

DOI:10.1523/JNEUROSCI.0447-12.2012Copyright © 2012 the authors 0270-6474/12/326053-08$15.00/0

The Journal of Neuroscience, May 2, 2012 • 32(18):6053– 6060 • 6053

ities. The necessity to introduce conduct-ing materials for EEG recordings in theenvironment of the magnetic resonancesystem may interfere with image acquisi-tion (Mullinger et al., 2008), and stronglydegrades EEG signal quality due to theinduction of electromagnetic currents(Kruggel et al., 2000; Debener et al., 2008;Yan et al., 2009). Most of the technicaldifficulties have now been solved, how-ever, and commercial systems are readilyavailable (Kruggel et al., 2001; Debener etal., 2007; Mullinger et al., 2011). Never-theless, depending on the research ques-tion under investigation and the methodsused for combined analyses, simultaneousEEG-fMRI does not necessarily yield boththe spatial and the temporal resolutionthat might be desirable given the proper-ties of each recording modality alone.Since the intrinsic features of the chosenanalysis method strongly influence theputative outcome, the choice of themethod used for data integration is of cru-cial importance.

Initial studies using both electrophysi-ology and hemodynamic data often fo-cused on qualitatively or quantitativelycomparing the outcomes of simultane-ously acquired but separately analyzedEEG and fMRI data (Horovitz et al., 2004;Mulert et al., 2004). To take full advantageof the information from multimodal da-tasets, analytic approaches were neededthat allowed more direct integration of in-formation across modalities. The tech-niques currently available, however, differin terms of their necessity for prior knowl-edge as well as their underlying biophysi-cal and mathematical assumptions.

Asymmetric data integrationAsymmetric approaches for EEG-fMRIintegration are characterized by a biasedweighting of modalities in as much asinformation from one modality is usedto guide the analysis of the other. Mostinfluential of these approaches havebeen fMRI-informed EEG and EEG-informed fMRI.

fMRI-informed EEGfMRI-informed EEG (Fig. 1) aims at alle-viating the spatial EEG inverse problem byguiding electromagnetic source imagingusing results obtained from fMRI (Heinzeet al., 1994; Babiloni et al., 2000, 2002). Todo this, the head geometry and most rele-vant biophysical characteristics of thebrain (e.g., tissue conductivity) are firstestimated to establish a forward modelfrom which, given a simulated neuralevent, the paths of currents to the scalp

can be calculated (Hallez et al., 2007).These forward models differ dramaticallyin terms of their neuroanatomical detail,ranging from simple interleaved spheresto individually approximated corticalrepresentations computed from struc-tural MR images. Given such a forwardmodel and the measured data, source lo-calization algorithms can then be appliedthat try to find the most optimal constel-lation of neural generators best explainingthe observed scalp potential field. This in-verse modeling step incorporates assump-tions on how neuronal activity itself isapproximated; e.g., either by a limited num-ber of single equivalent current dipoles rep-resenting focal mass neuronal activity(Scherg and Berg, 1991; Mosher et al., 1992)or a large number of widely distributed cur-rent sources (Fig. 1; Hamalainen andIlmoniemi, 1994).

To derive concurrent spatiotemporalinformation from scalp-recorded EEG us-ing information from fMRI, for example,the number of potential EEG generators(or equivalent current dipoles) can be in-ferred from the pattern of activations infMRI maps. These dipoles are then seededto locations in the brain corresponding tolocal fMRI maxima, subsequently allow-ing the time course of neural activity foreach of these locations to be estimated.

However, many researchers prefer so-called distributed source models whichcompute a reconstruction of neuroelec-tric activity at each point in a 3D grid ofpossible current sources (Hamalainenand Ilmoniemi, 1994). In this context, thestatistical maps of the fMRI result can beused to confine the putative source spaceby providing the probability of a particu-lar region being the origin of the electro-physiological signal (Dale et al., 2000; Ouet al., 2010). Hence, whichever of thesecombinations is chosen, fMRI data help toreveal the locations while EEG-derived in-formation provides the time courses ofneural events at millisecond resolution(Vanni et al., 2004; Wibral et al., 2009,2010).

Bledowski et al. (2006), for example,gathered EEG and fMRI data, albeit dur-ing separate sessions, while subjects per-formed a visual working memory task.Subsequent to a unimodal fMRI analysis,multiple discrete dipoles were seeded tothe activation maxima as revealed fromfMRI. Using this fMRI-derived sourcemodel, the time courses of relevant brainregions could be derived with high tem-poral resolution. Thereby, the sequence ofneuronal processing stages during work-ing memory retrieval could be studiedconcurrently at both the spatial and tem-

Figure 1. Illustration of fMRI-informed EEG source reconstruction. To estimate the location and activity of active corticalpatches in the brain that lead to measurable EEG signal changes on the scalp, forward or head models are constructed fromindividual MR images. Here, volumes representing skin, skull, and brain tissue have been extracted. Based on such a model and theEEG time courses as well as corresponding scalp topographies (the pattern of EEG activity as recorded on a participant’s head), EEGsources can be inferred (inverse modeling). Statistical maps from a standard fMRI analysis are used to further constrain possiblesource constellations. The procedure used here for inverse modeling computes a high number of dipolar sources distributed acrossthe brain, each of which is characterized by its position, orientation (pointing direction of an arrow), and strength (as indicated bycoloring).

6054 • J. Neurosci., May 2, 2012 • 32(18):6053– 6060 Huster et al. • Toolbox

poral scale, revealing serial and parallelprocessing elements in working memory.

Hence, although approaches for fMRI-informed EEG are used on simultaneouslymeasured data, their strengths are mostapparent in the context of separate re-cordings. As information derived fromsingle trials is not taken into account, it isquite possible to work with EEG and fMRIdata acquired under comparable, but notnecessarily identical, experimental cir-cumstances. This allows for an individualoptimization of experimental conditionsfor each modality, for example by avoid-ing the otherwise inevitable gradient- andcardiac-related artifacts in the EEG whenmeasured concurrently with fMRI. Thevalidity of this approach strongly relies onthe twin assumptions that the fMRI signalat a given location does carry informationabout the local dipole activity and thatneural generators of scalp-recorded EEG/MEG do trigger a metabolic response(Wibral et al., 2010). While there is sub-stantial evidence that under many cir-cumstances there is indeed a tight couplingbetween electrophysiological and fMRI

markers of neuronal activity (Logothetisand Pfeuffer, 2004), situations that violatesuch assumptions can easily be conceived.This would be the case, for example, whenregions exhibiting an fMRI responseremain electrically silent at the scalp (e.g.,due to cortical “closed-field” geometry—aconstellation of sources leading to a cancel-lation of current flow; Nunez and Silber-stein, 2000) or when a stimulus-triggeredsynchronization of neuronal activity leadsto a detectable electrophysiological responseat no metabolic cost (which might be thecase when the mean firing rate of the popu-lation does not change). Whereas the firstsituation is unproblematic for an fMRI-weighted source reconstruction, the latteris not: an fMRI-based weighting willadversely affect the reconstruction ofsources in brain regions devoid of clearfMRI activations, eventually leading togross localization errors. However, bycarefully combining different methods forEEG source analyses, it might be possibleto identify such ambiguous situations(Wibral et al., 2010).

EEG-informed fMRIElaborating on the idea of a direct cou-pling between EEG and fMRI, EEG-informed fMRI considers associations ofchanges over time between EEG and fMRIsignals at a within-subject level (Fig. 2;Ullsperger, 2010). Here, EEG is prepro-cessed to a point where a specific EEG fea-ture of interest over the time course of therecording can be extracted. In event-related designs this corresponds to thequasi-continuous stream of epochs sur-rounding the events of interest (e.g., thepresentation of a stimulus or the occur-rence of a behavioral response). Severalfeatures of the EEG are suitable, such asERP amplitudes (Debener et al., 2005),ERP latencies (Benar et al., 2007), EEGsynchronization and phase coherence(Mizuhara et al., 2005; Jann et al., 2009),or the power within specific EEG fre-quency bands (Scheeringa et al., 2009).Whichever EEG-derived features are used,the basic assumption is that their fluctua-tions over time covary with fluctuationsseen in the fMRI signal during the course ofan experiment.

With this method, the processing offMRI data simply follows standard proce-dures up to the point of first-level statistics,usually including slice-time correction,spatial realignment, spatial normaliza-tion, and smoothing (Strother, 2006).When setting up the regressors for first-level statistics (i.e., calculating statisticaltests for a single subject), however, theseregressors are not derived solely from thetiming of stimulus onsets convolved witha modeled hemodynamic response. Rather,the modeled hemodynamic responses areadditionally parameterized using the trial-wise extracted EEG feature, such that an en-larged potential amplitude in the single-trialEEG leads to an upscaling of the hemody-namic response function for a given event(Fig. 2). The resulting indices representingthe model fit (beta estimates) are then testedat the group level either against other condi-tions or simply against zero to reveal brainregions exhibiting significant covariationsand, thus, coupling of electrophysiologicaland hemodynamic responses.

Debener et al. (2005), for instance,used EEG-informed fMRI to test whetherEEG correlates of performance monitor-ing (more specifically, the error-relatednegativity; ERN) predicts fMRI signalchanges in the midcingulate cortex. Foreach erroneous response recorded duringflanker task performance, the amplitudeof the single-trial ERN was determinedfrom EEG. This information was subse-quently used for a parameterized analysis

Figure 2. Schematic of EEG-informed fMRI. For an event of interest (e.g., the occurrence of a response error), a parameter valueof an EEG feature (e.g., the amplitude of an ERP) is extracted from every trial that includes this event. With respect to fMRI, theonsets of these events during the course of the experiment are known, and the fMRI signal changes caused by hemodynamicresponses following the events are mathematically modeled. This is done via convolution: the multiplication and summation of avector of zeros and ones representing the event onsets and the hemodynamic response function (mathematically modeled fMRIsignal changes following an event). The result is the predicted time course of fMRI signal changes which can then be statisticallycompared with the observed fMRI signal for each voxel (volume element) of a brain scan. In the case of EEG-informed fMRI, notonly is this model determined by event onsets and the hemodynamic response function (blue model prediction), but the expectedhemodynamic responses are additionally parameterized using the extracted EEG feature: signal changes following events on trialswith a large EEG response are scaled up as compared with trials with smaller EEG responses (green model prediction).

Huster et al. • Toolbox J. Neurosci., May 2, 2012 • 32(18):6053– 6060 • 6055

of the fMRI data. Indeed, this analysisscheme was able to prove the associationof midcingulate activations with varia-tions in ERN amplitudes while differenti-ating it from activity profiles of otherfrontal regions and the anterior insula.Moreover, single-trial amplitudes predictedtask behavior in the subsequent trial, dem-onstrating that trial-by-trial fluctuations arenot random noise but carry functional in-formation. A similar analysis technique wasdeveloped by Eichele et al. (2005) to test thetemporal sequence of ERP-related fMRIactivations.

EEG-informed fMRI assumes onlythat the neural properties contributing tothe signals captured by both modalitiespartly overlap and exhibit a linear associ-ation. In contrast to fMRI-informed EEG,no specific assumption is made about thespatial organization of activation pat-terns. This implies that this strategy canyield effects in brain structures or net-works that are not necessarily the bio-physical generators of, say, the ERPrecorded at the scalp (Debener et al., 2005;Minati et al., 2008). One also has to bear inmind that single EEG epochs contain highlevels of noise, and single-trial EEG anal-yses therefore have to rely on techniquesthat increase the signal-to-noise ratio ofindividual trials. In the frequency do-main, the consideration of a narrow fre-quency band of the EEG can be thought ofas a filtering mechanism. Alternatively a

frequently used algorithm that enables theunmixing of spatiotemporally overlap-ping EEG signals before feature extractionis independent component analysis (ICA;James and Hesse, 2005; Onton et al.,2006). In any case, only a fraction of theongoing brain activity can be used for anEEG-informed fMRI analysis at one timeand the selection of the feature of interestis usually determined by the researchquestion addressed. For example, whereasboth ERP amplitudes and latencies havebeen shown to be correlated with hemo-dynamic responses, latencies are less likelyto directly reflect activations of potentialgenerators (Benar et al., 2007).

Neurogenerative modelingThe methods described in the followingsection aim to benefit from the comple-mentary information available from bothmodalities while avoiding an a priori biasof either (Valdes-Sosa et al., 2009; Rieraand Sumiyoshi, 2010; Rosa et al., 2010).Although they require a certain amount ofprior knowledge, these approaches followlogic comparable to that of EEG sourcemodeling (Fig. 3). A forward model (orgenerative model) first concurrently spec-ifies those physiological processes thatgive rise to EEG and those that give rise tofMRI data. Based on such forward com-putations, brain states can then be recon-structed from simultaneous EEG-fMRIrecordings that best explain the observed

data (the inverse modeling). Within sucha framework, data integration relies onthe ability to simulate the biophysical pro-cesses of electrophysiological and hemo-dynamic signal generation.

As the crucial element, a forwardmodel for multimodal data incorporatesassumptions about the link between elec-trophysiological and hemodynamic activ-ities as well as the neurovascular couplingcascade (Logothetis and Wandell, 2004;Stephan et al., 2004; Raichle and Mintun,2006), i.e., the sequence of neural eventsgiving rise to changes in cerebral bloodflow that in turn cause variations in fMRIsignals. Hence, depending on richness ofdetail, different levels of description maybe involved (Fig. 3; Rosa et al., 2010),ranging from cellular mechanisms medi-ating neurovascular coupling to connec-tivity patterns at a gross morphologicallevel (Friston et al., 2008; Sotero andTrujillo-Barreto, 2008; Valdes-Sosa et al.,2009). Given real multimodal data, thesecond step of the procedure incorporatesinverse modeling: estimating neural activ-ity (e.g., EPSPs, along with other parame-ters) from the simultaneous EEG-fMRIobservations. An intuitive approach is torepeatedly generate EEG-fMRI simula-tions and choose the one with the best fit tothe observed data. However, computation-ally less demanding alternatives have beenproposed that are already in use (Friston etal., 2008; Valdes-Sosa et al., 2009).

Figure 3. Illustration of neurogenerative models for EEG-fMRI data analysis. This approach relies on mathematically modeling the dynamics of neural ensembles at various scales ranging fromgross connectivity patterns to cellular events. In addition, biophysical forward models must specify the transformation of neural events to the measured EEG and fMRI signals. Based on these data,the result of a predefined neural activity pattern can be simulated (forward modeling) or, given a multimodal EEG-fMRI dataset, the most likely neural events can be inferred based on the observedEEG-fMRI signal properties (inverse modeling).

6056 • J. Neurosci., May 2, 2012 • 32(18):6053– 6060 Huster et al. • Toolbox

The interested reader might consider arecent study by Bojak et al. (2011) whichelucidates the application of neurogenera-tive models. The authors used simulateddata to investigate the plausibility of aneural population model for the analysisof simultaneous EEG-fMRI. Althoughconfined to only few regions of the cortex,the authors were able to identify the ef-fects of connectivity changes in the visualsystem on the activity pattern of simulta-neous EEG-fMRI data. This study effec-tively demonstrates the relevance of suchmodels for questions pertaining to the ba-sic mechanisms of EEG and fMRI signalgeneration.

The conceptual similarity betweenEEG source reconstruction and EEG-fMRI data integration by means of bio-physical modeling is obvious, as is theexcelling need for prior information withthe latter. Given that many aspects of theprocesses involved in fMRI and EEG sig-nal generation are not yet fully under-stood, current models focus on specificaspects while strongly simplifying others.For example, the cellular mechanisms un-

derlying neurovascular coupling have notyet been implemented in every detail (Ri-era and Sumiyoshi, 2010). Similarly, thereconstruction of neural activity using abrain-wide model has hitherto not beenaccomplished, which is not surprisinggiven the complexity of the necessarymodels and the resulting computationaldemands. Thus, the ability to comparemodels of various complexity and differ-ing physiological specificity should beconsidered one of the biggest strengths ofthe neurogenerative approach (Rosa et al.,2010).

Multimodal data fusionOver the past years machine learning al-gorithms have successfully been appliedto both EEG and fMRI data (Besserve etal., 2007; Pereira et al., 2009). Whereasunsupervised approaches are often usedto decompose given datasets into latentvariables, e.g., for artifact correction withEEG or the identification of functionalnetworks from fMRI (Mennes et al., 2010;Joel et al., 2011), supervised approacheshave drawn much attention due to their

classification capabilities. An example isthe prediction of the category of a pre-sented stimulus from its evoked brain re-sponse (Mitchell et al., 2008; Haynes,2009). When applied to EEG-fMRI data,most of these approaches hold the advan-tage of a genuine multivariate analysisscheme: the full spectrum of availableinformation in multimodal datasets isexplored. These “data fusion” methodsuse a common or symmetric model tojointly assess information from bothmodalities, whereas “integration” ap-proaches usually overlay or bias onemodality with the other.

Relevant methods for multimodal datafusion strongly differ in the degree towhich they rely on physiological assump-tions, require prior information, or shareother similarities with the approaches dis-cussed above. ICA, for example, is an un-supervised learning method which is usedto discover hidden factors (independentcomponents) from a set of observationssuch that the identified components aremaximally statistically independent (notmerely uncorrelated; Bell and Sejnowski,1995; Hyvarinen and Oja, 2000). Here, itis assumed that the observed data origi-nate from a linear mixture of these un-derlying independent components. Anumber of frameworks have been pro-posed to use ICA for multimodal data fu-sion. Taking joint ICA (Calhoun et al.,2006, 2009; Mijovic et al., 2012) as an ex-ample, the differing biosignal modalitiesare first processed separately, and ICA issubsequently applied to examine the rela-tionships between data types (Fig. 4).Hence, fMRI statistical maps and ERPdata of all subjects are merged into a singlematrix and subjected to a joint ICA.Whereas in the unimodal case either tem-porally (EEG) or spatially (fMRI) inde-pendent components are revealed, thismultimodal approach provides a jointspatiotemporal decomposition with jointindependent components correspondingto electrophysiologically measured re-sponses (indicating the timing of signalchanges) alongside associated clusters ofactive regions (indicating spatial originsof signal changes). Using this technique,Calhoun et al. (2006) were able to exam-ine the spatiotemporal dynamics of theauditory oddball response. Activationsin auditory cortex and thalamus weremapped to the N1 auditory evoked poten-tial, whereas brainstem, temporal lobe, andmedial frontal activity showed a responseprofile corresponding to the P3 ERP.

Other methods extend the amount ofdata processed by additionally consider-

Figure 4. Schematic illustration of joint ICA. EEG and fMRI for a set of subjects are concatenated within the same matrix andsubjected to a joint ICA. This procedure helps to capture the variance in brain responses across subjects and modalities on whichbasis a decomposition to independent components can be computed. The resulting components represent spatial (fMRI) andtemporal (EEG) characteristics of brain responses. Here, the lower row shows the averaged ERP (blue) and time courses of twocomponents (red) alongside their associated spatial maps.

Huster et al. • Toolbox J. Neurosci., May 2, 2012 • 32(18):6053– 6060 • 6057

ing variations at the level of single trials,thereby not relying on prior informationfrom within-subject statistics (Martínez-Montes et al., 2004; Goldman et al., 2009;Correa et al., 2010), or by first extractingcomponents separately from each modal-ity, and only then examining the patternof cross-modal correlations (Eichele et al.,2005; De Martino et al., 2011; Huster etal., 2011).

Although the majority of algorithms ap-plied here have a long history in computerscience, their application to neuroscientificproblems is a rather new development.Nonetheless, these methods have proven tobe of high utility by providing unbiasedand multivariate analysis schemes for si-multaneous EEG-fMRI. In addition,many of these approaches are, or couldbe, easily generalized to frameworks thatinclude other modalities, such as diffusiontensor imaging or genetic data (for review,see De Martino et al., 2010; Sui et al., 2012).On the other hand, as there is already a va-riety of different mathematical approachesfor multimodal data fusion, the selection ofa fusion model tailored for a specific re-search question might sometimes be diffi-cult to accomplish. Moreover, some of thecurrent approaches still rely on canonicalmodels of the hemodynamic response or in-volve other physiological presumptions,thereby not yet taking full advantage of thefull power of these algorithms.

ConclusionDespite its rather recent development, avariety of differing methods have alreadybeen proposed for the concurrent analysisof EEG and fMRI data. The taxonomy ap-plied here provides a natural startingpoint from which to better judge the po-tential advantages and limitations of suchanalysis approaches. However, a givenprocedure might very well share featuresof more than one of the categories pro-posed here. Ultimately, the method cho-sen will strongly depend on the researchquestion addressed.

When one is primarily interested in theneural generators of scalp EEG phenom-ena, fMRI-informed EEG is the method ofchoice. This approach is founded on wellestablished methods for EEG source re-construction, and so constitutes the mostdirect technique for assessing relatedresearch questions. Additionally, it pos-sesses high and well defined spatiotempo-ral resolution. It does not, however, takeinto account the variability of neuralevents seen across the course of an exper-imental session, and thereby neglects apotentially powerful source of informa-

tion. Analyses based on EEG-informedfMRI, on the other hand, use this exactphenomenon and have already shown theability not only to link physiological mea-sures of different modalities with eachother, but also to expose associations be-tween physiology and variations in cogni-tion, perception, and behavior. However,the temporal precision available withthis technique is somewhat elusive, as itcould potentially reveal neuroanatomicalstructures whose activity patterns are lin-early linked to those of the generators,even though their engagement might pre-cede or succeed the EEG event under in-vestigation. Hence, EEG-informed fMRIseems most appropriate for revealingfunctional networks characterized by alinear relationship of cross-modal activitypatterns identified at the single trial level.Neurogenerative models are particularlystrong for testing hypotheses about thephysiological mechanisms and biophysi-cal properties underlying EEG and fMRIsignal generation, as well as their interre-lationship, making these models powerfultools in the field of theoretical neurobiol-ogy. Although the application of suchmodels to whole brain analyses and com-plex experimental paradigms is currentlylacking, its spatiotemporal resolution is inprinciple limited only by the precision ofmathematical modeling and computa-tional power. Finally, turning to the appli-cation of multivariate fusion methods forsimultaneous EEG-fMRI, a definite rec-ommendation is somewhat hindered bythe algorithmic heterogeneity of this field.Associated approaches are well suited toaddress a variety of research questions asdivergent as, for example, the identifica-tion of links between modalities, the pre-diction of behavior jointly from EEG andfMRI, or the exposure of hidden factorscommon to both modalities. Here, unsu-pervised learning algorithms with theirdata mining capabilities might thus con-stitute a counterpart to the model-drivenneurogenerative approaches.

ReferencesBabiloni F, Carducci F, Cincotti F, Del Gratta C,

Roberti GM, Romani GL, Rossini PM, BabiloniC (2000) Integration of high resolution EEGand functional magnetic resonance in the studyof human movement-related potentials. Meth-ods Inf Med 39:179–182.

Babiloni F, Babiloni C, Carducci F, Del Gratta C,Romani GL, Rossini PM, Cincotti F (2002)Cortical source estimate of combined high reso-lution EEG and fMRI data related to voluntarymovements. Methods Inf Med 41:443–450.

Bell AJ, Sejnowski TJ (1995) An information-maximization approach to blind separation

and blind deconvolution. Neural Comput7:1129 –1159.

Benar CG, Schon D, Grimault S, Nazarian B,Burle B, Roth M, Badier JM, Marquis P,Liegeois-Chauvel C, Anton JL (2007) Single-trial analysis of oddball event-related poten-tials in simultaneous EEG-fMRI. Hum BrainMapp 28:602– 613.

Besserve M, Jerbi K, Laurent F, Baillet S, Martin-erie J, Garnero L (2007) Classification meth-ods for ongoing EEG and MEG signals. BiolRes 40:415– 437.

Bledowski C, Cohen Kadosh K, Wibral M, RahmB, Bittner RA, Hoechstetter K, Scherg M,Maurer K, Goebel R, Linden DE (2006)Mental chronometry of working memory re-trieval: a combined functional magnetic reso-nance imaging and event-related potentialsapproach. J Neurosci 26:821– 829.

Bojak I, Oostendorp TF, Reid AT, Kotter R(2011) Towards a model-based integration ofregistered electroencephalography/functionalmagnetic resonance imaging data with realis-tic neural population meshes. Philos TransactMath Phys Eng Sci 369:3785–3801.

Calhoun VD, Adali T, Pearlson GD, Kiehl KA(2006) Neuronal chronometry of target de-tection: fusion of hemodynamic and event-related potential data. Neuroimage30:544 –553.

Calhoun VD, Liu J, Adali T (2009) A review ofgroup ICA for fMRI data and ICA for jointinference of imaging, genetic, and ERP data.Neuroimage 45:163–172.

Correa NM, Eichele T, Adali T, Li YO, CalhounVD (2010) Multi-set canonical correlationanalysis for the fusion of concurrent singletrial ERP and functional MRI. Neuroimage50:1438 –1445.

Dale AM, Liu AK, Fischl BR, Buckner RL, BelliveauJW, Lewine JD, Halgren E (2000) Dynamicstatistical parametric mapping: combiningfMRI and MEG for high-resolution imaging ofcortical activity. Neuron 26:55–67.

Debener S, Ullsperger M, Siegel M, Fiehler K, vonCramon DY, Engel AK (2005) Trial-by-trialcoupling of concurrent electroencephalogramand functional magnetic resonance imagingidentifies the dynamics of performance mon-itoring. J Neurosci 25:11730 –11737.

Debener S, Strobel A, Sorger B, Peters J, Kranczi-och C, Engel AK, Goebel R (2007) Improvedquality of auditory event-related potentials re-corded simultaneously with 3-T fMRI: re-moval of the ballistocardiogram artefact.Neuroimage 34:587–597.

Debener S, Mullinger KJ, Niazy RK, Bowtell RW(2008) Properties of the ballistocardiogramartefact as revealed by EEG recordings at 1.5, 3and 7 T static magnetic field strength. IntJ Psychophysiol 67:189 –199.

De Martino F, Valente G, de Borst AW, EspositoF, Roebroeck A, Goebel R, Formisano E(2010) Multimodal imaging: an evaluation ofunivariate and multivariate methods for si-multaneous EEG/fMRI. Magn Reson Imaging28:1104 –1112.

De Martino F, de Borst AW, Valente G, Goebel R,Formisano E (2011) Predicting EEG singletrial responses with simultaneous fMRI andRelevance Vector Machine regression. Neuro-image 56:826 – 836.

6058 • J. Neurosci., May 2, 2012 • 32(18):6053– 6060 Huster et al. • Toolbox

Eichele T, Specht K, Moosmann M, JongsmaMLA, Quiroga RQ, Nordby H, Hugdahl K(2005) Assessing the spatiotemporal evolu-tion of neuronal activation with single-trialevent-related potentials and functional MRI.Proc Natl Acad Sci U S A 102:17798 –17803.

Friston KJ, Trujillo-Barreto N, Daunizeau J(2008) DEM: a variational treatment of dy-namic systems. Neuroimage 41:849 – 885.

Goldman RI, Stern JM, Engel J Jr, Cohen S (2002)Simultaneous EEG and fMRI of the alpharhythm. Neuroreport 13:2487–2492.

Goldman RI, Wei CY, Philiastides MG, GersonAD, Friedman D, Brown TR, Sajda P (2009)Single-trial discrimination for integrating si-multaneous EEG and fMRI: identifying corticalareas contributing to trial-to-trial variabil-ity in the auditory oddball task. Neuroimage47:136 –147.

Gotman J, Pittau F (2011) Combining EEG andfMRI in the study of epileptic discharges. Epi-lepsia 52 [Suppl 4]:38 – 42.

Grech R, Cassar T, Muscat J, Camilleri KP, FabriSG, Zervakis M, Xanthopoulos P, Sakkalis V,Vanrumste B (2008) Review on solving theinverse problem in EEG source analysis.J Neuroeng Rehabil 5:25.

Hallez H, Vanrumste B, Grech R, Muscat J, DeClercq W, Vergult A, D’Asseler Y, CamilleriKP, Fabri SG, Van Huffel S, Lemahieu I(2007) Review on solving the forward prob-lem in EEG source analysis. J Neuroeng Reha-bil 4:46.

Hamalainen MS, Ilmoniemi RJ (1994) Inter-preting magnetic fields of the brain: minimumnorm estimates. Med Biol Eng Comput32:35– 42.

Haynes JD (2009) Decoding visual conscious-ness from human brain signals. Trends CognSci 13:194 –202.

Heinze HJ, Mangun GR, Burchert W, Hinrichs H,Scholz M, Munte TF, Gos A, Scherg M, Jo-hannes S, Hundeshagen H (1994) Com-bined spatial and temporal imaging of brainactivity during visual selective attention in hu-mans. Nature 372:543-546.

Herrmann CS, Debener S (2008) Simultaneousrecording of EEG and BOLD responses: a his-torical perspective. Int J Psychophysiol67:161–168.

Horovitz SG, Rossion B, Skudlarski P, Gore JC(2004) Parametric design and correlationalanalyses help integrating fMRI and electro-physiological data during face processing.Neuroimage 22:1587–1595.

Huster RJ, Eichele T, Enriquez-Geppert S, Woll-brink A, Kugel H, Konrad C, Pantev C (2011)Multimodal imaging of functional networksand event-related potentials in performancemonitoring. Neuroimage 56:1588 –1597.

Hyvarinen A, Oja E (2000) Independent com-ponent analysis: algorithms and applications.Neural Netw 13:411– 430.

Ives JR, Warach S, Schmitt F, Edelman RR,Schomer DL (1993) Monitoring the pa-tient’s EEG during echo planar MRI. Electro-encephalogr Clin Neurophysiol 87:417– 420.

James CJ, Hesse CW (2005) Independent compo-nent analysis for biomedical signals. PhysiolMeas 26:15–39.

Jann K, Dierks T, Boesch C, Kottlow M, Strik W,Koenig T (2009) BOLD correlates of EEG al-

pha phase-locking and the fMRI default modenetwork. Neuroimage 45:903–916.

Joel SE, Caffo BS, van Zijl PCM, Pekar JJ (2011)On the relationship between seed-based andICA-based measures of functional connectiv-ity. Magn Reson Med 66(3):644 – 657.

Kruggel F, Wiggins CJ, Herrmann CS, von Cra-mon DY (2000) Recording of the event-related potentials during functional MRI at3.0 Tesla field strength. Magn Reson Med44:277–282.

Kruggel F, Herrmann CS, Wiggins CJ, von CramonDY (2001) Hemodynamic and electroenceph-alographic responses to illusory figures: record-ing of the evoked potentials during functionalMRI. Neuroimage 14:1327–1336.

Laufs H (2008) Endogenous brain oscillationsand related networks detected by surfaceEEG-combined fMRI. Hum Brain Mapp29:762–769.

Laufs H, Krakow K, Sterzer P, Eger E, Beyerle A,Salek-Haddadi A, Kleinschmidt A (2003)Electroencephalographic signatures of atten-tional and cognitive default modes in sponta-neous brain activity fluctuations at rest. ProcNatl Acad Sci U S A 100:11053–11058.

Logothetis NK (2008) What we can do and whatwe cannot do with fMRI. Nature453:869 – 878.

Logothetis NK, Pfeuffer J (2004) On the natureof the BOLD fMRI contrast mechanism.Magn Reson Imaging 22:1517–1531.

Logothetis NK, Wandell BA (2004) Interpretingthe BOLD signal. Annu Rev Physiol66:735–769.

Martínez-Montes E, Valdes-Sosa PA, MiwakeichiF, Goldman RI, Cohen MS (2004) Concur-rent EEG/fMRI analysis by multiway PartialLeast Squares. Neuroimage 22:1023–1034.

Mennes M, Wouters H, Vanrumste B, Lagae L,Stiers P (2010) Validation of ICA as a tool toremove eye movement artifacts from EEG/ERP. Psychophysiology 47:1142–1150.

Michel CM, Murray MM, Lantz G, Gonzalez S,Spinelli L, Grave de Peralta R (2004) EEGsource imaging. Clin Neurophysiol115:2195–2222.

Mijovic B, Vanderperren K, Novitskiy N, VanrumsteB, Stiers P, Van den Bergh B, Lagae L, Sunaert S,Wagemans J, Van Huffel S, De Vos M (2012)The “why” and “how” of JointICA: results froma visual detection task. Neuroimage 60:1171–1185.

Minati L, Rosazza C, Zucca I, D’Incerti L, ScaioliV, Bruzzone MG (2008) Spatial correspon-dence between functional MRI (fMRI) activa-tions and cortical current density maps ofevent-related potentials (ERP): a study withfour tasks. Brain Topogr 21:112–127.

Mitchell TM, Shinkareva SV, Carlson A, ChangKM, Malave VL, Mason RA, Just MA (2008)Predicting human brain activity associatedwith the meanings of nouns. Science320:1191–1195.

Mizuhara H, Wang LQ, Kobayashi K, Yamagu-chi Y (2005) Long-range EEG phase syn-chronization during an arithmetic taskindexes a coherent cortical network simul-taneously measured by fMRI. Neuroimage27:553–563.

Moosmann M, Ritter P, Krastel I, Brink A, Thees S,Blankenburg F, Taskin B, Obrig H, Villringer A

(2003) Correlates of alpha rhythm in functionalmagnetic resonance imaging and near infraredspectroscopy. Neuroimage 20:145–158.

Mosher JC, Lewis PS, Leahy RM (1992) Multipledipole modeling and localization from spatio-temporal MEG data. IEEE Trans Biomed Eng39:541–557.

Mulert C, Lemieux L (2009) EEG-fMRI: physio-logical basis, technique, and applications, Ed1. Berlin: Springer.

Mulert C, Jager L, Schmitt R, Bussfeld P, PogarellO, Moller HJ, Juckel G, Hegerl U (2004) In-tegration of fMRI and simultaneous EEG:towards a comprehensive understanding oflocalization and time-course of brain activityin target detection. Neuroimage 22:83–94.

Mulert C, Pogarell O, Hegerl U (2008) Simulta-neous EEG-fMRI: perspectives in psychiatry.Clin EEG Neurosci 39:61– 64.

Mullinger KJ, Yan WX, Bowtell R (2011) Reduc-ing the gradient artefact in simultaneous EEG-fMRI by adjusting the subject’s axial position.Neuroimage 54:1942–1950.

Mullinger K, Debener S, Coxon R, Bowtell R(2008) Effects of simultaneous EEG record-ing on MRI data quality at 1.5, 3 and 7 tesla.Int J Psychophysiol 67:178 –188.

Nunez PL, Silberstein RB (2000) On the rela-tionship of synaptic activity to macroscopicmeasurements: does co-registration of EEGwith fMRI make sense? Brain Topogr13:79 –96.

Onton J, Westerfield M, Townsend J, Makeig S(2006) Imaging human EEG dynamics usingindependent component analysis. NeurosciBiobehav Rev 30:808 – 822.

Ou W, Nummenmaa A, Ahveninen J, BelliveauJW, Hamalainen MS, Golland P (2010)Multimodal functional imaging using fMRI-informed regional EEG/MEG source estima-tion. Neuroimage 52:97–108.

Pereira F, Mitchell T, Botvinick M (2009) Ma-chine learning classifiers and fMRI: a tutorialoverview. Neuroimage 45:S199 –S209.

Raichle ME, Mintun MA (2006) Brain work andbrain imaging. Annu Rev Neurosci29:449 – 476.

Riera JJ, Sumiyoshi A (2010) Brain oscillations:ideal scenery to understand the neurovascularcoupling. Curr Opin Neurol 23:374 –381.

Rosa MJ, Daunizeau J, Friston KJ (2010) EEG-fMRI integration: a critical review of biophys-ical modeling and data analysis approaches.J Integr Neurosci 9:453– 476.

Scheeringa R, Petersson KM, Oostenveld R, Nor-ris DG, Hagoort P, Bastiaansen MCM (2009)Trial-by-trial coupling between EEG andBOLD identifies networks related to alpha andtheta EEG power increases during working mem-ory maintenance. Neuroimage 44:1224–1238.

Scherg M, Berg P (1991) Use of prior knowledgein brain electromagnetic source analysis.Brain Topogr 4:143–150.

Sotero RC, Trujillo-Barreto NJ (2008) Biophys-ical model for integrating neuronal activity,EEG, fMRI and metabolism. Neuroimage39:290 –309.

Stephan KE, Harrison LM, Penny WD, Friston KJ(2004) Biophysical models of fMRI re-sponses. Curr Opin Neurobiol 14:629 – 635.

Huster et al. • Toolbox J. Neurosci., May 2, 2012 • 32(18):6053– 6060 • 6059

Strother SC (2006) Evaluating fMRI prepro-cessing pipelines. IEEE Eng Med Biol Mag25:27– 41.

Sui J, Adali T, Yu Q, Chen J, Calhoun VD (2012)A review of multivariate methods for multi-modal fusion of brain imaging data. J Neuro-sci Methods 204:68 – 81.

Ullsperger M (2010) EEG-informed fMRIanalysis. In: Simultaneous EEG and fMRI:recording, analysis, and application (UllspergerM, Debener S, eds), pp 153–160. New York: Ox-ford UP.

Ullsperger M, Debener S (2010) Simultaneous

EEG and fMRI: recording, analysis, and appli-cation. New York: Oxford UP.

Valdes-Sosa PA, Sanchez-Bornot JM, Sotero RC,Iturria-Medina Y, Aleman-Gomez Y, Bosch-Bayard J, Carbonell F, Ozaki T (2009) Modeldriven EEG/fMRI fusion of brain oscillations.Hum Brain Mapp 30:2701–2721.

Vanni S, Warnking J, Dojat M, Delon-Martin C,Bullier J, Segebarth C (2004) Sequence ofpattern onset responses in the human visualareas: an fMRI constrained VEP source analy-sis. Neuroimage 21:801– 817.

Wibral M, Bledowski C, Kohler A, Singer W,

Muckli L (2009) The timing of feedback toearly visual cortex in the perception of long-range apparent motion. Cereb Cortex19:1567–1582.

Wibral M, Bledowski C, Turi G (2010) Integra-tion of separately recorded EEG/MEG andfMRI data. In: Simultaneous EEG and fMRI:recording, analysis, and application (Ull-sperger M, Debener S, eds), pp 209 –234.New York: Oxford UP.

Yan WX, Mullinger KJ, Brookes MJ, Bowtell R (2009)Understanding gradient artefacts in simultane-ous EEG/fMRI. Neuroimage 46:459–471.

6060 • J. Neurosci., May 2, 2012 • 32(18):6053– 6060 Huster et al. • Toolbox