DD143X EXAMENSARBETE INOM DATALOGI,GRUNDNIVÅ, 15 HPSTOCKHOLM, SVERIGE 2016

Heatmap Visualization ofNeural Frequency DataVisualisering av neural frekvensdata somvärmekarta

RODRIGO ROA RODRÍGUEZ

ROBERT LUNDIN

KTHSKOLAN FÖR DATAVETENSKAP OCH KOMMUNIKATION

Supervisor: Alexander KozlovExaminer: Örjan Ekeberg

Abstract

Complex spatial relationships and patterns in multivariate data are

relatively simple to identify visually but di�cult to detect computation-

ally. For this reason, Anivis, an interactive tool for visual exploration of

multivariate quantitative pure serial periodic data was developed. The

data has four dimensions depth, laterality, frequency and time. The data

was visualized as an animated heatmap, by mapping depth and laterality

to coordinates in a pixel grid and frequency to color. Transfer functions

were devised to map a single variable to color through parametric curves.

Anivis implemented heatmap generation through both weighted sum and

deconvolution for comparison reasons. Deconvolution exhibited a to have

better theoretical and practical performance. In addition to the heatmap

visualization a scatter-plot was added in order to visualize the causal

relationships between data points and high value areas in the heatmap

visualization. Performance and applicability of the tool were tested and

verified on experimental data from the Karolinksa Institute’s Department

of Neuroscience.

Abstrakt

Komplexa spatiala monster och forhallanden i multivariat data ar rel-

ativt svara att identifiera via berakningar men simpla att identifiera vi-

suellt. Att visualisera data for denna typ av data-analys anvands ofta i

manga olika typer av falt. Detta motiverade utvecklingen av Anivis; ett

interaktivt verktyg for visuell utforskning av multivariat kvantitativ data

av neural aktivitet. Anivis anvander sig av dataset baserade pa experi-

mentell data fran en forskningsgrupp pa Karolinska Institutets Institution

for Neurovetenskap. Dessa fyrdimensionella dataset bestar av matningar

fran neuroner i form av deras position, aktivitet i form av frekvens och

tidpunkt. Denna data anvands for att generera en animerad heatmap,

dar neuroners frekvensvarden visas i form av farg. Frekvensvardena om-

vandlades till fargvarden via overgangsfunktioner som kopplar numeriska

varden till fargvarden via parametriserade kurvor. Anivis lyckades imple-

mentera tva olika metoder for att generera heatmap, viktade summor och

dekonvolution. Dessa tva metoder jamfordes med varandra, varav dekon-

volution visade sig vara den teoretiskt och praktiskt e↵ektivaste meto-

den. Utvecklingen av Anivis visade aven behovet for ett punktdiagram

for att visualisera forhallandet mellan matta frekvensvarden och spatial

frekvensfordelning i heatmappen.

Contents

1 Introduction 1

1.1 Problem statement . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2 Scope and limitations . . . . . . . . . . . . . . . . . . . . . . . . 2

1.3 Disposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Background 4

2.1 Neurons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2.2 Reflexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2.3 Previous studies . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2.4 Visualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.5 Color Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.5.1 Monochrome . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.5.2 RGB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.5.3 HSL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.5.4 HCL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.6 Color transfer functions . . . . . . . . . . . . . . . . . . . . . . . 11

2.7 Heatmaps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.8 Kernel image filtering . . . . . . . . . . . . . . . . . . . . . . . . 13

3 Method 14

3.1 Data structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

3.2 Visual mappings . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

3.2.1 Mapping depth and laterality to on screen coordinates . . 15

3.2.2 Mapping frequency to color . . . . . . . . . . . . . . . . . 15

3.3 Heatmap calculation . . . . . . . . . . . . . . . . . . . . . . . . . 16

3.4 Animation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

3.5 Interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

4 Results 18

4.1 Tools and design choices . . . . . . . . . . . . . . . . . . . . . . . 18

4.2 Scatter-plot implementation . . . . . . . . . . . . . . . . . . . . . 19

4.3 Heatmap implementation . . . . . . . . . . . . . . . . . . . . . . 20

4.3.1 Weighted sum . . . . . . . . . . . . . . . . . . . . . . . . . 20

4.3.2 Deconvolution . . . . . . . . . . . . . . . . . . . . . . . . 21

4.4 Color transfer functions . . . . . . . . . . . . . . . . . . . . . . . 23

4.4.1 Grayscale . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

4.4.2 HSL Rainbow . . . . . . . . . . . . . . . . . . . . . . . . . 24

4.4.3 Cubehelix . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

4.4.4 HCL Heat . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

4.5 Rendering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

4.6 Animation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

4.6.1 Animation loop . . . . . . . . . . . . . . . . . . . . . . . . 28

4.6.2 Frame rate capping . . . . . . . . . . . . . . . . . . . . . . 28

4.7 Interactivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

5 Discussion 30

5.1 Heatmaps in Anivis . . . . . . . . . . . . . . . . . . . . . . . . . 30

5.2 Image quality of deconvolution . . . . . . . . . . . . . . . . . . . 31

5.3 Scatter-plot as a complement to heatmap visualization . . . . . 32

5.4 Future research . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

6 Conclusion 33

1 Introduction

In neuroscience, the study of reflex related neural activity seeks to correlateactivity patterns with the di↵erent reflexes. Neural activity consists of oscilla-tions made of individual neuron spikes. Experiments by Zelenin, Hsu, Lyalka,Orlovsky, and Deliagina (2014) suggest that the neuron activity overarchinglymatches motor response. This makes it desirable to explore the data in terms ofspatial and temporal trends rather than numerical values of individual neurons.Stimuli induced patterns of neural activity must be isolated from backgroundactivity (Zelenin et al., 2014). Heatmaps could be suitable for this purposesince complex spatial relationships and patterns in multivariate data are rel-atively simple to identify visually but di�cult to detect computationally, andparticularly in the case of interactive visualizations (Rheingans, 1992).

Heatmaps are a popular visual structure for encoding quantitative intensityvalues spatially as color (Duchowski, Price, Meyer, & Orero, 2012). Whileoriginally developed to illustrate financial market information, the visual struc-ture has gained wide adoption in biotechnology and medicine (Akers, 2015).This might be because heatmaps produce similar visualizations to those re-sulting from well-established imaging techniques such as computed tomogra-phy(CT) scans, where color transfer functions can be used to di↵erentiate be-tween anatomical components such as bone, cartilage, muscle and blood vesselsby means of mapping densities to di↵erent colors (for an example see Kindlmannet al., 2005). In regards of brain activity, heatmaps produce similar visualiza-tions to positron emission tomography(PET) which assesses neural activity indi-rectly by measuring blood flow or other metabolic processes in di↵erent regionsof the brain (Handbook of Laboratory and Diagnostic Tests, 2013).

Figure 1: (A) A 3D reconstruction of the brain and eyes from a CT scan, (B) Atransaxial slice of the brain from a PET scan and (C) a heatmap visualizationof neural frequency data. Sources: (A) By Dale Mahalko - Own work, CCBY-SA 3.0. (B) By Jens Maus - Own work, Public Domain. (C) By PavelZelenin - Own work, all rights reserved.

Although static visualizations of neural activity data as heatmaps have previ-

1

ously been employed by Zelenin et al. (2014), an exploratory search of currentliterature in neuroscience failed to reveal any similar attempts by other re-searchers. This visualization is denominated as static because Zelenin et al.(2014) has created individual images rather than an animation or an interac-tive program. While animated visualizations of neural activity were found incurrent literature, these come from imaging living specimens and do not rep-resent frequency(e.g. Fetcho & O’Malley, 1995; Muto, Ohkura, Abe, Nakai, &Kawakami, 2013). The only interactive visualization of neuron frequency datathat was found in literature was an staple diagram that was not animated,did not include spatial information and only presented deviation from meanfrequency by individual neurons (Carlis & Konstan, 1998).

This project aims to develop spatiotemporal visualization of neural frequencydata in the form an interactive animated heatmap. The purpose of this visual-ization is to be used as a tool in data analysis, assisting in pattern recognitionand hypothesis formulation.

1.1 Problem statement

Complex spatial relationships and patterns in multivariate data are di�cult todetect computationally but relatively simple to identify visually, so the goal ofthis project will be to visually represent multivariate neural frequency data asan interactive animated heatmap.

1.2 Scope and limitations

The method of visualization will be theorized first and then implemented as anapplication. The implementation will be assessed in terms of time and memorycomplexity and image quality. Furthermore, optimizations and heuristics willbe attempted to improve their performance.

This project will not analyze or investigate the characteristics of activity dis-tribution between neurons. Although the data originates in a study in neuro-science, this project does not intend to conduct any further research in thatfield. The dataset’s significance to this project is limited to the numerical val-ues recorded from the experiments. What is of interest in this case is not theinterpretation of the values nor their medical and biological connotations, butthe structure of the data from a computer science perspective. This projectwill also not analyze how to best visually represent or highlight distributiontrends across neurons nor how the visualization impacts data analysis. Instead,this report will discuss how the data is processed in order to visualize it as aheatmap.

2

1.3 Disposition

This report will document the findings from the development of the Anivisprogram.

Section 2 Background will provide underlying the theory in both neuroscienceand data visualization on which Anivis is built upon. It will also present theneurological studies behind the neural frequency data.

Section 3 Method will describe in detail the structure of the neural data aswell as the theoretical reasoning behind the implementation of Anivis.

Section 4 Results presents the resulting implementation of the theory andfunctionality described in section 3, as well as any insights gained from theimplementation process.

Section 5 Discussion will overweight from a critical standpoint the chosenmethods of implementation from section 4 as well consider possible alternativesolutions and future research.

Section 6 Conclusion will present a general assessment of the report based onsection 3, 4 and 5.

3

2 Background

This section will present the underlying theory in neuroscience and visualization.An introduction in neuron and reflex theory is followed by previous studies inneuroscience that studied postural limb reflex induced neuron activity. This sec-tion will also explain the need for information visualization, introduce heatmapsas a visual structure for representation of data and how transfer functions con-vert quantitative values to color.

2.1 Neurons

Neurons are cells that transfer information throughout the body by the means ofelectrical or chemical impulses. Information is exchanged between neurons viainterconnected neural networks. Neural activity consists of oscillations whereneurons spike individually (Goldstein, 2014).

2.2 Reflexes

Reflexes are unconsciously triggered bodily movements or actions built in tothe nervous system. It has been shown that some reflexes still function inbodies devoid of consciousness or brain activity, mainly those located in thespine (Spittler, Wortmann, von During, & Gehlen, 2000). Futhermore, somestudies have suggested that spinal nerves can generate neural activity to supportpostural activities without input from the cortex (Honeycutt, Gottschall, &Nichols, 2009)

2.3 Previous studies

This project builds upon earlier research on neural activity in relation to postu-ral reflexes, specifically on the experiments realized by Hsu, Zelenin, Orlovsky,and Deliagina (2012) and Zelenin et al. (2014).

Hsu et al. (2012) studied the neuron activity in relation to activation of posturallimb reflexes (PLR). The study was based on measurements of firing frequenciesof neurons in a gray matter section in the spinal cord of decerebrated quadrupedssubjected to galvanic vestibular stimulation. The activation of PLR evoked neu-ral activity which were recorded through electrodes implanted in the gray matterarea of a cross-section of the spinal cord. With its forelimbs suspended in a ham-mock, the decerebrated quadruped’s hindlimbs where positioned in a hemiflexed

4

stance on a horizontal platform. The platform consisted of a left and a rightpart, both of which were capable of rotation; producing the hind limbs to tiltaccordingly. Test recording consisted of the readings from tilting the platform ina angle and then returning to the the starting position, producing a movementcycle. The tilt in the platform resulted in flexion-extension movements of thehips and hind limbs, which evoked PLRs.

The neuron activity are measured by the frequency of impulse transmissionsflowing trough the neuron, also called firing frequency. Each recording wasdivided into 12 bins by grouping by the movement performed during the test.For each bin, the neuron firing frequencies were calculated and averaged overall identical bins from all movements cycles. The bins were then grouped intoflexion and extension bins. The firing frequency for each neuron was calculatedby averaging each group and then taking the mean of the flexion and extensionbins.

From these measurements, the authors grouped the neurons into two groupsbased on their firing frequency during limb extension and flexion, named E- andF-neurons respectively. The authors also stated that the findings indicated thatthe spinal neurons are contributing to the generation of PLRs.

Zelenin et al. (2014), conducted a follow-up study to (Hsu et al., 2012) withsimilar experimental design, frequency calculations as well as grouping by E-and F-neuron classification. (Zelenin et al., 2014) aimed to study and revealdistribution trends of neurons across the gray matter in the spinal cord basedPLR-related activity and additionally tried to estimate the PLR-related neuronsactivity to the sensory input from the limbs of the body.

To reveal spatial trends in neural activity a heatmap was generated based onmeans of frequencies of each neuron. Local means for each point in space werecalculated through weighted average (eq.1).

f =

nPi=1

wifi

nPi=1

wi

(1)

To interpolate in between neurons, the frequency should decay in relationshipto distance to the recorded neuron. Frequencies were treated as if normallydistributed in space, so Gaussian weights were utilized (eq.2).

w(d) = e�d

2

D

2 (2)

Where d is the distance to a recorded neuron and D is the standard deviationwhich was arbitrarily set to 0.4 by Zelenin et al. (2014). The local frequencymeans were then mapped to color by a transfer function, using MATLAB’s jetcolor map.

5

The results by Zelenin et al. (2014) suggest that neuron-wise activity matchesmotor response and provides further evidence for spinal neurons contributingto the generation of PLRs. Furthermore they suggested an expansion of eachof the previous E- and F-neurons groupings into four di↵erent groups basedon the pattern of activity during the extension and flexing motions. Theseresults motivate the interest in motor control research towards the analysis ofspatiotemporal patterns of activity in neural populations within the spinal cord.

2.4 Visualization

Visualization can be defined as visual representations of data that amplify hu-man cognition (Card, Mackinlay, & Shneiderman, 1999). Besides a mean forconveying both abstract and concrete ideas, visualization can also be used to elu-cidate the nature of the data itself. More specifically, information visualizationrefers to visualizations that represent data visually in order to take advantageof perceptual qualities of human vision such as parallel processing, grouping,abstraction and aggregation (Tory & Moller, 2004).

It could be argued that visualization is unnecessary since statistical analysisof data is su�cient to find underlying trends in the data and its distribution.However, in Graphs in Statistical Analysis, Anscombe (1973) demonstrated theimportance of visualizing data by showing that numerical data alone could bemisleading. This was exemplified through four datasets known as Ascombe’sQuartet (see table 2.4).

I II III IVx1 y1 x2 y2 x3 y3 x4 y410 8.04 10 9.14 10 7.46 8 6.588 6.95 8 8.14 8 6.77 8 5.7613 7.58 13 8.74 13 12.74 8 7.719 8.81 9 8.77 9 7.11 8 8.8411 8.33 11 9.26 11 7.81 8 8.4714 9.96 14 8.1 14 8.84 8 7.046 7.24 6 6.13 6 6.08 8 5.254 4.26 4 3.1 4 5.39 19 12.512 10.84 12 9.13 12 8.15 8 5.567 4.82 7 7.26 7 6.42 8 7.915 5.68 5 4.74 5 5.73 8 6.89

Table 1: Ascombe’s Quartet

What is is significant about Ascombe’s Quartet is that though all four datasetshave the same sample mean, variance, correlation coe�cient and linear regres-sion, their graphical representation promptly reveals them to be widely di↵erent

6

in nature (see figure 2).

Figure 2: Ascombe’s quartet visualized as scatter-plots. The blue lines are thelinear regression of the data-points.

This raises the need for tools for overviewing complex data so that patternsand anomalies can be found. However, there are many hurdles that need tobe overcome in order to visualize neural activity patterns. For one, the visualrepresentation of multidimensional data, such as neural activity through time, isgraphically challenging; as it requires the representation of a higher dimensionaldomain in a two dimensional plane. This usually accomplished through the useof abstraction where additional dimensions are mapped to color and geometry(Yau, 2014).

2.5 Color Spaces

Color spaces, also refered to as color models, are mathematical abstractionsof color. Instead of representing colors as wavelengths in the electromagneticspectrum, color spaces represent color perceptually as linear combinations ofbasis vectors.

7

2.5.1 Monochrome

The simplest imaginable color space able to represent di↵erent numerical valuesconsists of an individual basis vector. Due to there being no other basis vectors,all colors are just the basis vector multiplied with a constant coe�cient.

While simple to implement, monochrome might not be desirable for visualizationsince an untrained human with normal color vision can discriminate a few millioncolors (Leong, 2006) but only about 30 shades of gray (Kreit et al., 2012, for apractical example see figure 3).

Figure 3: 50 shades of gray in a sequential disposition. Any two consecutiveshades might be indistinguishable to the human eye.

2.5.2 RGB

The RGB color space derives its name from the three components that it utilizesas basis vectors: red, green and blue. Hence, the color space elicits a cubegeometry (see figure 4). These three colors are used as primary colors. Di↵erentamounts of light of the primary colors are added in order to create all possibleoutput colors, hence this is an additive color model.

8

Figure 4: The RGB color space. Source: By Jacob Rus, SharkD, derivativework, CC BY-SA 3.0, via Wikimedia Commons

RGB is commonly used in computer systems since it is the de facto method forspecifying colors on a cathode-ray tube or other light-emitting displays (OxfordReference, 2008).

However, RGB is counter-intuitive to people accustomed to subtractive colormodels (PhotonStarTecnology Ltd., 2016). People might even be unaware of theexistence of additive color models work since subtractive color mixing as thatis what is encountered in paints, dyes, ink, pigments and other media. Hence,predicting the resulting color from the parametrization can be a challengingtask. It has even been argued that it is virtually impossible since the dimensionsdo not correspond to any perceptual properties of the resulting color (Zeileis,Hornik, & Murrell, 2009).

2.5.3 HSL

HSL is a rearrangement of the RGB geometry to be more intuitive and perceptu-ally sensible. In this case the rearrangement consists of using polar coordinatesto represent the color space in the form of a cylinder (see figure 5).

HSL is considered a natural representation color model since colors are de-composed based on physiological characteristics (such as hue, saturation andluminance) (Sarifuddin & Missaoui, 2005). However, the luminance componentof the HSL model, lightness, does not match human perception (Sarifuddin &

9

Missaoui, 2005). This is because HSL assumes that all hues of color have thesame perceptual luminance which is not the case for human beings. Hence, anyform of metric distance such as Euclidean norm or cylindric distance will failto capture di↵erences in color as perceived by humans (Sarifuddin & Missaoui,2005)

Figure 5: The HSL color space. Source: By Jacob Rus, SharkD, derivativework, CC BY-SA 3.0, via Wikimedia Commons

2.5.4 HCL

HCL is a color space designed by Sarifuddin and Missaoui (2005) to more ac-curately represent color perceptually. The authors sought to create a moree↵ective color space for content-based image and video retrieval through imageprocessing and computer vision.

10

Figure 6: The HCV color space, similar in appearance to the HCL color space,(for an actual image of the HCL color space please see Sarifuddin & Missaoui,2005). Source: By Jacob Rus, SharkD, derivative work, CC BY-SA 3.0, viaWikimedia Commons

2.6 Color transfer functions

A transfer function is a function that maps di↵erent data values to an opticalproperty, such as color or opacity (Pfister et al., 2001). There are three typesof color scheme generated by transfer functions: qualitative, sequential anddiverging (Brewer, 1994).

A qualitative color scheme maps arbitrary values to arbitrary colors. However, aqualitative color scheme is only suitable for nominal or categorical data (Brewer,1994).

Figure 7: An illustrative example of a qualitative color scheme.

The transfer functions for sequential and diverging color schemes, on the otherhand, do not operate arbitrarily. Since color spaces are spanned by orthogonalbasis vectors, where a color is a linear combinations of the vectors, any paramet-ric curve that traverses the color space could be used to map an input parameter

11

value to an specific color output (Rheingans, 1992).

If the domain and range of the parametric color curve are specified so that valuesare logically arranged, from high to low, as well as light to dark, or vice versa,the result is a sequential color scheme (Brewer, 1994).

Figure 8: An illustrative example of a sequential color scheme.

On the other hand, if what is desired is to visualize deviation from a criticalvalue, then two light to dark transitions can be placed to either side of thecritical value. This is what is called a diverging color scheme (Brewer, 1994).

Figure 9: An illustrative example of a diverging color scheme.

2.7 Heatmaps

Heatmaps, also sometimes called intensity maps or point density interpolation,are visual structures for a spatial visualization of data (DeBoer, 2015). The termheatmap was first coined by Cormack Kiney in the 90s (Neovision Hypersystems,1998). Originaly developed as a tool for financial analysis, heatmaps have gainedbroad approval in biological and medical research (Akers, 2015). There evenexist prior examples of neural activity visualizations that encode a variable asa color spatially; such as positron emission tomography (PET) and functionalmagnetic resonance (fMRI)

At a fundamental level, heatmaps are implemented as spatial matrices withcells colored after their values (Huang, 2013). Specifically, heatmaps encode acontinuous quantitative variable as a color in space (Peterson, 2009) through acolor transfer function to a sequential color scheme (For an illustrative examplesee figure 10).

12

Figure 10: A heatmap using the grayscale sequential color scheme. High val-ues are mapped to lighter colors and low values to dark colors. The mappingfunctions domain is 0 to 9 and the range black to white.

2.8 Kernel image filtering

Convolution is a method in image processing that is used to apply character-istics, such as filtering or blurring. When processing an image, convolution isapplied to determine a value, like opacity or color, of the pixels in the image.The kernel matrix is a parameter of the convolution that defines the kind oftransformation to be applied on the individual pixels. In detail, the value ofeach pixel is determined by aggregating the values of the pixel and its neighborsmultiplied element-wise with the values in the kernel matrix (Ludwig, 2007).

Deconvolution is the reverse process to convolution. Unlike convolution, whereeach pixel is determined by the neighboring pixel values, deconvolution insteadtakes an individual pixel with given values and uses its value to determine thatof the neighboring pixels.

One of the applications of deconvolution is to recreate images without a prioriknowledge of the image being recreated. Recreation might be necessary if ei-ther the image is incomplete or has been distorted or modified by an unknownmethod. The latter is known as blind deconvolution (Ayers & Dainty, 1988)though the same method can also be applied for reconstruction of the former(Herrity, Raich, & Hero III, 2008).

13

3 Method

This section presents how to build the heatmap visualization tool in theory. Thistool graphically represents the data specified on subsection 3.1 Data structure.The visualization maps the neurons location to a 2-dimensional pixel grid, gen-erates frequency values for all pixels based on their distance to the data pointsand maps these values to color. Time is not mapped, so variation in frequencyvalues across time must be presented as animation. Additionally, suggestionsare given in regards of possible user-interaction.

3.1 Data structure

In development of the visualization program, we will use two datasets providedby a research group, lead by professor Tatiana Deliagina, from the Departmentof Neuroscience at the Karolinska Institute. The datasets contain neural datafrom a spinal cord section of decerebrated preparations as described in Zeleninet al. (2014). These datasets contain a neuron’s spatial position and a rangeof firing frequencies over time. The two datasets contain readings of 210 and107 neurons respectively, both with frequency readings for every 1 ms and atotal test time of 407 ms. Since the time intervals are equidistant, the data canbe classified as what is known as pure serial periodic data (Carlis & Konstan,1998).

A mathematical model of the data would be a four dimensional body withlaterality, depth, frequency and time as dimensions. Nevertheless, since this ispure serial periodic data, time is uninteresting to visualize and only the threeother dimensions are visualized. Furthermore, regular time intervals make itpossible to subdivide the body in time into equidistant three dimensional slicesof laterality, depth and frequency. It is these slices that are utilized as framesin the heatmap visualization, with laterality being mapped to width, depth toheight and frequency to color.

However, the input data files are in comma separated values (CSV) format. CSVfiles store data in plain text in a tabular format, where every comma delimitatesa cell and every breakline a row. Due to this limitation, the four dimensionaldata is encoded as a two dimensional matrix where each row contains the data ofan individual neuron. The rows consist of laterality as well as depth(mm), andadditional frequency fields, one for every millisecond of the recording. Frequencyfields are arranged sequentially in chronological order and contains the firingfrequency(kHz).

14

x y fi,0 · · · fi,406x0 y0 f0,0 · · · f0,406x1 y1 f1,0 · · · f1,406...

......

. . ....

xn�1 yn�1 fn�1,0 · · · fn�1,406

Table 2: CSV data file structure. xi is the laterality of data point i in millime-ters, yi is the depth of data point i in millimeters, fi,t is the firing frequency ofdata point i at the millisecond t of the test, where 0 i < n and 0 t 406

3.2 Visual mappings

3.2.1 Mapping depth and laterality to on screen coordinates

In the visualization, depth and laterality are converted to pixel coordiates (u, v)by linear scaling. More specifically, by multiplying them by an arbitrary pixelscaling factor proportional to window height. Hence:

x = laterality · pixel factory = depth · pixel factor

(3)

Since depth and laterality are given as continuous millimeter coordinates and apixel grid is discrete, these values would normally require some kind of integerapproximations. Nevertheless, since neurons are never directly displayed onscreen but only used to calculate pixel frequency values, then no discretization isrequired (for more detail on how pixels frequencies are calculated see subsection3.3 Heatmap calculation).

3.2.2 Mapping frequency to color

A color transfer function c(f), where f is the input frequency value, is to bedevised. In accordance to what was said in subsection 2.6, there are two thingsto determine: the color space and the parametric curve. Hence, all possibletransfer functions should be implementable in the following manner:

c(f) = g1(f)~e1 + · · ·+ gN (f)~eN (4)

Where ~gi(f) are arbitrary functions of frequency, ~ei are the basis vectors of thecolor space and N is the dimensionality of the color space.

15

3.3 Heatmap calculation

The heatmap is generated by calculating frequency for each pixel at coordi-nates (u, v), by accumulating a normally distributed frequency that decays overdistance from every data point.

The frequency distribution of an individual data point over the entire image canbe modeled after the bivariate Gaussian point spread function(PSF):

I(u, v, i) = I0e� (x

i

�u)2+(yi

�v)2

2�2 where I0 = I(xi, yi) = fi (5)

However, plain aggregation of the frequency distributions would result in ad-ditive artifacts such as those described by DeBoer (2015)(see figure 11 andcompare to figure 12), where overlaps result in values of higher frequency thanthe original data points. This is because the above model does not account foroverlap or compensate for it. This is demonstrably erroneous since, by thatlogic, two identical measurements of the same point would result in twice asmuch frequency.

Figure 11: Examples of heatmap behavior without normalization. The crossesrepresent the positions of the data points. A. are two data points at d = 2r B.are two data points at d = r and C. is a single data point measured twice.

Figure 12: Examples of heatmap behavior with normalization. The crossesrepresent the positions of the data points. A. are two data points at d = 2r B.are two data points at d = r and C. is a single data point measured twice.

The visualization tool could handle overlap in a similar manner to Zelenin et al.(2014) by normalizing with the sum of weights (see eq.1). However, the weights

16

are calculated in accordance to the model described in eq.5 in the followingmanner (compare to eq.2):

w(u, v, i) = e�(x

i

�v)2+(yi

�u)2

2�2 (6)

The equation for calculation pixel frequency values hence becomes:

f(u, v) =

nPi=0

w(u, v, i)fi

nPi=0

w(u, v, i)(7)

Where (u, v) are pixel coordinates, for other variables see table 2.

Afterwards, the final step necessary is to convert pixel frequency values intocolor through a transfer function as described in 3.2.2.

3.4 Animation

Changes in frequency values over time should be presented as an animation.An animation loop is needed to create an animation. The loop continuouslyreplaces the image on screen. To replace the image two procedures are required:an update call and a draw call, executed in that order. The update call changesthe contents of the image and the draw call displays the image on the screen.

3.5 Interaction

Rheingans (1992) proposed that interactivity and animation for exploration ofquantitative multivariate data has a positive e↵ect in terms of accuracy anduser confidence in information gathering. The features employed by Rheingans(1992) will be included. In regards to animation, these consist of the abilityto play and pause the animation, step an arbitrary number of frames and cus-tomizable frame rate. In addition, the user will be able to jump to arbitrarypoints in time by either writing the frame number or using a slider.

While Rheingans (1992) focused on color mappings and animation for visualcolor-based-systems, the visualization tool will o↵er wider user interaction. Otherinteractive features include loading own test data, customizable visualizationquality, customizable axes and choice of color scheme.

17

4 Results

Anivis was the name given to the software implementation of the visualizationtool described in section 3 Method. Anivis successfully implements an interac-tive animated heatmap visualization of neural frequency data (figure 13 left).In addition to the heatmap, Anivis also visualizes the discrete frequency valuesof individual neurons for comparison (figure 13 right).

In this section, the implementation methods of Anivis will be presented in detail.For the sake of comparison, all figures of the visualizations are of the same frameand dataset.

Figure 13: Visualizations in Anivis. Neural frequency data is presented asanimated interactive visualizations. A heatmap to the left and a scatter-plot tothe right.

4.1 Tools and design choices

Anivis is intended to be used by a research group working on computers with aWindows operating system (version not specified). As a majority of operatingsystems (including Windows, OSX and Linux) include a web browser, Anivis

18

was developed as a web application. This format ensures portability and enablesexecution without prior software installation on the system.

Anivis is written in Javascript, a programming language supported in all mod-ern web-browsers (MozillaDN, 2016). Furthermore, the Javascript environmentenables the usage of third party libraries. One such library is D3.js (DataDriven Documents), a Javascript library for document manipulation based ondata (Bostock, Ogievetsky, & Heer, 2011). Anivis uses D3.js to parse the datafiles, manipulate the DOM structure of the web application and create databindings between DOM elements and data values.

4.2 Scatter-plot implementation

One of the main disadvantages of a heatmap visualization is that it is impossibleto determine the exact origin of high value areas. High value areas may originatein either individual data points with extreme values or synchronized activity ofseveral neighboring data points (see figure 14).

Figure 14: A. Heatmap B. Scatter-plot; C. Heatmap with a superposed scatter-plot. All three images display the same dataset. Please notice that even toughthe left and right blob look similar in the heatmap, the scatterplot revealsthat the left blob originates in synchronized activity and the right one due theextreme value of the central data point.

In order to visualize the causal relationship between the data points and theheatmap, Anivis also creates a scatter-plot of the data points for compari-son(figure 13 right). The scatter-plot is of the same scale as the heatmap.So, in order to place the data points on the screen only the corresponding pixelcoordinates are necessary, these are obtained by following the steps on 3.2.1and then rounding to the nearest integer value. The scatter-plot even uses thesame color transfer function with the same domain and range as the heatmap,which turns out to be suitable since the heatmap is normalized, but which isalso desirable in order to facilitate comparison.

19

4.3 Heatmap implementation

Two main algorithmic approaches were implemented to calculate the heatmapin the manner described in subsection 3.3. These two approaches were weightedsum (the naive approach) and deconvolution.

Although both approaches aim to visualize the data as the same graphical image,they do so through di↵erent methods. In the weighted sum approach, eachpixel’s frequency value is increased for every data point based on its distanceto the data point and its frequency. The other approach is deconvolution. Incontrast with convolution, where the color values of neighboring pixels are usedto determine the value of each pixel; in deconvolution, the value of individualdata-points are used to determine the color of adjacent pixels.

4.3.1 Weighted sum

The weighted sum is carried out as described by eq. 7 in subsection 3.3, byemploying three di↵erent nested loops. Nevertheless, this approach has anO(u

max

vmax

n) time complexity (where umax

is the visualizations width in pixelsand v

max

its height) and might be too slow for general usage.

However, this algorithm can be made practically employable through memo-ization. Since all data points remains at fixed positions through time, and theweights values are dependent only on their distance to the data point, then allweights need only be calculated once for the entire animation. Furthermore,even the weight sums can be pre-calculated. This e↵ectively eliminates all op-eration inside the divisor sums, and replaces them for look-ups. In summary,while this optimization does not reduce the time complexity of any operationsit reduces the number of necessary operations inside the triple loop to a singleone: multiplication between the weight and the frequency of the data point.

An additional heuristic, that synergizes with memoization, is to not performany calculations if the weights are small enough (wi < "). Since the value ofthe weights decreases exponentially, large portions of the image should be nearzero (or zero due to float precision) and can be safely ignored. The reason theseregions can be ignored without any loss in quality is that color spaces have alimited resolution.

The exact size of "max

so that no quality degradation occurs cannot be univer-sally determined. This is because it depends entirely on the resolution of thecolor space, but is also a↵ected by the parametric color curve. Since how close" is to "

max

has only a marginal e↵ect in performance, a value of " = 0.001 wasused as a heuristic.

20

4.3.2 Deconvolution

A more sophisticated approach than weighted sum with memoization, is tocalculate the heatmap by taking advantage of dynamic programming instead ofmemoization. Since only the pixels close enough to the data points will havetheir frequency values significantly increased, and all distant values will be nearzero or actually zero due to limitations in resolution, there is no need to traversethe entire canvas adding value for every data point.

Instead, a m⇥m Gaussian kernel K is calculated as follows:

Km,m =

0

BBB@

k0,0 k1,0 · · · km�1,0

k0,1 k1,1 · · · km�1,1...

.... . .

...k0,m�1 k1,m�1 · · · km�1,m�1

1

CCCAwhere k(i, j) = e�

(i�m

2)2+(j�m

2)2

2�2

(8)

Afterwards only the aggregate of individual contributions of each data point toneighboring pixels needs to be calculated as follows:

1 f o r ( datapo int in data ) {2 // For every number in the matrix3 f o r ( var j = 0 ; j < m; j++) {4 // Ca l cu la t e the p ixe l ’ s y coo rd inate5 y = datapo int . y � f l o o r ( s ideLength /2) + j ;6 f o r ( var i = 0 ; i < m; i++) {7 // Ca l cu la t e the p ixe l ’ s u coord ina te8 u = datapoint . x � f l o o r ( s ideLength /2) + i ;9 // I f that p i x e l i s ou t s i d e the image do nothing

10 i f (u<0 | | v<0 | | u >= u max | | v >= v max ) cont inue ;11

12 // Otherwise add to that p i x e l s va lue13 back bu f f e r [ v ] [ u ] += datapoint . f [ time ] ⇤ K[ j ] [ i ] ;14 }15 }16 }

After this process is complete, it is necessary to normalize all the values. Themain challenge of normalizing the values is generating the sum of all the kernelvalues at intersections between matrices. However, this sum can be achievedthrough deconvolution by modifying only one line of code in the above algorithm:

1 // Otherwise add to that p i x e l s va lue2 weight sum [ v ] [ u ] += K[ j ] [ i ] ;

21

Afterwards, all that is needed to normalize the back bu↵er is to divide its fre-quency values with the sum of all kernel weights at that pixel.

The reason calculation of kernel weights was not included in the pseudocode fordeconvolution is that deconvolution benefits from memoization as well. Sincethe data points do not move, the sum of kernel weights need only be calculatedonce for the entire animation. Unlike calculation of all weights in the weightedsum approach, which is slow in comparison to frame rendering in spite of havingthe same time complexity, calculation of the kernel weight sum is equivalent tocalculating an additional extra frame when the animation is started.

Deconvolution has a time complexity ofO(m2n) since the calculation of pixel fre-quency values dominates over normalization of the values which is only O(m2).This o↵ers much better performance in comparison to weighted sum (O(u

max

vmax

n))as m is likely to be smaller than the visualizations width (u

max

) and height(v

max

). In fact, a kernel of dimensions umax

⇥ vmax

would be functionally equalto weighted sum.

While a smaller kernel improves performance, a kernel of insu�cient size willlead to block artifacts in the heatmap visualization as the weights at the matrixedges will not be near zero (see figure 15).

Figure 15: Block artifacts created by deconvolution with a kernel of insu�cientsize (50x50).

22

On the other hand deconvolution with a kernel of su�cient size is functionallyidentical to the weighted sum approach (see figure 16 and (Duchowski et al.,2012)).

Figure 16: A. Heatmap produced by using weighted sum B. Di↵erence imagebetween A and C; C. Heatmap produced by deconvolution with a large kernelstill smaller than the image size.

4.4 Color transfer functions

Four di↵erent transfer functions were implemented, grayscale, HSL Rainbow,Cubehelix and HCL Heat. Grayscale and HSL Rainbow are our own implemen-tation of common color schemes; Cubehelix and HCL Heat are custom transferfunctions and color schemes. HCL Heat we have designed ourselves and Cube-helix was designed by Green (2011).

Before going into individual transfer functions, it should be mentioned that notonly are users able to choose the preferred transfer function but also customize itby selecting the input domain bounds. Any values outside the domain boundsare clamped to the closest domain bound. Where the floor is f

o↵set

and theceiling f

ceiling

. These values will be treated as numerical constants althoughin practice they function as parameters. By default when Anivis is bootedfo↵set

= 0 and fceiling

= fmax

.

4.4.1 Grayscale

Grayscale is a straight line from (0, 0, 0) to (1, 1, 1) in the RGB color space.Hence, the parametric color curve function is:

23

c(f) =

2

4RedGreenBlue

3

5 =

2

64

ffceiling

+ fo↵set

ffceiling

+ fo↵set

ffceiling

+ fo↵set

3

75 (9)

While not optimal for visualization of data due to the reasons stated in 2.5.1,there are uses for the grayscale transfer function such as monochrome printing.

Figure 17: Anivis heatmap through the grayscale transfer function.

4.4.2 HSL Rainbow

The HSL Rainbow was implemented as a circle arc around the cylindrical HSLcolor space (see fig. 5).

The HSL Rainbow arc has a height of lightness

max

2 , a radius of saturationmax

anda range of [0, 3⇡

4 ]. Hence, the parametric color curve function is as follows:

24

c(f) =

2

4Hue

SaturationLightness

3

5 =

2

43⇡4 � ( f

fceiling

+ fo↵set

) 3⇡410.5

3

5 (10)

While not optimal for visualization of data due to the reasons stated in 2.5.3,this transfer function was implemented in Anivis by request of the researchgroup at the Deparment of Neuroscience.

Figure 18: Anivis heatmap through the HSL Rainbow transfer function.

4.4.3 Cubehelix

The transfer function of the Cubehelix color scheme is implemented through thecubehelix.js library; a plug-in library for D3.js that implements Cubehelix.

What distinguishes the Cubehelix color scheme from HSL rainbow is that theCubehelix color scheme is designed to account for perceptual di↵erences ofbrightness between the colors. Hence, higher values are assigned brighter colorsand lower values darker colors.

25

Figure 19: Anivis heatmap through the Cubehelix transfer function.

4.4.4 HCL Heat

HCL Heat is an ascending arc in the HCL color space. This transfer functionwas specifically created for Anivis and is meant to be a compromise betweenthe HSL Rainbow and Cubehelix.

c(f) =

2

4Hue

ChromaLuminance

3

5 =

2

64

13⇡9 � ( f

fceiling

+ fo↵set

) 11⇡91

0.05 + 0.8( ffceiling

+ fo↵set

)

3

75 (11)

Not only are low values perceptually darker, but also cooler colors, in the samemanner that high values both brighter and warmer. Cool colors are those whichare perceived to be receding or ground, while warm colors are salient instead(Oxford Reference, n.d.).

26

Figure 20: Anivis heatmap through the HCL Heat transfer function.

4.5 Rendering

Anivis uses two bu↵ers for calculating the visualization each frame, a backbu↵er and a display bu↵er. The back bu↵er is used to store calculated pixelfrequency values while the display bu↵er stores pixel color values. Both have amemory complexity of ✓(u

max

vmax

), as the amount of pixels grows quadraticallyin relationship to canvas size, but canvas width and height need not be equal.

The back bu↵er is updated through the update call. The update call calculateseach pixels frequency value as described in subsection 4.3 and stores them in aback bu↵er. Each frame is calculated on request and when calculating a newframe, the back bu↵er is overwritten. The display bu↵er is updated through thedraw call. A draw call converts every frequency value in the back bu↵er to colorand then stores them in the display bu↵er. When draw terminates, the displaybu↵er contains the computed image data employed by the HTML5 canvas todisplay the graphics on screen.

27

4.6 Animation

4.6.1 Animation loop

Anivis represents changes in firing frequency over time as animation by updatingthe image data in an HTML5 canvas. To create the animation, static imagesare displayed in quick succession in the canvas through the processes describedin subsection 4.5 rendering; we will refer to these individual static images asframes.

Frames are drawn on a request-basis as to enable the user to freely move toarbitrary points in time during and outside any playing animation. To draw aframe, the frequencies for each pixels in a frame must be calculated and thenstored in the back bu↵er through an update call and then converted to colorand sent to a display bu↵er through the draw call.

Normally animation could be implemented by a simple animation loop (as de-scribed in subsection 3.4) or by setting up a callback function to execute atregular intervals of time. However, both approaches are flawed, since web pagescripts are executed sequentially and in the same thread as the user interface(Caballero, 2013; Resig, 2008). The first approach would permanently lock theuser interface, as the scheduler waits for the infinite animation loop to finish.The second approach, implemented through setInterval(), was functionalenough to be used in the first working versions of Anivis, even though it is nottechnically correct.

The reason setInterval() is not correct in regards the amount of frames persecond (FPS), is that the frame rate will not match the specified value. Thisis because setInterval() does not implement the intervals by executing thecallback on a separate thread but through scheduling in the current thread, andhence, if blocked from immediate execution, the callback will be postponed untilthe next possible execution point (Resig, 2008).

Anivis implements the animation loop through requestAnimationFrame(); whichtakes a callback as argument and sets it up for execution just before repaintingthe contents of the window (i.e. the browser’s draw call). However, requestAnimationFrame()only sets up the callback for the next repaint. Hence, the animation-functionmust repeatedly re-schedule itself to be executed before each frame.

4.6.2 Frame rate capping

Since Anivis implements the animation loop through a callback-by-request ba-sis. Inside the callback itself there is a timer that compares the time when

28

previous frame was requested with the designated frame rate. If enough timehas passed, the animation function will request for a new frame to be pro-cessed and animated and save what time the new frame was requested. Ifthe designated time has not elapsed, the function will do nothing. Regardless;each call of the animation function will push it self to be called back with therequestAnimationFrame().

4.7 Interactivity

Anivis is an interactive tools that allows the user to control the visualization aswell as the which data to visualize. Anivis is able to visualize any data of thein format described in subsection 3.1 and can change which data to visualizedynamically. The animation can be controlled be either a play-pause feature orby directly selecting the desired frame by number. The frame rate can also bechanged dynamically. The the color scale can also be changed dynamically forboth the heatmap and the scatter-plot.

29

5 Discussion

This section will provide an critical analysis of the design and implementationspresented in section 4 Results.

5.1 Heatmaps in Anivis

While this project is partially based upon the heatmap produced by Zelenin et al.(2014), there are fundamental di↵erences between the heatmaps visualizationsin that article and those produced by Anivis.

Format wise, Zelenin et al. (2014) groups frequency values into bins and averagesin time through sliding average. However, Anivis needs not aggregate lapses oftime as it can display changes in frequency as animation.

Another di↵erence in terms of calculations is that Anivis employs a di↵erentkind of Gaussian weighting function(compare eq.2 to 6) modeled after the Gaus-sian PSF (point spread function). Nevertheless, both weighting functions stillproduce indistinguishable heatmap images (see figure 21).

Figure 21: A. Heatmap produced by Anivis B. Di↵erence image between Aand C; C. Heatmap from a modified version of Anivis that employs the sameGaussian weight formula as Zelenin.

30

5.2 Image quality of deconvolution

It was shown in 4.3.2 that a large kernel will produce identical results to theweighted sum approach(fig. 16) and a small kernel results in visual artifacts(fig.15). However, what was not mentioned is what happens for kernel sizesin between. E↵ectively, there is an interesting middle-ground where no visualartifacts occur but the image still does not entirely match the weighted sumresult (fig. 22).

Figure 22: A. Heatmap produced by using weighted sum B. Di↵erence imagebetween A and C; C. Heatmap produced by deconvolution with a kernel smallerthan the image size (150x150).

This middle ground is, at least perceptually, indistinguishable from larger kernelsor the weighted sum approach. This is significant because any reduction the sizeof the kernel will significantly increase performance in terms of frame rate, asthe number of calculations is directly proportional to the product of the amountof data points; the kernel’s height and width; but not the image size.

Which means that while it might be possible to calculate the minimum kernelsize that still produces images identical to weighted sum, that kernel size mightstill not be optimal. This is because the only purpose of the heatmap is tovisualize data and both images are arguably equally as e�cient to that end.For that reason we suggest that if an optimal kernel size is to be found it shouldbe through perceptual studies rather than mathematical calculations.

Ironically, however, a fixed optimal kernel size might not be optimal in itself.This is because while a users seeking to extract individual images from the

31

application might require image fidelity, users interested in creating videos orfluid interaction might rather prioritize frame-rate over image quality, even ifthere are perceptible di↵erences in image quality. Instead we argue that theuser should be able to interactively be able choose the quality level accordingto their needs.

5.3 Scatter-plot as a complement to heatmap visualization

As mentioned in subsection 4.2, Anivis implements a scatter-plot as a way tocomplement the heatmap. While heatmaps may be useful for identificationof clustering in the data, heatmaps have e↵ectively no way of distinguishingbetween data points with extreme values and clusters in the data, as shown in4.2. This raises the question whether or not the heatmap itself is su�cient asvisual structure for the purpose of this kind of data-analysis.

The scatter-plot was originally implemented in Anivis in order to compare thedata point to the heatmap. As development continued, it became apparent thatthe scatter-plot was necessary in order to make the images produced by Anivisunambiguous; both for the intended end-user and for the purpose of this report.

However, having both visualizations side by size may make di�cult to pin-pointspecific locations in both the heatmap and the scatterplot. A possible solutionis to superpose the scatter-plot as an overlay to the heatmap (as was done infigure 14 C) so that any regions can accurately be related to the data-points inthe canvas.

5.4 Future research

Anivis shows it is possible to implement a heatmap through CPU calculations.However, this might not be the best approach since a GPU (Graphical Process-ing Unit) approach would likely outperform the current CPU implementation.This is because the calculations are performed pixel-wise and hence can be eas-ily parallelized, and while current processors have multiple cores in the singledigits, GPUs may contain up to thousands of cores (What is GPU Computing?,n.d.).

32

6 Conclusion

Anivis is an successful implementation of animated heatmaps of spatially dis-tributed neural frequency data. A heatmap is a continuous representation ofdata whereas the data gathered consists of discrete measurements. Therefore,the processing of the data consists of interpolating and extrapolating frequencyvalues for every pixel in the visualization. Once calculated these values shouldadditionally be converted to color.

During development it was shown that deconvolution was the most e�cientapproach for spatial interpolation of frequency values, both theoretically andpractically. Benchmarking was deemed unnecessary since both methods had alarge di↵erences in execution time.

The implementation of the scatter-plot was an appropriate design choice as itaddresses one of the main weaknesses of the visual structure of heatmaps. Sincethere no way to di↵erentiate clustering from extreme values, we would like torecommend other heatmaps with weighted data points to superimpose a scatterplot.

In terms of future research, it would probably be of interest to parallelize thesuggested approaches both weighted sum and deconvolution as a GPU imple-mentation and to determine an optimal kernel size through perceptual studies.

33

References

Akers, W. (2015). Visual resource monitoring for complex multi-project environ-ments. International Journal of System of Systems Engineering , 6 (1/2),112. Retrieved from http://dx.doi.org/10.1504/IJSSE.2015.068814

doi: 10.1504/ijsse.2015.068814Anscombe, F. J. (1973, feb). Graphs in statistical analysis. The American

Statistician, 27 (1), 17. Retrieved from http://dx.doi.org/10.2307/

2682899 doi: 10.2307/2682899Ayers, G. R., & Dainty, J. C. (1988, jul). Iterative blind deconvolution method

and its applications. Optics Letters, 13 (7), 547. Retrieved from http://

dx.doi.org/10.1364/OL.13.000547 doi: 10.1364/ol.13.000547Bostock, M., Ogievetsky, V., & Heer, J. (2011). D3: Data-Driven Documents.

IEEE Trans. Visualization & Comp. Graphics (Proc. InfoVis). Retrievedfrom http://vis.stanford.edu/papers/d3

Brewer, C. A. (1994, may). Guidelines for use of the perceptual dimensionsof color for mapping and visualization. In J. Bares (Ed.), Color hardcopy and graphic arts III. SPIE-Intl Soc Optical Eng. Retrieved fromhttp://dx.doi.org/10.1117/12.175328 doi: 10.1117/12.175328

Caballero, L. (2013). Dev.Opera — Better Performance With requestAnimation-Frame. Retrieved 2016-4-17, from https://dev.opera.com/articles/

better-performance-with-requestanimationframe/

Card, S. K., Mackinlay, J. D., & Shneiderman, B. (1999). Readings in informa-tion visualization: using vision to think. Morgan Kaufmann.

Carlis, J. V., & Konstan, J. A. (1998). Interactive visualization of serial periodicdata. In Proceedings of the 11th annual ACM symposium on user interfacesoftware and technology - UIST ’98. Association for Computing Machinery(ACM). Retrieved from http://dx.doi.org/10.1145/288392.288399

doi: 10.1145/288392.288399DeBoer, M. (2015). Understanding the Heat Map. Carto-

graphic Perspectives(80), 39–43. Retrieved from http://

www.cartographicperspectives.org/index.php/journal/article/

view/cp80-deboer/1420 doi: 10.14714/CP80.1314Duchowski, A. T., Price, M. M., Meyer, M., & Orero, P. (2012). Aggregate gaze

visualization with real-time heatmaps. In Proceedings of the symposiumon eye tracking research and applications - ETRA ’12. Association forComputing Machinery (ACM). Retrieved from http://dx.doi.org/10

.1145/2168556.2168558 doi: 10.1145/2168556.2168558Fetcho, J., & O’Malley, D. M. (1995). Visualization of active neural circuitry

in the spinal cord of intact zebrafish. Journal of neurophysiology , 73 (1),399–406.

Goldstein, E. (2014). Sensation and perception. Belmont, CA: Wadsworth,Cengage Learning.

Green, D. (2011). A colour scheme for the display of astronomical intensityimages. arXiv preprint arXiv:1108.5083 . Retrieved from http://astron

-soc.in/bulletin/11June/289392011.pdf

34

Handbook of laboratory and diagnostic tests. (2013). Farlex and Partners.Retrieved from http://medical-dictionary.thefreedictionary.com/

PET+scan+of+the+brain

Herrity, K., Raich, R., & Hero III, A. O. (2008, feb). Blind reconstruction ofsparse images with unknown point spread function . In C. A. Bouman,E. L. Miller, & I. Pollak (Eds.), Computational imaging VI. SPIE-Intl SocOptical Eng. Retrieved from http://dx.doi.org/10.1117/12.779253

doi: 10.1117/12.779253Honeycutt, C. F., Gottschall, J. S., & Nichols, T. R. (2009, mar). Electromyo-

graphic Responses From the Hindlimb Muscles of the Decerebrate Catto Horizontal Support Surface Perturbations. Journal of Neurophysiol-ogy , 101 (6), 2751–2761. Retrieved from http://dx.doi.org/10.1152/

jn.91040.2008 doi: 10.1152/jn.91040.2008Hsu, L.-J., Zelenin, P. V., Orlovsky, G. N., & Deliagina, T. G. (2012, apr).

E↵ects of galvanic vestibular stimulation on postural limb reflexes andneurons of spinal postural network. Journal of Neurophysiology , 108 (1),300–313. Retrieved from http://dx.doi.org/10.1152/jn.00041.2012

doi: 10.1152/jn.00041.2012Huang, M. L. (2013). Innovative Approaches of Data Visualization and Visual

Analytics. IGI global.Kindlmann, G. L., Weinstein, D. M., Jones, G. M., Johnson, C. R., Capecchi,

M. R., & Keller, C. (2005). Practical vessel imaging by computed to-mography in live transgenic mouse models for human tumors. Molecularimaging , 4 (4), 417.

Kreit, E., Mathger, L. M., Hanlon, R. T., Dennis, P. B., Naik, R. R., Forsythe,E., & Heikenfeld, J. (2012, sep). Biological versus electronic adaptivecoloration: how can one inform the other? Journal of The Royal SocietyInterface, 10 (78), 20120601–20120601. Retrieved from http://dx.doi

.org/10.1098/rsif.2012.0601 doi: 10.1098/rsif.2012.0601Leong, J. (2006). Number of Colors Distinguishable by the Human Eye.

Retrieved 2016-04-18, from http://hypertextbook.com/facts/2006/

JenniferLeong.shtml

Ludwig, J. (2007). Image Convolution. Retrieved from \url{http://

web.pdx.edu/~jduh/courses/Archive/geog481w07/Students/

Ludwig ImageConvolution.pdf}

MozillaDN. (2016, apr 1). JavaScript. https://developer.mozilla.org/

en-US/docs/Web/JavaScript. (Accessed: 2016-04-16)Muto, A., Ohkura, M., Abe, G., Nakai, J., & Kawakami, K. (2013, feb).

Real-Time Visualization of Neuronal Activity during Perception. CurrentBiology , 23 (4), 307–311. Retrieved from http://dx.doi.org/10.1016/

j.cub.2012.12.040 doi: 10.1016/j.cub.2012.12.040Neovision Hypersystems, I. (1998, 03 3). Heatmaps (Trademark No. US

75263259). Retrieved from http://tsdr.uspto.gov/documentviewer

?caseId=sn75263259&docId=ORC20060126060905#docIndex=4&page=1

Oxford Reference. (n.d.). colour temperature. Retrieved from //www

.oxfordreference.com/10.1093/oi/authority.20110803095625612

35

Peterson, G. (2009). GIS Cartography. Informa UK Limited. Retrievedfrom http://dx.doi.org/10.1201/9781420082142 doi: 10.1201/9781420082142

Pfister, H., Lorensen, B., Bajaj, C., Kindlmann, G., Schroeder, W., Avila, L., . . .Lee, J. (2001). The transfer function bake-o↵. IEEE Comput. Grap. Appl.,21 (1), 16–22. Retrieved from http://dx.doi.org/10.1109/38.920623

doi: 10.1109/38.920623PhotonStarTecnology Ltd. (2016). ”how leds produce white light”.

Retrieved 2016-5-11, from http://www.photonstartechnology.com/

learn/how leds produce white light

Resig, J. (2008). How JavaScript Timers Work. Retrieved 2016-4-17, fromhttp://ejohn.org/blog/how-javascript-timers-work/

RGB color model. (2008). Retrieved 2016-05-08, from http://

www.oxfordreference.com/view/10.1093/oi/authority

.20110803100418271

Rheingans, P. (1992). Color change, and control of quantitative data display.In Visualization, 1992. visualization’92, proceedings., ieee conference on(pp. 252–259). Institute of Electrical & Electronics Engineers (IEEE).Retrieved from http://dx.doi.org/10.1109/visual.1992.235201 doi:10.1109/visual.1992.235201

Sarifuddin, M., & Missaoui, R. (2005). A new perceptually uniform color spacewith associated color similarity measure for content-based image and videoretrieval. In Proc. of acm sigir 2005 workshop on multimedia informationretrieval (mmir 2005) (pp. 1–8).

Spittler, J. F., Wortmann, D., von During, M., & Gehlen, W. (2000, jun).Phenomenological diversity of spinal reflexes in brain death. Eur JNeurol , 7 (3), 315–321. Retrieved from http://dx.doi.org/10.1046/

j.1468-1331.2000.00062.x doi: 10.1046/j.1468-1331.2000.00062.xTory, M., & Moller, T. (2004, jan). Human factors in visualization research.

IEEE Trans. Visual. Comput. Graphics , 10 (1), 72–84. Retrieved fromhttp://dx.doi.org/10.1109/tvcg.2004.1260759 doi: 10.1109/tvcg.2004.1260759

What is GPU Computing? | High-Performance Computing | NVIDIA|NVIDIA.(n.d.). Retrieved 2016-05-10, from http://www.nvidia.com/object/

what-is-gpu-computing.html

Yau, N. (2014). FlowingData. com Data Visualization Set. John Wiley & Sons.Zeileis, A., Hornik, K., & Murrell, P. (2009). Escaping rgbland: selecting colors

for statistical graphics. Computational Statistics & Data Analysis, 53 (9),3259–3270.

Zelenin, P. V., Hsu, L.-J., Lyalka, V. F., Orlovsky, G. N., & Deliagina, T. G.(2014, nov). Putative spinal interneurons mediating postural limb reflexesprovide a basis for postural control in di↵erent planes. European Journalof Neuroscience, 41 (2), 168–181. Retrieved from http://dx.doi.org/

10.1111/ejn.12780 doi: 10.1111/ejn.12780

36