DESIGN OF A WEB GRAPHIC USER INTERFACE FOR REAL-TIME EEG INTERPRETATION

A Degree Thesis

Submitted to the Faculty of

Escola Tècnica d'Enginyeria de Telecomunicació de

Barcelona

Universitat Politècnica de Catalunya

by

Víctor Peña Romero

In partial fulfilment

of the requirements for the degree in

TELECOMMUNICATIONS TECHNOLOGIES AND

SERVICES ENGINEERING

Advisor: Emanuel Popovici

Co-advisor: Climent Nadeu

Barcelona, February 2020

1

Abstract

This project proposes a method for easier interpretation of Electroencephalogram (EEG)

data acquired from newborn babies so that it can be understood by a wider range of

medical professionals. The equipment currently used requires highly qualified experts in

order to detect possible seizures in newborns.

The project is divided into two blocks. The main block is the implementation of a system

that enables the communication between the device recording the EEG data from the baby

and a server. The second block consists of the design of a front-end user web interface

that plots a graph of the EEG data.

The communication part is based on an Arduino board. The board processes the acquired

EEG data and sends it via a Wi-Fi connection to a server creating the backbone of the

system. When the information is received, it is sent to the client side for real-time plotting.

2

Resum

Aquest projecte proposa una tècnica per facilitar la interpretació de les dades de

electroencefalograma (EEG) adquirides de nounats perquè puguin ser interpretades per

un grup de professionals mèdics més ampli. L’equip utilitzat avui dia requereix experts

altament qualificats per a detectar possibles convulsions en nadons.

El projecte es divideix en dos blocs. El principal és la implementació del sistema que

permet que el protocol de comunicació enviï les dades EEG adquirides del nadó a un

servidor. El segon bloc consisteix en el disseny de la interfície web d'usuari que traça el

gràfic amb les dades EEG.

La part de comunicació es basa en una placa Arduino. La placa processa les dades EEG

ja adquirides i les envia a través d'una connexió Wi-Fi a un servidor creant la base del

nostre sistema. Quan la informació és rebuda, s'envia a la part del client per a la

representació del gràfic en temps real.

3

Resumen

Este proyecto propone un método para facilitar la interpretación de los datos de

electroencefalogramas (EEG) adquiridos de recién nacidos, para que puedan ser

interpretados por un grupo de profesionales médicos más amplio. El equipo utilizado hoy

en día requiere expertos altamente calificados para detectar posibles convulsiones en

bebés.

El proyecto se divide en dos bloques. El principal, es la implementación del sistema que

permite que el protocolo de comunicación envíe los datos EEG adquiridos del bebé a un

servidor. El segundo bloque consiste en el diseño de la interfaz web de usuario que traza

el gráfico con los datos EEG.

La parte de comunicación se basa en una placa Arduino. La placa procesa los datos EEG

ya adquiridos y los envía a través de una conexión Wi-Fi a un servidor creando la base del

sistema. Cuando la información es recibida, se envía a la parte del cliente para la

representación del gráfico en tiempo real.

4

Acknowledgements

I would like to thank Dr. Andriy Temko and Dr. Climent Nadeu for giving me the opportunity

to work in INFANT Research Center and EmbeddedSystems@UCC, University College

Cork. I would also like to thank Dr. Emanuel Popovici for his supervision during the project.

I wish to express my most sincere thanks to Sergi Gomez, PhD candidate of the Electrical

and Electronic Engineering Department in UCC for the unconditional help and support

throughout the project and my stay in Ireland. I would also like to thank Mark O’Sullivan for

his patience and help with the project.

Last but not least, I would also like to thank all my family, especially Raúl Escalante for all

their support during my stay abroad.

5

Revision history and approval record

Revision Date Purpose

0 29/11/2019 Document creation

1 20/01/2020 Document revision

DOCUMENT DISTRIBUTION LIST

Name e-mail

Victor Peña Romero victorpromero@gmail.com

Emanuel Popovici e.popovici@ucc.ie

Climent Nadeu Camprubí Climent.nadeu@upc.edu

Written by: Reviewed and approved by:

Date 29/11/2019 Date 20/01/2020

Name Victor Peña Romero Name Emanuel Popovici

Position Project Author Position Project Supervisor

6

Table of contents

Abstract ............................................................................................................................................................. 1

Resum ............................................................................................................................................................... 2

Resumen ........................................................................................................................................................... 3

Acknowledgements ........................................................................................................................................... 4

Revision history and approval record ................................................................................................................ 5

Table of contents ............................................................................................................................................... 6

List of Tables: .................................................................................................................................................... 8

1. Introduction ............................................................................................................................................... 9

1.1. Project Background ......................................................................................................................... 9

1.2. Project overview and goals ............................................................................................................ 10

1.3. Project requirements and specifications ........................................................................................ 11

1.4. Work Plan (Tasks and Gantt diagram) ........................................................................................... 11

1.5. Incidences and deviations ............................................................................................................. 13

2. State of the art of the technology used or applied in this thesis: ............................................................. 14

2.1. Background ................................................................................................................................... 14

2.2. Data Acquire – ESP Wi-Fi Board ................................................................................................... 15

2.3. Data transmission .......................................................................................................................... 16

2.3.1 UDP ............................................................................................................................................... 16

2.3.2 TCP ............................................................................................................................................... 16

2.4. Real-time communications ............................................................................................................ 18

2.5. Charting libraries............................................................................................................................ 19

3. Methodology / project development: ....................................................................................................... 20

3.1. Arduino (sender side) .................................................................................................................... 20

3.2. Server (receiver side) .................................................................................................................... 21

3.3. Client ............................................................................................................................................. 22

3.4. Charting ......................................................................................................................................... 22

3.5. Usability ......................................................................................................................................... 24

3.6. GUI ................................................................................................................................................ 25

3.7. Window resize event ...................................................................................................................... 26

4. Results ................................................................................................................................................... 27

5. Budget .................................................................................................................................................... 29

6. Conclusions and future development: .................................................................................................... 30

Bibliography: ................................................................................................................................................... 31

Appendices: .................................................................................................................................................... 32

1. Work Packages ...................................................................................................................................... 32

1.2 Milestones .......................................................................................................................................... 34

Glossary .......................................................................................................................................................... 35

7

List of Figures

Figure 1: Scheme of the whole system………………………...………………………………11

Figure 2: Block diagram……………………………………………………………..…………..12

Figure 3: Gantt diagram……………………………………………………………...………….13

Figure 4: Duplicated broadcast data……………………………………………………...……14

Figure 5: Neonatal Brain Stethoscope………………………………………………….……..15

Figure 6: Sparkfun ESP32 Board. Taken from https://www.sparkfun.com/..…...…….……16

Figure 7: UDP Datagram………………………………………………………………………..17

Figure 8: Comparison between UDP and TCP.

Taken from https://www.colocationamerica.com/blog/tcp-ip-vs-udp.................................18

Figure 9: Client – Server scheme communication. Taken from https://docs.citrix.com/en-

us/citrix-sd-wan-wanop/10-2/how-html5 works.html ……………………………….………..19

Figure 10: D3.js logo……………………………………………………………………….……20

Figure 11: Plotly.js logo………………………………………………………………………….20

Figure 12: Parsed values loop………………………………………………………………….21

Figure 13: WS code connection at the server side……………….……………….………….22

Figure 14: WS code connection at the client side…………………………………………….22

Figure 15: Whole file scheme of the project…………………………………………………..23

Figure 16: Time split axis…………………………………………………….………………….24

Figure 17: Dropdown menu……………………………………………………..………………24

Figure 18: First implementation of two buttons………………………………………………..25

Figure 19: Performance of the main button…………………………….…………….……….25

Figure 20: Android APK version ……………………………………………………….………25

Figure 21: Window resize event……………………………………………………..…………26

Figure 22: Comparison of data received in the server side………………………….……….27

Figure 23: 1Hz Sine wave as input value..……………………………………………………27

Figure 24: Final design web page………………………………………………….…………..28

8

List of Tables:

Table 1: Parse values HEX to Floats…………………………………………………………...21

Table 2: General project budget………………………………………………………………..29

Table 3: Components list…..……………………………………………………………………29

Table 4: Personnel table………………………………………………………………………...29

9

1. Introduction

1.1. Project Background

The real-time EEG interpretation project is a small piece of the whole NeuroBell project.

Neurobell is one of the many projects carried out at the INFANT Research Centre. This

project began with the aim of increasing the number of medical professionals capable of

diagnosing abnormalities in neonatal brain signals (i.e. EEG).

Accurate interpretation of EEG data is a difficult task. Therefore, a high level of expertise

and training is required. Despite the fact that EEG data is the most commonly used data

for interpreting neonatal brain function, there is still a lack of experts in the field to interpret

the EEG data in a timely manner.

The research team hosted at UCC was challenged with implementing a solution for

facilitating EEG monitoring using sonification and deep learning AI via an Android app to

provide all the information in a more user-friendly interface so that it would expand the

number of medical professionals capable of interpreting EEG data.

The main focus of this project is the android app aspect, which provides visualization of a

single-channel of EEG, a seizures detector via traffic-light indication through trained deep

learning convolutional neural networks (CNNs) and the data sonification which is useful to

detect changes in the EEG morphology in the event of neonatal seizures.

This project will contribute to the research of multi-platform systems to be compatible with

a wide range of devices.

Figure 1: Scheme of the whole system

10

1.2. Project overview and goals

The project was carried out at Embedded Systems Research Group in the School of

Engineering and with the Irish Centre for Fetal and Neonatal Translational Research

(INFANT) at University College of Cork (UCC). The purpose of this project is to implement

a communication system between an Electroencephalogram (EEG) data acquisition block

and a web-client block, which enables user-friendly interpretation of EEG data in task the

of neonatal seizure detection.

The whole system overview consists of three blocks as illustrated in Figure 1. The first

block is the EEG data acquisition system, where the data is acquired through electrodes

and sent it to the Arduino board. This block has already been developed by the research

team at UCC. Therefore, this project will use EEG data already acquired and stored in text

files. However, it will be sent by simulating the real-time acquisition.

The second and main block consists of the communication protocol to push the EEG data

to a server using a NodeJS application. The Arduino board acts as a Wi-Fi transceiver and

sends the data to a private access point where the server is connected in order to receive

and store the EEG data.

The final block starts with the communication between the server and the client side using

the WebSocket protocol so that each client that connects to the private access point would

be able to access and send data to the real-time EEG graph.

The project’s main goals are:

a. Implement an Arduino script that processes the EEG data acquired and converts

the board into a Wi-Fi transceiver.

b. Implement a server capable of receiving and processes the data.

c. Configure the communication protocol between server and client side.

d. Design the front-end GUI for the HTML web-page.

e. Ensure reliable, real-time plotting of the EEG data.

11

1.3. Project requirements and specifications

Project requirements:

- A fully functional communication between the acquisition block and client side without

losing data packets.

- Build a user web interface similar to the actual android app in order to display the graph.

Project specifications:

- Non-stop EEG data-flow communication to the server when request is done.

- Real-time graph on the client side.

- High-speed data plotting, with a minimum of 256 samples per second.

- Chart window length of up to 20 seconds of EEG data

1.4. Work Plan (Tasks and Gantt diagram)

The work plan has been divided into 4 different work packages as follow in figure 2:

Figure 2: Block diagram

12

The final Gantt diagram of the whole project is presented in the Figure 3:

Figure 3: Gantt diagram

13

1.5. Incidences and deviations

Initial time expectations were not measured correctly as some parts of the project suffered

from some unexpected incidences, which required more time. In the final work plan, the

signal processing blocks referred as “Running Algorithms” was removed and proposed as

future development.

A similar situation has happened with the initial block of “Scalability” which was supposed

to be done once the final webpage was correctly performing. This block was supposed to

implement more than one input signal. Unfortunately, during the first few months of

development, it was decided that there was not enough time for such an ambitious final

product, since the product development time was reduced by one month. Therefore, a

decision to remove the less relevant features was taken.

With regard to the Gantt diagram, as the initial time-frame for the project development was

6 months, it was decided to reduce this by one month to allow for time to prepare the final

presentation of the project, which was due in the beginning of the 6th month, as seen in

Figure 3.

Despite the previous time adjustments, the project has been in constant development until

the implementation of the WebSocket protocol, where almost 3 weeks was dedicated to

solving the problem where only a constant amount of values (around 300 values) were

transferred from the server to the client. After a long period of debugging, the problem was

fixed by controlling the generation and closing of new client connections.

After all the incidences were fixed, the system still had a drawback. In the last blocks of the

project, in the broadcast connection, when more than one connection is required, the

application does not work as expected.

When the first user connects to the webpage, the system is able to plot the data correctly

as it is sending the data for the first time. However, when a second user tries to connect to

the web page, it will receive the data duplicated affecting the first user’s plotting speed.

The duplicated data received by the second client connection is shown in the Figure 4 with

a sine wave as an input signal, which has been zoomed into. In the event of more than one

simultaneous user, the data is multiplied by the numbers of new connections.

Figure 4: Duplicated broadcast data

14

2. State of the art of the technology used or applied in this

thesis:

2.1. Background

Seizures are a disorder that many people have to face daily and can be really challenging,

especially in the case of newborn babies. The most common tool to detect this disorder is

electroencephalography (EEG), which enables real-time recording of the electrical activity

of the brain.

This method usually requires special and expensive equipment, the main elements consist

of electrodes placed on the along the scalp and an amplifier box to acquire the data. The

most commonly used electrodes require abrasive creams and conductive gels to reduce

skin impedance which makes this process quite invasive and time-consuming [1]. Studies

have shown that without the usage of EEG interpretation, less than 9% of seizures would

be detected [2].

The use of the EEG for newborn babies requires a highly-qualified healthcare experts who

are very limited. The lack of resources has prompted research in the area of automated

seizure detection [3]. There is a simpler form of EEG which can be interpreted by neonatal

intensive care unit staff, called amplitude integrated EEG (aEEG). Although it is simpler to

interpret, it has several limitations as there are not still enough experts able to interpret the

data.

In recent years the demand for improved brain health monitoring has increased. Therefore,

research is being conducted in order to implement more portable equipment that requires

less expensive tools and tries to avoid the time-consuming patient preparation by using dry

electrodes without a reduction in EEG signal accuracy. [4].

The whole scheme of the system consists of dry electrodes attached along the scalp of the

newborn, a low-cost portable EEG acquisition board that process the data and pushes it

via Bluetooth Low Energy (BLE) connection to an Android device where a friendly GUI is

implemented.

Figure 5: Neonatal Brain Stethoscope

15

Previous studies have explored EEG sonification to facilitate EEG interpretation in an

easier way, so that non-experts can make use of this feature. It has been demonstrated

that the visual sense is less accurate than human hearing in assessing the spatial and

temporal evolution of frequency [5].

The algorithm implemented in [6] reflects the higher sensitivity of the human hearing

system to the presence of the rhythm. A phase vocoder is developed in order to detect the

frequency changes over time in the audio mapping the spectrum by preserving the phase

coherence [7].

For long-term usage of raw EEG data, a deep learning convolutional neural network (CNN)

was implemented in [8] for the task of seizure detection. The architecture implemented

enabled an optimal feature extraction and classification stages in one routine [1]. Different

layer depths were tested and compared in [8].

2.2. Data Acquire – ESP Wi-Fi Board

On the communication side, research has been done in order to select the most efficient

board to meet the system requirements.

Arduino has a wide variety of microcontrollers that enables connection to internet. Many

years ago, Arduino released the first version of a “Wi-Fi Shield”, a basic board that enabled

users to plug it onto other modules so they could connect it with the network.

A more powerful board was released by Arduino MKR1000 combining different features

from past Arduino boards including the Wi-Fi Shield.

There are several variants of the MKR1000 board in the market and different companies

have released different microcontrollers based on the original one compatible with the

Arduino IDE. It prompted research into the most convenient board for a data

communication with the lowest power consumption and price.

Sparkfun electronics developed a low-consumption board called ESP8266 module which

is an integrated chip with Wi-Fi connection. The main goal is to allow the microcontroller to

access a Wi-Fi network. Sparkfun also released an advanced version of the previous model,

the ESP32 module which still works with the same specifications as the 8266 with Wi-Fi

compatibility, but it adds support for Bluetooth low-energy which is really useful as BLE is

used in the previously completed research [1].

Figure 6: Sparkfun ESP32 Board.

16

The decision to use the ESP32 was taken due to its low power consumption, meaning it

can operate with a voltage range of 2.2 to 3.6V [9]. It also supports IPv4 and

TCP/UDP/HTTP/FTP protocols that are really useful for the EEG data communication to

the server.

2.3. Data transmission

The communication between the Wi-Fi transceiver board and the server, requires a

communication network protocol in order to send the EEG data in small packets. At the

early stage of the project, some research was conducted in order to decide which protocol

was the most suitable for our system requirements.

The two main network protocols are the User Datagram Protocol (UDP) and the

Transmission Control Protocol (TCP) which both run on the top of the internet protocol (IP).

2.3.1 UDP

UDP it is a non-connection-oriented protocol, in other words, it enables process-to-process

communication. It does not establish a connection or check if the destination computer is

ready to receive the data, it only sends the data directly. UDP is used to transfer data at

fast rate as it does not wait for any reception confirmation. It does not guarantee data

delivery or re-transmit lost packets, so it is less reliable. It is commonly used to transmit

data such as audio and video files.

It provides port numbers to identify the different user requests and can send a value that

represents the number of bits in a transmission message, also called checksum. The UDP

messages are encapsulated and sent in IP datagram as seen in the following example:

Figure 7: UDP Datagram

2.3.2 TCP

TCP establishes a connection between the source and the destination computer before

initiating the communication. One of its characteristics is that it is highly reliable, since it

uses handshake or 3-way system, flow control, error and congestion. TCP ensures that the

data sent from the source computer is accurately received by the destination computer. In

case the received data is not in the appropriate format, TCP re-transmits it.

TCP can also be written as TCP/IP, as the internet protocol (IP) handles the actual delivery

of the data and TCP keeps the track of packets of data transmitted from a message, also

called segments.

TCPs accuracy of delivering packets makes the process slower, so the connection speed

is affected dramatically.

17

In Figure 8 a comparison between both protocols is made. It is easy to identify the fast

sending response from the sender to the receiver.

Many client-server applications that have a request and response use UDP instead of

taking the trouble to establish and then release a connection [10].

Regarding the project’s specifications, a higher priority has been given to a high-speed

communication, leading us to a UDP implementation on the ESP32 board. Hence, in this

project, the board will send EEG data segments in packets to the server at high speed and

the server will receive and process them without sending any acknowledge back to the

sender.

Figure 8: Comparison between UDP and TCP.

18

2.4. Real-time communications

Real-time communications have been growing enormously. Our daily activities involve real-

time web applications. There are many ways of implementing such web applications, the

first model of client connections is the HTTP model that uses a long-polling technique.

Long-polling keeps a HTTP connection open until the server has data to be pushed to the

client, this process increases the latency as it leads a bunch of data being transferred to

the server every time a HTTP request is done.

The second most common tool for real-time applications is called WebSocket (WS). This

method provides a continuous and low-latency connection that enables the server and the

client to send messages to each other at any time. This technique is also called full-duplex

connection over a TCP single connection. Whereas, HTTP provides a half-duplex

communication with a slower speed connection, as tested in [11].

The WS procedure begins with the handshake between the client and the server. Despite

the HTTP latency, the client sends a regular HTTP request to the server by pushing a

header to inform the server side that the client wants to establish a WebSocket connection.

The are some libraries that implements WS technology to prove the interface. The WS

interface is usually divided into two parts: the client side that runs inside the browser and

the server side that works with Node.js.

Socket.IO is a library that carries all the WS functionalities and adds multiple improved

functions such as providing the event-based communication between the browser and the

server. It also enables a connection established in the presence of proxies, support for

broadcasting and fallback options. This library is usually used for instant messaging chats.

Both, the Socket.IO library and plain WebSockets has been tested and implemented

through this project. Finally, the project works with WS so that every time the server

receives data, it automatically pushes it to the client and the client is able to display it. In

contrast, when the connection is made with Socket.IO, a manual interval has to be set on

the client side to send requests to the server providing possible latency between the

requests.

Figure 9: Client – Server scheme communication

19

2.5. Charting libraries

In order to visualise the EEG data, the project implements a graph on the front-end user

interface to plot the received data on a clear and simple graph.

The chart generation is the crucial part for a user-friendly GUI, as the graph is the main tool

for doctors to interpret the data. Different libraries dominated by SVG (Scalable Vector

Graphics) have been implemented throughout the project in order to reach the performance,

speed and resolutions required. Plotly was the library used for in the final implementation

of this project:

D3.js

D3 is one of the most common libraries used as it is open source and free to use. Its

emphasis on web standards provides compatibility with most browsers and it contains

extensive and powerful tools that enable you to implement almost any kind of chart with

really useful APIs and documentation to guide you in changing the layout with CSS.

Chart.js

Chart.js is a simple, yet flexible open source JavaScript library. It plots the data smoothly

with some predefined animations that let the graph look modern. However, the major

drawbacks are the limited number of chart types and CSS customization.

Plotly.js

Plotly differs from the others as it is an online library for data analysis and visualization. It

provides very accessible documentation and tutorials. Each of the graphics can be

exported in .png, PDF, SVG and EPS formats, and you can choose the width and height

measurements of the visualization to suit the requirements for EEG visualisation.

Figure 10: D3.js logo

Figure 11: Plotly.js logo

20

3. Methodology / project development:

As previously stated, this project is divided into three different blocks. The first block

develops the data acquisition, which had already been implemented by the time this project

began. Therefore, text files of previously acquired EEG data was used.

3.1. Arduino (sender side)

Regarding the connection block, a member of the team (Mark O’Sullivan) implemented the

UDP protocol, establishing the connection from the board to a private WLAN using C

programming language in the Arduino IDE.

The sender implementation on the Arduino board consists of firstly reading the EEG values,

where each float value is parsed into four bytes. The decision of parsing the data into four

bytes was taken after some previous implementations with sending float values directly

resulted in only HEX values being received and many packets were lost.

Therefore, by sending four bytes at the sender part and receiving the HEX values at the

receiver, it is easier to assign an array of length four for each value received and parse it

back into floats.

The 32 bits value is sent to an array of 4 values where they are sent one by one

consecutively inside the following loop:

*(float*)(byteV) = buff[k];

for (int b=0; b<4; b++){

bytes[4*k+b] = byteV[b];

Udp.write(byteV[b]);

} Figure 12: Parsed values loop

As it can be seen in the following example, in order to parse the HEX value ‘4’ into bytes,

an array of length 4 is needed as it converts to “0100”.

Table 1: Parse values HEX to Floats

HEX

FLOAT

21

3.2. Server (receiver side)

Moving back to the receiver creation, the project is mainly implemented with JavaScript for

the whole connection block shown in table 1.

A proper connection from the server with the private access point has to be done which

uses the UDP protocol in order to establish the communication from the board to the server

by assigning the port and IP.

Once the connection is implemented, some tests were completed to check that the data

received has no data loss and the hex to float parsing is done correctly for each value.

A client side must be implemented in order to display data on a HTML web page

implementation.

There are different protocols which establish connections between a server and a client.

Research was conducted to find the most efficient and suitable protocol that suits the

system requirements. As a result, the system uses the WebSocket (WS) protocol

[mentioned in 2.4].

The WS protocol requires a basic code structure as seen in Figure 13. It enables the sender

to send or broadcast data to one or more clients at the same time.

function onSocketConnect(ws) {

sender.on('message', function(EEG) { //EEG = data received from the sender (Arduino)

for (let client of clients) {

client.send(JSON.stringify(f32));

};

});

};

Figure 13: WS code connection at the server side

Regarding the client side, the following code is the basic scheme for the reception and

storage of the data in the variable “EEG” which was used in the code.

socket.onmessage = function(event) {

var EEG = JSON.parse(event.data);

};

Figure 14: WS code connection at the client side

22

3.3. Client

The main purpose of the client is to receive and plot the data in a HTML file that is usually

called index.html, where a chart displays the values. A lot of time was invested in the

implementation of different graphs until finding that plotly [mentioned in 2.5] was best for

plotting the EEG values smoothly.

The firsts steps of the client development were the reception of values as seen previously,

pushing the data received to the html file, and the creation of the html file in order to start

displaying the values in the browser.

The CDN (Content Delivery Network) is used to include the plotly features into the HTML

file, it enables the client to use these features for further chart customization aside from the

CSS settings.

A basic representation of the whole file scheme is shown in Figure 15, which gives a better

understanding of the system flow.

The Arduino file runs a function that sends a chunk of data every 100ms to the server. The

WS protocol establishes the connection between the server and the client, enabling them

to send messages to each other at any time without an explicit request. Therefore, the

client is able to refresh or update the chart synchronously, and packet delivery latency only

falls on the sender Arduino part.

3.4. Charting

The main feature implemented has been the time-scale representation. The chart is

supposed to contain 20 seconds of EEG data within the chart.

The system sends one packet that contains 25 EEG values every 100ms, which implies

that the client side should be able to update the chart at an average rate of 250 values per

second. A further implementation of the windows size is done in order to compute the sizes

for filling the 20 second plot.

Time stamps have been assigned for each value received in order to have a better

understanding of the data that is being plotted.

On the top of that, a dropdown menu was implemented that enables the final user to change

the timescale according to their preference with 5, 10, 20 and 30 second options.

Figure 15: Whole file scheme of the project

23

The procedure to change the timescale is determined by equation 1, which re-computes

the segment each time the user selects a new value. Speed is the time value selected on

the GUI and Fs is the sampling frequency. A maximum buffer value is defined in order to

prevent overlap of the window size, which is set to 30. The result of the equation S is the

number of samples that need to be cropped in order to reduce the window size.

𝑆 = 𝐹𝑠 × (30 − 𝑆𝑝𝑒𝑒𝑑) (1)

Once the S value is known, the charting script will only show the values of the buffer B from

S to length(B) (i.e. the last values in the array). Note that when Speed = 30, S = 0, which

means that all values on the buffer are shown (from 0 to length(B)).

A similar implementation was completed for the amplitude range. The unit of EEG

amplitude data is 𝜇𝑉 , the system is predefined to plot at 100 𝜇𝑉 . However, a second

dropdown menu has also been implemented with different amplitude ranges of 10, 20, 50,

100, 200 and 500 𝜇𝑉.

Regarding the graph settings, a third option has been implemented in order to make the

graph more efficient at the time the browser calls the chart for the first time.

The three settings mentioned were placed on the right side of the web page where a pink

section wraps them to display it in a simple but user-friendly interface.

The chart is displayed on the left side of the page, taking approximately 70% of the whole

horizontal length screen, while the graph settings occupy the remaining 30%.

At the top-right side of the chart, a small option menu which is implemented by default in

the plotly library. This enables the user to take a screenshot of the data plotted, zoom in-

out, reset axes and other options.

Figure 17: Dropdown menu

Figure 16: Time split axis

24

3.5. Usability

Two buttons were implemented to enable the user to start and stop the reception and

plotting of the EEG data in the chart. Despite both buttons working correctly, a usability

study was completed to optimize the performance of the buttons. As a result, the individual

start and stop buttons were removed and merged into only one push button that does both

tasks and occupies less space.

The first time the graph runs, by default it starts at amplitude 0, with no signal, the user has

to click on the “START” button in order to let the client side know when it has to start pushing

the data into the web page and updating the chart with new incoming values.

In case it is required to stop the plot, the user would have to click again on the button that

stops updating the graph with EEG values and pushes ‘0’ values to the graph as no input.

The decision of implementing dropdown menus for the selection of the Amplitude and Time

was also carefully studied in order to ensure the most efficient usage.

Several types of menus and buttons were studied. Radio buttons are really intuitive as the

user is able to see at glance all the options available. However, despite its simplicity, it

occupies a lot of unnecessary space in the screen.

Another type of button studied was the slider button which gives the user the possibility to

choose from a wide range of values. This button could have been useful for selecting the

time measured in seconds. However, it has a lack of precision when selecting a specific

value.

For this reason, this GUI contains dropdown menus where the user is able to click and

choose between a long range of values, while occupying very little space.

Figure 18: First implementation of two buttons

Figure 19: Performance of the main button

25

3.6. GUI

The last stage implemented was the design of the final GUI. Many designs were completed

throughout the project, with the several implementations and tests of the different charts.

Each chart commonly uses its own design by default. As already mentioned, some libraries

allowed us to change its design while others do not. Therefore, a draft design had to be

done considering the default chart setting driving the web design into a different idea from

what was originally expected.

Since the final implementation of the chart was completed using the plotly library, it was

possible to change the predefined graph settings, such as colours, fonts, grid options and

even the plotting line directly in the JavaScript function.

In terms of the web page layout, the system has been designed as similar as possible to

the android version in order to establish a multi-platform/device system.

A basic colour pallet used in the previously designed android app version can be seen in

Figure 20. In the image below, the patient ID can be seen, this project did not implement

this feature due to the magnitude of the development.

In both cases, the design has been implemented using RGB colour codes, so it was easy

to use same reference colours for the dark grey background and the pink colour that

represents the INFANT Research Centre brand.

In comparison to the Android version, this project uses a white background colour and a

pink plotting line. This simple change was done in order to ease future work on the project,

as more than one trace of EEG is often plotted within one graph, therefore, future

implementations will be able to add more traces which can be easily identified by assigning

different line colours to each of them.

A basic header has been added to the web page where the INFANT logo can be found on

the right side while the project and app name are shown on the left. It is a fixed header for

aesthetic reasons, the header stays in the same place even if the page is scrolled down,

the element would not leave a gap as it is displayed above the background content.

Figure 20: Android APK version

26

3.7. Window resize event

Regarding the two sections on the webpage, the chart and the graph settings block, an

implementation was done in order to detect when the user is readjusting the window size.

In the case which the user shrinks the window, a threshold has been set to detect when

both sections are about to overlap and then the right section goes to the bottom part so

that the chart can keep a proportional scale size for a proper interpretation.

Figure 21: Window resize event

The same implementation was done for the opposite case where the user enlarges the

window so that both sections have enough space to be displayed together, then the blocks

come back to inline section as the original layout.

27

4. Results

At the end of the project, the initial goal was successfully achieved. Despite several tests

with different libraries for plotting charts, the system is able to send the EEG data from the

text file to a server through a Wi-Fi transceiver and plot the data on a user-friendly web

GUI.

First tests were implemented in order to check the correct reception of parsed values on

the server side. In the following figure it can be seen the data that is received in

hexadecimal format directly from the Arduino board and the latter is the data parsed into

floats, specifically the 25-length array with EEG values that is then sent to the client side.

On the client side, in order to check that there was no data lost through the communication,

a 1Hz sine wave was generated so that it was easy to identify any lost packets in the signal

reception.

Figure 22: Comparison of data received in the server side.

Figure 23: 1Hz sine wave as input value

28

The final design is shown in Figure 24. The three main blocks can be easily identified. The

header with the project name and the Infant logo is at the top, while the two big sections

are in-line: the chart on the left side and the graph settings on the right side.

It can be seen in the chart that the EEG data is being plotted with an amplitude of 100 𝜇𝑉

and a time representation of 5 seconds on the x-axis.

Figure 24: Final design web page

29

5. Budget

The project has required free licence software*, one Arduino board and one Junior engineer.

The total budget of the project is:

Items Cost (€)

Components 510

Personnel 12000

Free licence Software 0

Total 12510

Table 2: General Project Budget

Components:

Component Units Cost

Arduino ESP32 Board 1 10€

Computer 1 500€

Table 3: Components list

Personnel:

Employee Annual Salary Months Cost

Research Assistant/Junior

engineer

24000

6 12000€

Table 4: Personnel table

*The Software required for this project has been the Arduino IDE and a source-code editor that enables JavaScript and HTML coding.

30

6. Conclusions and future development:

In conclusion, the project’s requirements and goals have been successfully achieved. The

development of a new platform for EEG interpretation has been completed, allowing the

end-user to connect (with a wide browser compatibility) to a web page in order to interpret

the real-time EEG data received in a user-friendly GUI in order to increase the demographic

of clinical staff capable of understanding and assessing the plotted EEG data.

Despite the functional performance of the system, the project did not merge the fully

functional acquisition block, therefore, a recommendation for future development would be

to start working on the early stages of this project in order to develop a connection directly

with the dry-electrodes connected to the newborn’s scalp. Another interesting improvement

would be an option on the GUI to connect to a newborn in real-time or instead to interpret

EEG data already stored on the server with off-line communication.

Meanwhile, more research and development could be done to explore and implement new

features on the web page. For instance, an interesting feature would be the option to add

more EEG signals using different traces on the chart, enabling comparisons between the

signals or even between different patients.

New projects could be proposed based on this system. Regarding the raw signal, some

process could be done in order to work with different blocks of signal processing as

developments of algorithms for EEG robustness, filtering and resampling or even the

addition of sonification algorithms.

31

Bibliography:

[1] O’Sullivan, M., Gomez, S., O’Shea, A., Salgado, E., Huillca, K., Mathieson, S., Boylan, G.,

Popovici, E., Temko, A. (2018) Neonatal EEG Interpretation and Decision Support Framework

for Mobile Platforms. In Proceedings of the 2018 40th Annual International Conference of the

IEEE Engineering in Medicine and Biology Society (EMBC), Honolulu, HI, USA.

[2] Murray, D.M., Boylan, G.B., Ali, I., Ryan, C.A., Murphy, B.P., Connolly, S. (2008). Defining the

gap between electrographic seizure burden, clinical expression and staff recognition of neonatal

seizures. Arch. Dis. Child. Fetal Neonatal Ed. 2008, v. 93, pp 187–191.

[3] Temko, A., Thomas, E., Marnane, W., Lightbody, G., Boylan, G. (2011). EEG-based neonatal

seizure detection with Support Vector Machines, Clinical neurophysiology: official journal of the

International Federation of Clinical Neurophysiology. v. 122, pp. 464–473.

[4] O’Sullivan, M., Pena, J.P., Bocchino, A., O’Mahony, C., Costello, D., Popovici, E., Temko, A.

(2017). Comparison of electrode technologies for dry and portable EEG acquisition. In

Proceedings of the 2017 7th IEEE International Workshop on Advances in Sensors and

Interfaces (IWASI), Vieste, Italy, pp. 15–20.

[5] Khamis, H., Mohamed, A., Simpson, S., Mcewan, A. (2012). Detection of temporal lobe seizures

and identification of lateralisation from audified EEG. Clinical neurophysiology: official journal of

the International Federation of Clinical Neurophysiology. v. 123. pp. 1714-20.

[6] Gomez, S., O'Sullivan, M., Popovici, E.M., Mathieson, S., Boylan, G.B., Temko, A. (2018). On

sound-based interpretation of neonatal EEG. 2018 29th Irish Signals and Systems Conference

(ISSC), Ireland.

[7] Flanagan, J. L. and Golden, R. M. (1966) Phase vocoder, in The Bell System Technical Journal,

v. 45, no. 9, pp. 1493-1509.

[8] O'Shea, A., Lightbody, G., Boylan, G., Temko, A. (2018). Investigating the impact of CNN depth

on neonatal seizure detection performance. In International Conference of the IEEE Engineering

in Medicine and Biology Society (EMBC), Honolulu, HI, USA.

[9] ESP32 Thing Hookup Guide, https://learn.sparkfun.com/tutorials/esp32-thing-hookup-

guide?_ga=2.168659048.854105827.1575999121-1872461687.1575999121 (Last access

December 2019)

[10]Molina E.M., “Protocolo de datagramas de usuario (UDP)”:

http://www.redtauros.com/Clases/Fundamentos_Redes/07_Protocolo_Internet_UDP.pdf (Last

access November 2019)

[11]Matt West, (October 2013). An Introduction to WebSockets: https://blog.teamtreehouse.com/an-

introduction-to-websockets (Last access November 2019)

32

Appendices:

1. Work Packages

Project: Design of a web graphic user interface for real-time

and off-line EEG interpretation WP 1

Major constituent: Background/Introduction to Arduino Start event: 16/09/2019

End event: 30/09/2019 Short description:

Studying the state-of-art of Arduino boards, the different

possible implementations.

Study the different code languages required.

Internal task T1: Check for Arduino ESP-32 basics.

Internal task T2: Simple first implementation of Arduino scripts.

Internal task T2: Get familiar with the HTML, CSS languages.

Project: Design of a web graphic user interface for real-

time and off-line EEG interpretation WP2

Major constituent: hardware prototype, software Start event: 01/10/2019

End event: 13/10/2019

Short description:

The Android board implementation in order to send

EEG data via web-socket. Analyze the most efficient

way of sending packets with different protocols.

Establish the communication between servers and

monitoring devices and implement the receiver part

using JavaScript protocols.

Internal task T1: Implement the Arduino sender script part.

Internal task T2: Set up of the access point in order to settle a communication path.

Internal task T3: Understand how NodeJS works.

Internal task T4: Implement the server to receive the EEG data.

33

Project: Design of a web graphic user interface for real-time

and off-line EEG interpretation WP 3

Major constituent: SW Start event: 14/10/2019

End event: 18/12/2019 Short description:

Web page implementation

Plot the EEG data received.

HTML and CSS course at Code Academy.

Internal task T1: Web page implementation

Internal task T2: Plot the EEG data received.

Internal task T3: HTML + CSS course

Internal task T4: Implement the webpage using HTLM, CSS.

Project: Design of a web graphic user interface for real-

time and off-line EEG interpretation WP 4

Major constituent: SW Start event:18/12/2019

End event: 30/12/2019 Short description:

Implement and design a user interface webpage to

display the EEG data considering the usability of the

final user.

Reach a multiple connection and adapt the system for

more than one EEG data sender.

Internal task T1: Design the user interface usability.

Internal task T2: Integrate the graph into the webpage.

Internal task T3: Scalability.

T1: FINISHED

T2: FINISHED

T3: Pending on external

project

34

Project: Design of a web graphic user interface for real-time

and off-line EEG interpretation WP 5

Major constituent: SW

Short description:

Implement the established filtering and down-sampling of

the EEG signal acquired through the board to finish the

system.

Pending on external project

Internal task T1: Filtering

Internal task T2: Down sampling

1.2 Milestones

WP Task Short title Milestone / deliverable Date

2 4 Reception of the EEG data Receive the data sent from ESP32 13/10/2019

3 1 Webpage Running webpage with chart 21/10/2019

3 4 Display EEG Display the chart with proper EEG data 18/12/2020

5 2 Final product Final UI with real-time EEG acquisition 26/01/2020

35

Glossary

EEG: Electroencephalogram

aEEG: Amplitude integrated electroencephalography

BLE: Bluetooth Low Energy

CNN: Convolutional Neural Network

TCP: Transmission Control Protocol

UDP: User Datagram Protocol

IP: Internet Protocol

WS: WebSocket

HTTP: Hypertext Transfer Protocol

SVG: Scalable Vector Graphics

IDE: Integrated Development Environment

CDN: Content Delivery Network

GUI: Graphical User Interface

RGB: Red, Green, Blue

CSS: Cascading Style Sheets

JS: JavaScript