BRINDISYS BRAIN-COMPUTER INTERFACE DEVICES TO SUPPORT

INDIVIDUAL AUTONOMY IN LOCKED-IN INDIVIDUALS

Febo Cincotti

adottato da

Goal of the project

• Development of an assistive system aimed at

preserving the communication capabilities of a

person who progressively loses his/her muscular

force, coping with his/her residual abilities.

In other words?

• Conventional assistive technologies must rely on a physical interaction mediated by muscular contraction

• Brain-Computer Interfaces (BCIs) detect the user’s intention by processing his/her brain activity

• We demonstrated that this technology can be used in an assistive device designed for persons with ALS

3

BRAIN COMPUTER

INTERFACES [4]

What a BCI is

“Brain–computer interfaces (BCI’s) give their users communication and

control channels that do not depend on the brain’s normal output

channels of peripheral nerves and muscles.”

“A BCI changes the electrophysiological signals from mere reflections of

CNS activity into the intended product of the activity: messages and

commands that act on the world”

Wolpaw et al. 2002, clinph

5

BCI: a logical scheme

Cognitive processing

(Intention)

Modification of

Brain Signals Signal Features

Classification

(Translation)

of Intent

appropriate feature extraction

appropriate feedback strategy

us

er

tra

inin

g

(le

arn

ing

)

co

mp

ute

r train

ing

(ca

libra

tion

)

Environment

[6]

Neuroelectrical features for BCIs

Rhythms Evoked potentials

Steady-state EPs Slow Cortical Potentials

[7]

P300 Potential

• The P300 is an event-related potential, dominating at parietal electrode sites.

• P300 follows unexpected sensory stimuli or stimuli that provide task related information

[8]

From Selllers & Donchin 2006, clinph

Oddball stimulus Stimuli

Brain responses

[9]

Relevant stimulus

10

P300 Speller

[11]

BCI application

or

BCI-powered application

?

BCI AS PART OF ASSISTIVE

TECHNOLOGIES [12]

Augmentative and Alternative Communication

• AT software running on PC

– Designed to be controlled through alternative input channels

[13]

Support to persons with ALS

• Access to the assistive device is customizable to

the residual motor functions of the user

– Touchpad, joystick, sensors, …

– … and a Brain Computer Interface

• Usable in the locked-in condition

[14]

System overview

[15]

Communication Environmental Control

Brindisys Applications

Virtual keyboard

Vocalize

fixed

sentences

Alarm bell Domotic

kit

TV/stereo

Control

Appliances

ON/OFF

• Scanning modality

– Switch devices

• Dwelling function

– Mouse emulator

• P300-based BCI

– Overlay approach

Universal accessibility

Hardware modules

[18]

The role of users

• Design

• Interim feedback

• Final assessment

[19]

User-centered design

“Please score the

limitations present in

communication aids that

you use ”

User Survey: Aids limitations

Participants:

7 End Users

13 Caregivers

20 Stakeholders

User-centered

design

0 1 2 3 4

Weight

Limited wearability

Esthetics

Difficulty of use

Size

Limited number of…

Slowness

Lack of customizability

Limitation

“Please rate the usefulness

of the following possible applications of assistive

technology for interpersonal communication,

environmental interaction and

personal autonomy”

User Survey: Functionalities

Participants:

7 End Users

13 Caregivers

20 Stakeholders

User-centered

design

0 1 2 3 4

Games

Chat

Opening Doors, awnings and shutters

Send SMS

Icons for communication

Web browsing

Temperature Control

E-mail

Write text

Raise/lower bed or airmchair

Light Control

Alphanumeric character selection

TV/stereo Control

Alarm bell

Utility

At home with the Brindisys system

[22]

Evaluation: participants

8 persons with ALS were recruited from the ALS Centre of Sapienza University

of Rome (Policlinico Umberto I).

Participants

Sex Age ALSfrs-r Onset Conventional/Assistive input device

Subj1 M 55 13 spinal Automatic Scanning - 1 button

Subj2 M 59 37 spinal touch screen

Subj3 M 47 34 bulbar mouse

Subj4 F 75 38 bulbar mouse + keyboard

Subj5 F 72 34 bulbar mouse

Subj6 M 40 31 spinal Scanning - 2 buttons

Subj7 M 61 28 bulbar Scanning - 2 buttons

Subj8 F 72 41 bulbar mouse

Francesca Schettini

User-centered

design

The preliminary prototype was tested in

controlled experimental conditions

Evaluation

Prototype usability*

(effectiveness, efficiency and user satisfaction)

Conventional Assistive input

Communication Task

Environmental Control Task

P300 based BCI

Communication Task

Environmental Control Task

*(Riccio et al. 2011; Zickler et al. 2011)

User-centered

design

Results

Effectiveness Efficiency

User Satisfaction

70%

80%

90%

100%

ConventionalAssistive Input

P300 based BCI

Accuracy of selection

10

20

30

40

50

ConventionalAssistive Input

P300 based BCI

Workload (NASA-tlx)

6789

10

ConventionalAssistive Input

P300 based BCI

User Satisfaction (VAS 1-10)

60

70

80

90

100

Conventional AssistiveInput

P300 based BCI

System Usability Scale (SUS 0-100)

No statistical differences were detected

between the conventional/assistive

inputs and the BCI for Effectiveness,

Efficiency and User Satisfaction

CONCLUSIONS

[26]

What we already know

• An individual can learn to communicate in absence of muscular contraction

• This skill can be employed to establish a non-muscular alternative channel for the motor impaired.

• Real-time processing of biosignals provides a new tool for the neuroscience, with prospective implications for the clinical practice

• Only multidisciplinarity can boost research and development in the field of technological advancements for clinical applications

[27]

Open issues

• Does this skill persist in the late stages of ALS?

– Impaired eye movement

– decline in cognitive abilities, arousal and attention,

– extinction of goal-directed cognitive commands and intentions

• Limitations

– Speed

– Accuracy

– Usability

• Research platforms vs. commercial device

[28]

Acknowledgements

The Brindisys Consortium:

Sapienza Università di Roma • Dipartimento di Ingegneria

Informatica

• Dipartimento di Medicina e

Psichiatria

Università Magna Graecia di

Catanzaro

Associazione Crossing Dialogues

[29]

adottato da

IRCCS Fondazione Santa

Lucia di Roma

[30]

Thanks for your attention

[31]

NUOVE SLIDES DA INSERIRE NEL

CORPO DELLA PRESENTAZIONE: [32]

• The final prototype was re-

adapted according to the users’

feedback collected during the

preliminary evaluation.

• Indipendent home evaluation

What’s Next?

Automatized procedures for

configuration and calibration

Prototype features should

comply with user needs

Text predictor

Francesca Schettini

User-centered

design

BCIs to assist communication in ALS patients

[34]

Software applications

[35]

The prototype

• Hardware modules – Tablet

– BCI

– Domotic kit

• Software applications – Communication

• Speller, text to speech

• SMS

• …

– Domotic control • Appliances

• Remote control

• …

[36]

Experimentations with ALS users

[37] Conclusioni->