Lunch Talk on Brain-Computer Interfacing Artificial Intelligence, University of Groningen

20
Lunch Talk on Brain-Computer Interfacing Artificial Intelligence, University of Groningen Pieter Laurens Baljon December 14, 2006 12:30-13:00

description

Lunch Talk on Brain-Computer Interfacing Artificial Intelligence, University of Groningen. Pieter Laurens Baljon December 14, 2006 12:30-13:00. Overview. What is a BCI? EEG-based BCI Preprocess, extract features, classify Functional correlates of features Our BCI Setup - PowerPoint PPT Presentation

Transcript of Lunch Talk on Brain-Computer Interfacing Artificial Intelligence, University of Groningen

Page 1: Lunch Talk on Brain-Computer Interfacing Artificial Intelligence, University of Groningen

Lunch Talk onBrain-Computer Interfacing Artificial Intelligence, University of Groningen

Pieter Laurens BaljonDecember 14, 2006

12:30-13:00

Page 2: Lunch Talk on Brain-Computer Interfacing Artificial Intelligence, University of Groningen

Overview

• What is a BCI?• EEG-based BCI

– Preprocess, extract features, classify– Functional correlates of features

• Our BCI Setup– Online, offline and simulation

• Clinical- or theoretical relevance (or both?)

Page 3: Lunch Talk on Brain-Computer Interfacing Artificial Intelligence, University of Groningen

What is a BCI

• Interface between the brain and computer– Normally: hands and arms, voice– Could be deficient through stroke or ALS

• A BCI:– “must not depend on the brain’s normal output

pathways of peripheral nerves and muscles”1

• Prosthesis connected to nerveendings is not a BCI

Page 4: Lunch Talk on Brain-Computer Interfacing Artificial Intelligence, University of Groningen

What is a BCI

Adapted from Carmena et al. 2003, in PLoS Biology 1(2)

Page 5: Lunch Talk on Brain-Computer Interfacing Artificial Intelligence, University of Groningen

What is a BCI(Spelling example)

YouTube: http://www.youtube.com/watch?v=yhR076duc8M

Page 6: Lunch Talk on Brain-Computer Interfacing Artificial Intelligence, University of Groningen

What is a BCI(Pong example)

YouTube: http://www.youtube.com/watch?v=qCSSBEXBCbY

Page 7: Lunch Talk on Brain-Computer Interfacing Artificial Intelligence, University of Groningen

What is a BCI

• Brain signal can come from – Invasive electrodes– Non-invasive measurements

• EEG, fMRI, etc.

• Underlying assumption– Intentions have discernible

counterpart in brain signal

Page 8: Lunch Talk on Brain-Computer Interfacing Artificial Intelligence, University of Groningen

EEG-based BCI

• Sub fields of EEG-based BCI:– Signal processing on the EEG– Cognitive task for the subject (psychology)– Designing computer application (HMS)

• Typical pattern-recognition pipeline1. Preprocessing2. Feature extraction3. Classification (not considered here)

Page 9: Lunch Talk on Brain-Computer Interfacing Artificial Intelligence, University of Groningen

The EEG: Preprocessing• Preprocessing

– Recombining electrodes can improve SNR

1. Spatial Filtering– Laplacian filters

• Subtract surrounding electrodes• Vary distance to surrounding electrodes

2. Statistical recombination– Independent-Component Analysis– Common-Spatial Patterns

Page 10: Lunch Talk on Brain-Computer Interfacing Artificial Intelligence, University of Groningen

The EEG: Feature Extraction

• Signal is recorded in 2 or more conditions– Features should correlate with condition.– They must be detectable in single trial

• Two principal approaches:– Brute force machine learning

• Combine all imaginable features– Features with a functional correlate

• Potential shifts: Bereitschafts potential• Rhythms: Alpha, mu, beta, etc.• P300: Particular waveform

Page 11: Lunch Talk on Brain-Computer Interfacing Artificial Intelligence, University of Groningen

The EEG: Sensorimotor Rhythm (SMR)

• Function of periodical brain activity• The predominance of a function

– Expressed by spectral power• Many rhythms are ‘idling-rhythms’.

– Alpha rhythm over occipetal lobe (~10Hz)– Mu rhythm over motor cortex (~10 Hz)

Page 12: Lunch Talk on Brain-Computer Interfacing Artificial Intelligence, University of Groningen

The EEG: Sensorimotor Rhythm (SMR)

University college, London & TU Graz

VR application, controlling a wheelchair

Page 13: Lunch Talk on Brain-Computer Interfacing Artificial Intelligence, University of Groningen

The EEG: (SCP) & P300

• Slow cortical potentials:– Low-pass filtered signal– E.g. Bereitschafts potential

• Ability to self regulate– Also used for neurofeedback– To treat ADHD

• P300 is ‘evoked potential’– Less training– Indicate attended target

Tetraplegic operating a speller application

Outline of a P300 speller application.

When target row/column is highlighted, it evokes a P300.

Page 14: Lunch Talk on Brain-Computer Interfacing Artificial Intelligence, University of Groningen

Training

• Subject: biofeedback– learning to control physiological ‘parameters’– E.g. Heartrate, EEG-components

• System: any Pattern Recognition method– BCI competition: Different sorts of data

• Complexity of classifier– Reduces ‘meaningfulnes’ of transformation?

Page 15: Lunch Talk on Brain-Computer Interfacing Artificial Intelligence, University of Groningen

Training

• No ‘continuous mutual learning’.– Mostly epoch based– Update the system in between sessions– Danger of oscillations in feedback loop.

• There is no between-subjects design yet– Due to large inter-subject variability (?)– Could elucidate

• Effect of non-linear vs. linear feedback on EEG

Page 16: Lunch Talk on Brain-Computer Interfacing Artificial Intelligence, University of Groningen

Our BCI Setup (online)

• General purpose framework: BCI2000• Modular setup for

– Amplifier driver– Signal processing– Application

• Open-source Borland C++• Large community: over 100 labs• Initial problems running BCI experiments

Page 17: Lunch Talk on Brain-Computer Interfacing Artificial Intelligence, University of Groningen

Our BCI Setup (offline)

• Offline analysis in MatLab– Framework to test pattern recognition

• Setup similar to BCI2000• Simple addition of new features, thus far:

– Preprocessing: ICA, CSP– Features: Spectral power, Hjorth– Classification: HMM, kNN, LDA, SVM

Page 18: Lunch Talk on Brain-Computer Interfacing Artificial Intelligence, University of Groningen

Our BCI Setup (simulation)

• Addition to BCI2000.• Signal source can model SMR changes• Collaboration with developers of BCI2000

• Simulation in order to:– reverse engineer inner workings of BCI2000– pretest settings for adaptivity

Page 19: Lunch Talk on Brain-Computer Interfacing Artificial Intelligence, University of Groningen

Clinical- & Theoretical relevance

• Most of the research is on healthy subjects• Clinical research poses problems:

– Proper operation requires extensive training– ALS Patients are only to learn control if they

had it before the injury.– Small body of potential subjects

• Birbaumer reports a“significant increase in quality of life”They normally cannot communicate at all.

Page 20: Lunch Talk on Brain-Computer Interfacing Artificial Intelligence, University of Groningen

References• [1] J. R. Wolpaw, N. Birbaumer, W. J. Heetderks, D. J. McFarland, P. H. Peckham, G. Schalk, E.

Donchin, L. A. Quatrano, C. J. Robinson and T. M. Vaughan, “Brain-computer interface technology: A review of the first international meeting,” IEEE Transactions on rehabilitation engineering, vol. 8, pp. 164–173, 2000.

• Slide 1. Cover of the book Mathilda, about a telekinetic girl. Illustration: Quentin Blake• Slide 3. PL Baljon (author) operating a BCI. Private collection. Photo: Mark Span.• Slide 5, 6. Movies from youtube, filmed at CeBIT from Fraunhofer BCI, Berlin BCI.• Slide 7. “Hans-Peter Salzmann gelang es 1996 erst nach monatelangem Training mit dem Thought

Translation Device, den Cursor zu steuern.” Source : University of Tübingen• Slide 12. “Controlling a wheelchair in a VR application” Source: University college, London & TU

Graz.• Slide 13. Tetraplegic operating a speller device: Source: NIBIB,

http://www.nibib.nih.gov/NewsEvents/Calendar/ExhibitBoothLetter grid is taken from the BCI2000 manual. It is an excerpt from a trial with a P300 speller application.