Elg5121 Project Prsentation Anisur Mohammad
-
Upload
chiemela-onunka -
Category
Documents
-
view
224 -
download
0
Transcript of Elg5121 Project Prsentation Anisur Mohammad
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
1/41
BRAIN COMPUTER INTERACTIONELG5121 (MULTIMEDIA COMMUNICATION)
Anisur Rahman , student ID: 3087689
Mohammad Upal Mahfuz, student ID: 5819545
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
2/41
Outline
Introduction to Brain Computer Interaction(BCI)
Early work (1970 2000)
Animal BCI
Success Stories
Human BCI
Invasive, Partially-invasive, Non-invasive BCIs Case Study: BCI in Health Care
Future of BCI: Technical Challenges
Ethical Considerations
Scientific American(Nov. 2008)
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
3/41
Introduction to BCI
A braincomputer interface (BCI) is a directcommunication pathway between a brain andan external device
sometimes called a direct neural interface or abrainmachine interface
BCIs are often aimed at:
assisting, augmenting or repairing human cognitive or
sensory-motor functions
Signal processing for BCI
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
4/41
Motivation:
Developing technologies for people withdisabilities:
Need to develop hardware and software to
disable people Assist blind people to visualize external images
Assist paralyzed people to operate external deviceswithout physical movement
Decode information stored on human brain (asmemory)
Decode information from brain to display
human thinking or dream on a screen
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
5/41
Early Work (1970-2000)
Animal BCI:
University of California started research in BCIs inthe 1970s (Schmidt et al 1978)
experimental BCI started on animals: monkey and rats Several laboratories managed to record signals from
monkey and rat cerebral cortex in order to operateBCIs
Scientist started to work on developing BCIalgorithm
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
6/41
Success Stories
Research in1970s found: monkeys can control the firing rates of individual and
multiple neurons Can generate appropriate patterns of neural activity
Algorithms were developed to reconstruct movements frommotor cortex neurons
In the 1980s: Research in JHU found a mathematical relationship
(based on a cosine function) between the electrical responses of single motor-cortex neurons in
rhesus macaque monkeys, and the direction that monkeys moved their arms
Established that dispersed groups of neurons in differentareas of the brain collectively controlled motor commands
(Schmidt et al 1978)
(Georgopoulos et al 1989)
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
7/41
Human BCI
Human BCI Types: Invasive
Partially invasive
Non Invasive
Invasive: Implanted directly into the grey matter of the brain during
neurosurgery
Partially-Invasive: Partially invasive BCI devices are implanted inside the
skull but rest outside the brain rather than within the greymatter
Non-Invasive: Implanted outside the skull
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
8/41
Invasive BCI
implanted directly into the grey matter
of the brain during neurosurgery
targeted repairing damaged sight
Providing new functionality to personswith paralysis
produce the highest quality signals
Vision Science:
Direct brain implants have been used to treat non-congenital (acquired) blindness
William Dobelle was one of the first scientists to comeup with a working brain interface to restore sight
The Human Brain(Lennon, J. 2010)
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
9/41
Invasive BCI (Continues)
William Dobelle (1st Generation):
First prototype was implanted into Jerry blinded in adulthood, in 1978
Single-array BCI containing 68 electrodes wasimplanted onto visual cortex
Succeeded in producing phosphenses thesensation of seeing light
The system included cameras mounted onglasses to send signals to the plant
Enable him to perform daily tasks unassisted
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
10/41
Invasive BCI (Continues)
William Dobelle (2nd Generation): More sophisticated implant enabling better
mapping of phosphenes into coherent vision
Jens Naumann blinded in adulthood (2002) wasable to drive slowly in the parking lot immediatelyafter the implant
Disadvantage:
Prone to Scalar tissue build up Causes signal to become weaker or even lost as
the body reacts to a foreign object
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
11/41
Partial Invasive BCI
Implanted inside the skull but rest outside thebrain
Produce better resolution signals than non-
invasive BCIs having a lower risk of formingscar-tissue in the brain than fully-invasiveBCIs.
Examples: Electrocorticography (ECoG)
Light Reactive Imaging BCI
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
12/41
Partial Invasive BCI (Continues)
Electrocorticography (ECoG)
Measures the electrical activity of the brain takenfrom beneath the skull
Electrodes are embedded in a thin plastic padthat is placed above the cortex, beneath the duramater.
First trialed in humans in 2004 by Eric Leuthardt and
Daniel MoranEnabled a teenage boy to play Space Invaders
using ECoG implant:
Controls are rapid, and requires minimal training
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
13/41
Partial Invasive BCI (Continues)
Light Reactive Imaging BCI Light Reactive Imaging BCI devices are still in the
realm of theory
involve implanting a laser inside the skull.
laser is trained on a single neuron and the neuron'sreflectance measured by a separate sensor. When the neuron fires, the laser light pattern and
wavelengths would change
Advantages of Partial Invasive BCI Better signal to noise ratio Higher spatial ratio
Better Frequency Range
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
14/41
Non-Invasive BCI
Recorded signal have been used to power muscleimplants and restore partial movement
Signals are weaken as skull dampens the signal
Although the waves are still detectable, it is hardto determine the area of the brain or the neuronthat created the signal
Examples:
Electroencephalography (EEG) Magnetoencephalography (MEG)
Magnetic resonance imaging (MRI)
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
15/41
Non-Invasive BCI (Continues)
Electroencephalography (EEG)
Most studied potential non-invasive interface
Fine temporal resolution
EEG in Mid1990s:
Niels Birbaumer (University of Tbingen in Germany) trainedseverely paralyzed people to self-regulate the slow corticalpotentials in their EEG
EEG signal was used as a binary signal to control acomputer cursor
Ten patients were able to move computer cursors bycontrolling their brainwaves
Slow required an hour to write 100 characters
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
16/41
Non-Invasive BCI (Continues)
EEG in 2000s:
Jessica Bayliss (University of Rochester) showed thatvolunteers wearing virtual reality helmets could controlelements in a virtual world using their P300 EEG readings
including turning lights on and off
bringing a mock-up car to a stop
(Ebrahimi et al. 2003)
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
17/41
Non-Invasive BCI (Continues)
Advantages of EEG
Ease of use
Portable
Low setup cost
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
18/41
Non-Invasive BCI (Continues)
Magnetoencephalography (MEG)
MEG is a technique for mapping brain activity byrecording magnetic fields produced by electrical currentsoccurring naturally in the brain
By using Arrays of SQUIDs (superconducting quantuminterference devices)
Application :
Localizing the regions affected by pathology, before surgical
removal
determining the function of various parts of the brain
(Ranganatha 2007)
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
19/41
Non-Invasive BCI (Continues)
Magnetic resonance imaging (MRI)
MRI is a technique used in radiology to visualize detailedinternal structures
Functional MRI or fMRI is a type of MRI scan that
measures the hemodynamic response (change in bloodflow) related to neural activity in the brain or spinal cord
fMRI allowed two users being scanned to play Pong in real-time
by altering their haemodynamic response or brain blood flow
Recent research in ATR (Advanced TelecommunicationsResearch, in Kyoto, Japan) on fMRI
allowed the scientists to reconstruct images directly from thebrain and display them on a computer.
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
20/41
Case-study: TOBI Project
TOBI (Tools for Brain Computer Interaction) Budget:12 millions
Duration: Nov. 2008 Dec. 2012
Coordinator: Ecole Polytechnique Fdrale de Lausanne
Selected list of partners:
T. U. Berlin, T.U. Graz, U. Heidelberg, U. of Glasgow and someothers.
Non-invasive type BCI applications
TOBI is a large European integrated project which will develop
practical technology for brain-computer interaction (BCI) that willimprove the quality of life of disabled people and the
effectiveness of rehabilitation.Ref. http://www.tobi-project.org/welcome-tobi
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
21/41
TOBI Project- (Motor Disability)
Four emerging application areas
Communications and control
Motor substitution
Entertainment
Motor recovery
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
22/41
1. Communication & Control
Deriving useful EEG control signals is high priority thaninteractions
As a result, BCI systems are often clumsy and awkward.
TOBI will provide suitable and comfortable devices to Suppress noise
Better dynamic properties
of control signals
Multimodal interfacesVisual
Audio
Haptic F/B(TOBI website, 2010)
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
23/41
2. Motor Substitution
High priority is to restore lost motor functions forthe disabled.
TOBI has worked on developing
neuroprostheses Case-1: Two operations will be developed
hand (grasping)
elbow (reaching)
assistive mobility Case-2:
User can mentally drive mobile robot.
(TOBI website, 2010)
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
24/41
3. Entertainment
Target group: Giving patients controls of ambientfeatures wall display, lighting and music
Non-verbal way of interaction.
New features under investigation
photo browsing
music navigation
Couple BCI interfaces to social networking
BCI-controlled games/interactive games
(TOBI website, 2010)
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
25/41
4. Motor Recovery
BCI enhances motor function recovery after acerebrovascular accident.
In addition to active and/or passive residual movements,
imaging movements can be a way to access the motorsystem in absence of any "real" movements
TOBI will introduce the mental practice of motor actionsvia BCI training, that might boost the clinical
rehabilitation strategies
This in turn would lead to a better
functional outcome.(TOBI website, 2010)
http://www.youtube.com/watch?v=gvR0kHm9fwo -
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
26/41
Future of BCI
Challenges have to overcome in hybrid BCI architectures
user-machine adaptation algorithms
BCI reliability analysis by exploiting users mentalstates
BCI performance analysis and confidencemeasures
Incorporate HCI to improve BCI
Development of novel EEG devices
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
27/41
Research Challenges (1/5)
Improve Non-invasive BCI based assistivetechnologies
Develop Hybrid BCI (hBCI)
Severe motor disabilities do not allow people tohave full benefit of current assistive products.
BCI
Enhancement
Assistive
Products (AP)
+
hBCI approach
The hBCI needs at least one BCI channel to work: otherchannel(s) can be AP input/biosignals or another BCI channel.
Millan et al. (2010)
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
28/41
Research Challenges (2/5)
Dynamic Adaptation Two-level adaptationprocess
First LevelSelf Adaptation
2nd levelDynamic Adaptation
The best interaction channelshould be dynamically chosen
The best EEG phenomena that eachuser better controls should bedynamically chosen:
P300 or SSVEP
This necessitates the development of novel trainingprotocols to determine the optimal EEG phenomenon foreach user, working on psychological factors in BCI.
Millan et al. (2010)
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
29/41
Research Challenges (3/5)
Improvement needs on current BCI outputs
Current BCI
has low bit rate,
noisy and has less reliability
Promising solution:
To adjust the dynamics of BCI, modern Human-ComputerInteraction (HCI) principles can be used
Alternative solution:
Use shared autonomy (or shared control) to shape thedynamics between user and brain-actuated device such thattasks are able to be performed as easily as possible.
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
30/41
Research Challenges (4/5)
BCI assisted technology can benefit from therecent research on the following-Recognition of users mental states
mental workload, stress, tiredness, attention level
Recognition of users cognitive processes awareness to errors made by the BCI
the dynamics and complexity of theinteraction will be simplified
This is another aspect of self-adaptation.
it will trigger OFF brain interaction and
move on to muscle-based interaction
High mentalworkload
Or stress levelOR
Example:
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
31/41
Research Challenges (5/5)
There are challenges to develop easy-to-use andaesthetic EEG equipment.
Issues to address: portability
aesthetic design
Aesthetic and engineering design should be merged.
One key issue for any practical BCI for disabledpeople.
Users dont want to look unusual socialacceptability
Example of advanced devices:
Dry electrodes instead of gel
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
32/41
Most Recent BCI News: (27 October2010)
http://spectrum.ieee.org/biomedical/bionics/braincomputer-interface-eavesdrops-on-a-daydream/
Scientists from Germany, Israel, Korea, the UnitedKingdom, and the United States have performed
combined experiments: Are able to monitor individual neurons in a human
brain associated with specific visual memories
Display visual memory onto a television monitor, andto replace with another
Scientists have found a neural mechanismequivalent to imagination and daydreaming the mental creation of images overrides visual input
http://spectrum.ieee.org/biomedical/bionics/braincomputer-interface-eavesdrops-on-a-daydream/http://spectrum.ieee.org/biomedical/bionics/braincomputer-interface-eavesdrops-on-a-daydream/http://spectrum.ieee.org/biomedical/bionics/braincomputer-interface-eavesdrops-on-a-daydream/http://spectrum.ieee.org/biomedical/bionics/braincomputer-interface-eavesdrops-on-a-daydream/http://spectrum.ieee.org/biomedical/bionics/braincomputer-interface-eavesdrops-on-a-daydream/http://spectrum.ieee.org/biomedical/bionics/braincomputer-interface-eavesdrops-on-a-daydream/http://spectrum.ieee.org/biomedical/bionics/braincomputer-interface-eavesdrops-on-a-daydream/http://spectrum.ieee.org/biomedical/bionics/braincomputer-interface-eavesdrops-on-a-daydream/http://spectrum.ieee.org/biomedical/bionics/braincomputer-interface-eavesdrops-on-a-daydream/http://spectrum.ieee.org/biomedical/bionics/braincomputer-interface-eavesdrops-on-a-daydream/http://spectrum.ieee.org/biomedical/bionics/braincomputer-interface-eavesdrops-on-a-daydream/http://spectrum.ieee.org/biomedical/bionics/braincomputer-interface-eavesdrops-on-a-daydream/http://spectrum.ieee.org/biomedical/bionics/braincomputer-interface-eavesdrops-on-a-daydream/http://spectrum.ieee.org/biomedical/bionics/braincomputer-interface-eavesdrops-on-a-daydream/ -
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
33/41
Most Recent BCI News: (Continues)
The researchers inserted microwires into thebrains of patients with severe epilepsy as part of apre-surgery evaluation to treat their seizures
The subjects were interviewed after the surgeryabout places theyd recently visited or movies ortelevision shows theyd recently seen:
images of the actors or visual landmarks the subjectshad described are shown on a display
Slides of the Eiffel Tower, for instance, or MichaelJacksonwho had recently died at the time of theexperimentwould appear on a screen.
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
34/41
Most Recent BCI News: (Continues)
Technical Challenges: about 5 million neurons in the brain encode for the
same concept, Cerf says.
Need to decode 5 millions neurons to get the completepicture
We are only able to read a limited number (for example:64 )
Complexity of Neuralnetwork (Lennon, J. 2010)
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
35/41
More news headings on BCI
Researchers Using Rat-Robot Hybrid toDesign Better Brain Machine Interfaces
Monkey Controls Advanced Robot Using
Its Mind
Monkey's Brain Can "Plug and Play" toControl Computer With Thought
IEEESpectrum(Oct. 2010)
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
36/41
Ethical Considerations
There has not been a vigorous debate about theethical implications of BCI
Important topics in neuroethical debate are:
Risk/benefit analysis
Obtaining informed consent
Possible side effects and consequences in life stylesfor the patient relatives
Professor Michael Crutcher expressed concernabout BCI specially for ear and eye implants: If only the rich can afford it, it puts everyone else at a
disadvantage
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
37/41
Ethical Considerations (Continues)
Clausen concluded in 2009:
BCIs pose ethical challenges, but these areconceptually similar to those that bioethicists haveaddressed for other realms of therapy
Recently more effort is made inside the BCIcommunity to initiate the development of ethicalguidelines for BCI research, development anddissemination
Requirements for the social acceptance: Sound ethical guidelines
Appropriately moderated enthusiasm in mediacoverage
Education about BCI systems
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
38/41
Conclusions
Brain Computer Interaction is:
Send outside signal to brain neuron
vision signal for blind person
Read the neuron signal To operate external devices without physical
intervention
To read memory or display user imagination
Significant progress in last ten years Technical challenges need to be overcome
Significant potential uses in medical science to
assist physically disabled persons
-
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
39/41
References
Ebrahimi, T.; Vesin, J., Garcia, G. Brain-Computer Interface in MultimediaCommunication. IEEE Signal Processing Magazine: January 2003.
TOBI Project by EU, Website: http://www.tobi-project.org/
J.d.R. Millan, R. Rupp, G.R. Muller-Putz, R. Murray-Smith4, C. Giugliemma, M.Tangermann, C. Vidaurre, F. Cincotti, A. Kubler, R. Leeb, C. Neuper, K.R. Muller, D.
Mattia (2010) "Combining Brain-Computer Interfaces and Assistive Technologies: State-of-the-Art and Challenges," Frontiers in Neuroscience, vol 4, August, 2010,doi:10.3389/fnins.2010.00161.
B.Z. Allison, C. Brunner, V. Kaiser, G.R. Muller-Putz, C. Neuper, and G. Pfurtscheller.Toward a hybrid brain-computer interface based on imagined movement and visualattention. J. Neural Eng., 7, 2010.
B.Z. Allison, E.W. Wolpaw, and J.R. Wolpaw. Brain-computer interface systems:
Progress and prospects. Expert Rev. Med. Devices, 4:463474, 2007. G. Alon, A.F. Levitt, and P.A. McCarthy. Functional electrical stimulation enhancement
of upper extremity functional recovery during stroke rehabilitation: A pilot study.Neurorehabil. Neural Repair, 21:207215, 2007.
K.D. Anderson. Targeting recovery: Priorities of the spinal cord-injured population.Neurotrauma, 21:13711383, 2004.
K.K. Ang, C. Guan, K.S.G. Chua, B.T. Ang, C. Kuah, C. Wang, K.S. Phua, Z.Y. Chin,
and H. Zhang. A clinical study of motor imagery-based brain-computer interface forupper limb robotic rehabilitation. In Proc. 31th A. Int. Conf. IEEE Eng. Med. Biol. Soc.,
http://www.tobi-project.org/http://www.tobi-project.org/http://www.tobi-project.org/http://www.tobi-project.org/ -
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
40/41
References (Continues)
F. Babiloni, F. Cincotti, L. Lazzarini, J.d.R. Millan, J. Mourino, M. Varsta, J. Heikkonen,L. Bianchi, and M.G. Marciani. Linear classification of low-resolution EEG patternsproduced by imagined hand movements. IEEE Trans. Neural. Syst. Rehabil. Eng.,8:186188, 2000.
J.D. Bayliss. Use of the evoked potential P3 component for control in a virtualapartment. IEEE Trans. Neural. Syst. Rehabil. Eng., 11:113116, 2003.
Schmidt, EM; McIntosh, JS; Durelli, L; Bak, MJ (1978). "Fine control of operantly
conditioned firing patterns of cortical neurons.". Experimental neurology 61 (2): 34969.doi:10.1016/0014-4886(78)90252-2. PMID 101388.
Georgopoulos, A.; Lurito, J.; Petrides, M; Schwartz, A.; Massey, J. (1989). "Mentalrotation of the neuronal population vector". Science 243 (4888): 234.doi:10.1126/science.2911737. PMID 2911737.
Ranganatha Sitaram,Andrea Caria,Ralf Veit,Tilman Gaber,Giuseppina Rota,AndreaKuebler and Niels Birbaumer(2007) "FMRI BrainComputer Interface: A Tool for
Neuroscientific Research and Treatment Kennedy, PR; Bakay, RA (1998). "Restoration of neural output from a paralyzed patient
by a direct brain connection.". Neuroreport9 (8): 170711.
Lennon, JF, The human Brain , http://www.jflennon.com/ResearchJFLTech.htm, Nov.2010
Cover Image source : Medic Magic website: http://medicmagic.net/wp-content/uploads/2010/03/human-brain.jpg
http://www.jflennon.com/ResearchJFLTech.htmhttp://www.jflennon.com/ResearchJFLTech.htm -
8/3/2019 Elg5121 Project Prsentation Anisur Mohammad
41/41
Questions & Answers