Contributions to Bionic Eye Research
-
Upload
darien-pardinas-diaz -
Category
Engineering
-
view
158 -
download
4
Transcript of Contributions to Bionic Eye Research
Contributions to Bionic Eye ResearchDARIEN PARDINAS DIAZ
2016/06/03
OutlineOverview of the Bionic Eye project
◦ First prototype and operation principle◦ Neurostimulator and electrode configurations◦ Existing psychophysics experiments
First prototype – Contributions◦ From single electrode stimulation to image recognition psychophysics◦ Pulse dose quantification and patient safety◦ Outcomes of the first trials
Second prototype – Contributions◦ Development of a Portable Vision Processor◦ Stimulation strategies and real-time pulse dose quantification algorithms◦ Stimuli validation framework◦ Psychophysics application redesigned
Summary
2
First prototype“Device intended to restore functional vision”
Joint effort between the Bionics Institute and CERA produced the first Australian prototype of a bionic eye tested in humans
3
Operation principle
IR Image: Centre for Eye ResearchAustralia (CERA)
4
NeuroBi – The neurostimulator
Common GroundMonopolar
Pseudo-Hex(Quasi-Monopolar)
Hexagonal
Electrode buffer (active/return) Pulse buffer (PW, PPD, IPG, etc.) Sequence buffer (µA, n)
5
Psychophysics Experiments Determine stimulation
thresholds Dynamic range Phosphene shape and
size characterisation SOP - Impedance
measurement
Characterise safe and effective electrical stimulation levels
Evaluate implant stability over the course of the clinical trials
6
EyeSee app
First prototype – Contributions
There were many experiments in place to study single-electrode stimulation, but not much work had been done to study the image recognition capabilities of the prototype
7
From single electrode stimulation to image recognition psychophysics
1. Integration of a vision processing library developed by NICTA CRL
Implemented with OpenCV
Provided different ways to sample the input image – NN, Lanczos2
C#/WPF -> C++/CLR wrapper to convert WPF image data into OpenCV and call the library
8
From single electrode stimulation to image recognition psychophysics
2. Development of a real-time stimulation algorithm capable of transforming phosphene intensities into pulse sequences
Other experiments used ImPreSS (latency)
NeuroBi had a time overhead to load the stimulation sequence
Implemented algorithm to convert phosphene intensities into interleaved train of pulses and upload with minimum overhead
Ability to play input movies at 10 FPS
9
From single electrode stimulation to image recognition psychophysics
3. Generation of synthetic static images and GIF movies of shapes of different sizes calculated in FOV units
Moving gratings, bars, circles, etc. multiple directions and speeds (°/sec)
6°width moving gratings speed = 4°/sec
3°width moving barspeed = 4°/sec
10
From single electrode stimulation to image recognition psychophysics
11
Eye gaze affects percept location
Data collation, transformation and visualisation
12
Pulse dose quantification and patient safety
NeuroBi had the ability to acquire and send voltage waveform, but this data was never recorded during patient sessions
Challenges:
-16bits resolution/100kHz sample rate for long periods of time
-Raw data didn’t specify from which electrode configuration
Proposed solution:
-Implemented an efficient data logging module that allowed recording all the stimulation activity for detailed analysis of the patient session
Allowed researchers to:
-Verify that established safety limits of pulse charge were observed and that stimulation pulses were delivered as intended
-Explore new dose limits by stimulating the eye while running a live retinal OCT
-Study impedance dependency on the electrode configuration, pulse parameters, stimulation period
13
Pulse dose quantification and patient safety
14
Automated report generation of patient sessions for analysis
Outcomes of the first trialsProof of concept that a suprachoroidal visual prosthesis can provide basic vision
Larger electrode surface area are more effective – less stimulation power
Simple monopolar proved to be the most effective electrode configuration
Percutaneous connector is not suitable for a take-home scenario
Continuous stimulation caused the phosphenes to fade after a few seconds
Phosphene location changes with eye gaze angle – eye tracker can help
Phosphenes overlap significantly with one another – brain plasticity can help
Neurostimulator needs to provide better real-time stimulation capabilities
Challenges to overcome
15
Second prototype – 44ChFIOperation Modes: Take-home system Stimulator for psychophysics
research
Pulse Streaming
PulseGeneration PIC
CI
CI
SPI
RF
Vision Processing
ImageCapture
Video
framesPhosphene
intensities
Pulse
train
Pulse
stream
ARM based processorRF
Implanted
EyeSeeApp
TCP
16
Second prototype – test bed
Load Board – LEDs version
Implant Emulators
PVP Unit
17
Stimulation strategies to mitigate phosphene fading effect“Continuous electrical stimulation causes the retina to desensitize and in humans it leads to a reduction in the brightness of phosphenes and also increases threshold” - Davuluri (2014)
• Phase width
• Interphase gap
• Anodic vs. cathodic first
• Phase amplitude attenuation
• Pulse period
18
Stimulation strategies design
• Suitable for both basic psychophysics and real-time video feed
• Producer-consumer paradigm
• Run efficiently on embedded hardware
class Stimulation Strategy
StimulusConfiguration
+ PhospheneDefinitions
+ PulseStrategies
PhospheneDefinition
+ ElectrodeConfigurations
+ MaxRating
+ MinRating
+ PulseStrategy
PulseStrategy
+ AnodicFirst
+ AttenuationFactor
+ InterPhaseGap
+ PhaseWidth
+ PulsePeriod
ElectrodeConfiguration
+ ActiveElectrodes
+ GammaFactor
+ MaximumCurrent
+ MinimumCurrent
+ ReturnElectrodes
Defines what electrodes
are involved in the
stimulation and the
range of currents that
could be used for
stimulation
Pulse parameters are
potentially different for
each pulse to be
generated
1..2
1
11
1..*
1
1..*
19
Proposed stimulation algorithmDesigned a generic algorithm capable of producing a pulse stream from any possible variation of pulse parameters (~3kpps)
Implemented as a priority queue of pulses with variable time slot
Algorithm very efficient with little memory allocation
act Stimulus Generation Algorithm
Initialise Phosphene
Requests
stimulation time
consumed?
Retriev e
first
phosphene
request
time_to_stimulate <= 0
Try
Generate
Pulse
A pulse can only be generated if
for all electrode configurations the
charge and energy dose are
below the allowed limits
pulse was
generated?
Append to
pulse train
Reschedule
Phosphene
Generat Null
frames to cov er
time_to_stimulate
Done with this
stimulation
request
Commit
pulse
Phosphene requests are
kept in a priority queue
sorted by their time to
generate the next pulse
[no]
[no]
[yes]
[yes]
[no]
[yes]
20
A real time dose tracker to ensure patient safety
Multi-window size dose trackers running over the same pre-allocated circular buffer
Efficient non-blocking binary pulse stream logger
Charge
Energy
Time
Charge
Energy
Time
Charge
Energy
Time
Charge
Energy
Time
Charge
Energy
Time
Now33ms ago1s ago
𝑄,𝐸 ≤ 𝐿𝑖𝑚𝑖𝑡33𝑚𝑠
𝑄,𝐸 ≤ 𝐿𝑖𝑚𝑖𝑡1𝑠
Charge
Energy
Time
Charge
Energy
Time
?
21
Automated validation of stimuli delivered
• 32 Independent Channels• 16 bits resolution• Fs = 250kHz
Integration test to stimulate and record the pulse waveforms
Extraction of pulse parameter from recorded signal (resolution error ≤ 4µs)
Algorithm to match extracted pulses with generated pulse train
Purpose: Validate pulse streaming from PVP to the electrode array
22
Psychophysics application redesigned No automated tests in place
Anti-patterns (God classes, code duplicates, no abstractions, etc.)
Highly coupled to the stimulation principle of NeuroBi and to predefined electrode configuration modes
Designed and implemented a device agnostic application based on generic stimulation conceptsIncreased application maintainability by reusing common functionality across experiment procedures – e.g. stimulus output visualisation, etc.Implementing unit tests for critical components, etc.
23
Stimulus Configuration Editor
24
Reusable infrastructure
25
Stimulator output visualisation
SummaryVery exciting project and great team of researchers
Very dynamic project – there is not much that can be anticipated
Got to apply a broad range of technical skills
Practiced to work effectively with legacy code
26
Thank you!
Questions?
27