Neural Prosthetic Engineering

Today- Oct. 31st

• Questions? Projects?

• Review

– Speech Processing

• Recent updates

– By major commercial CI manufacturers

– New Technologies

1

Neural Prosthetic Engineering2

Review

Neural Prosthetic Engineering

Human Speech Generation

3

• Vocal fold generates sound that consists of fundamental

and harmonic frequencies (source of sound)

• Vocal tract modifies amplitudes of these frequency

components to make the sound distinguishable from

others (articulation by vocal tract).

Neural Prosthetic Engineering

Consonants/vowels/formants

4

Consonant and vowels

• Articulations at various places in vocal tract

• Roughly speaking,

• Vowels are lower frequency sounds

• Consonants are higher frequency sounds

• Formants are distinct frequency bands in the sound

spectrogram

• F1 and F2 may be used to represent vowels

• Consonants may need additional higher frequency

formants to be distinguished

• F0 is the fundamental frequency of the pitch

Neural Prosthetic Engineering

How we hear

5

• Temporal (Rate) code theory

• Place code theory

• Volley theory

• We use both Temporal(Rate) and Place Cues in hearing.

Neural Prosthetic Engineering

Hearing by Cochlear Implant

6

• Extracted features are used

• F0/F1/F2

• Analog waveforms are used

• Compressed Analog

• CIS uses extracted features and biphasic pulse

stimulation at a constant rate

Neural Prosthetic Engineering

What to improve

7

1.

2.

Neural Prosthetic Engineering

Development of Present and future Cochlear Implant Products and Performance

8

Neural Prosthetic Engineering

CI Milestones

• 1960 Single channel CI on three patients by William House

• 1976 A Multichannel CI on two patients by Graeme Clark

• 1977 Bilger report confirmed effectiveness of multichannel CI

• 1985 Nucleus 22 became the first Multichannel CI approved by FDA

• 1991 CIS speech processing strategy (Blake Wilson)

• 1997 Clarion (Advanced Bionics) device approved by FDA

• 2001 Medel device approved by FDA

• 2000’s EAS and Bilateral CI

Other milestone

• 1997 DBS (Activa manufactured by Medtronic) approved by FDA as a treatment for essential tremor and Parkinson's disease. Dystonia in 2003, and OCD in 2009.

• 2013 Argus II retinal implant (manufactured by Second Sight Medical Products) approved by FDA

9

F.G.Zeng et al., (IEEE Review in Biomedical Engineering, 2008)

Neural Prosthetic Engineering

History of the Cochlear Implant

• Pioneers

– Andre Djourno and Charles Eyries (in Paris, 1957)

• Eyries implants Djourno's induction coils in two patients

• Alternating current transmitted to the coil produces perception of sound

• Early Developments in the Western Hemisphere

– William House, John Doyle, James Doyle (Los Angeles, 1960)

• Effect electrical stimulation during stapes surgery

• Implant 3 patients with a single gold electrode

– Blair Simmons (Stanford University, 1964)

• Develops a six-electrode system using a percutaneous plug ( Ineraid)

– Robert Michelson (San Francisco, 1970)

• Implant 3 patients using a gold two-electrode system ( Advanced Bionics)

– William House (Los Angeles, 1972)

• First wearable cochlear implant device using a centering coil and magnet

• House/3M single channel cochlear implant (approved by the FDA in 1984)

Djourno

(Physiologist)

Eyries (ENT surgeon)

William House

(ENT Surgeon)

House/3M

single channel device

10

Neural Prosthetic Engineering

History of the Cochlear Implant

• Development of a Mutichannel Device (1970-80s)

– Single channel device Very Poor speech understanding

– Competition

• Michelson, Merzenich, Robert Schindler (UCSF) Advanced Bionics Corp.

• Hochmair (Vienna, Austria) Med-El GmbH.

• Graeme Clark (The University of Melbourne in Australia)

– Research supported by public donation (commenced 1967)

– First Multichannel Cochlear Implant Patient (1978) Cochlear Ltd.

• FDA Approved Multichannel CI Manufacturers

– Cochlear (Australia) – 1985

– Advanced Bionics (Austria) – 1996

– Med-El (Austria) – 2001 (1994 – European release)

• Lasker~DeBakey Clinical Medical Research Award (2013)

– Graeme M. Clark, Ingeborg Hochmair and Blake S. Wilson

• For the development of the modern cochlear implant - a device that bestows hearing to individuals with profound deafness.

Rod Saunders (First multi-

channel CI patient) and

Graeme Clark

11

Neural Prosthetic Engineering

Product History

12

1978 (by Michelson)

A.A.Eshraghi (The Anatomical Record, 2012)

1973 (by House,

single channel)

Neural Prosthetic Engineering

Product History

13

1985 (Cochlear Ltd. Nucleus,

FDA approval, multi-channel)

www.cochlear.com

1989 (Cochlear Ltd.

Mini speech processor)

Neural Prosthetic Engineering

Product History

14

1997 (Cochlear Ltd. Sprint,

Digital signal processor (DSP))

www.cochlear.com

2002 (Cochlear Ltd. Nucleus®

24 Contour Advance™, design

for structure preservation)

Neural Prosthetic Engineering

Product History

15www.cochlear.com

2008 (Cochlear Ltd. Hybrid,

hearing aid + cochlear implant) 2009 (Cochlear Ltd. Nucleus 5)

2015 (Cochlear Ltd. Nucleus 6,

smart speech processor)

Neural Prosthetic Engineering

Two Recent Advances

16

• Bilateral electrical stimulation

• Combined electric and acoustic

stimulation (EAS) for patients with

residual, low-frequency hearing

http://www.otosurgery.org/

Neural Prosthetic Engineering 17http://www.otosurgery.org/

Results with

bilateral

implants using

independent

processors,

Müller et al.,

2002

Sentences,Noise Right

right both left

Perc

ent

Corr

ect

0

20

40

60

80

100

Sentences,Noise Left

right both left

Monosyllabic Words,No Noise

right both left

32

86

96

107

118

148

151

257

298

Subjects

right both left

Perc

ent

Corr

ect

0

20

40

60

80

100

Implant(s)

right both left right both left

Neural Prosthetic Engineering 18http://www.otosurgery.org/

Combined Electric Acoustic Stimulation(EAS)

■ Combined EAS: Hearing Aid (HA) + Cochlear Implant (CI) on same ear

■ Many implant candidates Good low-frequency hearing but poor high frequency hearing

■ Low-frequency Acoustic Hearing using a HA

High-frequency Electrical Hearing using a CI

■ Good speech perception in noisy environments

■ Latest EAS technique

- Surgery: the round window approach (conventional method: cochleostomy)

- Electrode: flex, Short & thin electrode (half insertion)

Hybrid CI Device (HA+CI)

Ski-slope type SNHL

Sentence discrimination tests for a EAS patient

In quiet SNR 15 dB SNR 10 dB

W. Gstoettner et al. (Acta Oto-Laryngologica, 2004)

Hearing Aid (HA)

Neural Prosthetic Engineering

Performance of Cochlear Implant

19

B. S. Wilson and M. F. Dorman (IEEE Trans Biomed Eng, 2007)

Test

Star patient’s score

B. S. Wilson & M. Dorman (Hearing Research, 2008)

Percent correct scores for 55 CI users

Neural Prosthetic Engineering 20

Cochlear Implant Manufacturers

■ Big 3 companies

- Cochlear, Australia

- Med-El, Austria

- Advanced Bionics, USA

■ Others

- Oticon(Former Neurelec), France

- Nurotron, China

- Todoc, S. Korea

Neural Prosthetic Engineering 21

Cochlear Ltd.

■ Nucleus series

- Market leader

■ Dual microphones

- Directionality

■ Wireless control

- Interaction with TV and smartphone

■ Colorful designs

- Users satisfaction

■ Water-resistance

- Living convenience

■ Electric acoustic stimulation (EAS)

- High-performance

■ Data logging

- Rehabilitation

www.cochlear.com

Neural Prosthetic Engineering 22

Cochlear Ltd.

■ Thinnest cochlear implant among 3 major manufacturers

■ Advanced off stylet electrode array

- Perimodiolar electrode array

- Near zero insertion force

www.cochlear.com

Neural Prosthetic Engineering 23

Cochlear Ltd.

■ Smart speech processor (SmartSoundiQ)

- Integrated hybrid mode

- Wireless connectivity

- Data logging and analysis: information for next fitting

- Speech in noise, wind, quiet, and music modes

- Dual microphones

- Wind noise reduction (WNR)

www.cochlear.com

Neural Prosthetic Engineering 24

Med-El

■ Synchrony series

- Freely rotating and self-

aligning magnet

■ EAS

■ Water-resistance

■ Color options

■ Test in rainy environment

■ Wind noise reduction

■ 3.0 T MRI (with magnet)

- Other groups 1.5 T MRI

www.medel.com

Neural Prosthetic Engineering 25

Med-El

www.medel.com

Neural Prosthetic Engineering 26

Med-El

www.medel.com

■ Various types of electrode array

- Long electrode: for stimulation full

cochlea

- Short electrode: for EAS

■ Structure preservation

- Atraumatic insertion

- Reimplantation

Neural Prosthetic Engineering 27

Advanced Bionics

www.advancedbionics.com

■ Water-resistance

■ Wireless control

■ Colorful design

■ Dual microphones

■ Acquired by Sonova in

2009/ Co-working with

Phonak (hearing aid company)

■ EAS – FDA approval in Aug.

2015

Neural Prosthetic Engineering 28

Advanced Bionics

www.advancedbionics.com

■ HiFocus mid-scala electrode array

- Pre-curved electrode

Neural Prosthetic Engineering

New Technologies

29

Neural Prosthetic Engineering

Next Version of Cochlear Implant?

30

www.cochlear.com

www.cochlear.com

Candidates

1. Silicon-based

CI

2. Polymer-

based CI

3. Optical

stimulation

…

Neural Prosthetic Engineering

Silicon-Based Device

31

• Advantages of silicon-based neural implant

– Batch process

• High-yield

• Mass production

– MEMS technology

• Miniaturization

• High-density

• Highly functional device

• Integration

• Disadvantages of silicon-based neural implants

– Brittleness

– Stiffness

– Long-term reliability

Neural Prosthetic Engineering

Silicon-Based Device

32J.Wang and K.D.Wise, J.MEMS 2009.

J.Wang and K.D.Wise, J.MEMS 2008.

Neural Prosthetic Engineering

Polymer-Based Neural Implant

33

• Conventional neural implants

– Titanium package

• Highly hermetic

• MR image artifact problem

– Wire-based electrode array

• Manual fabrication, limited integration density

of contacts

• Polymer-based neural implants

– Thin & compact

– Simpler manufacturing process

• MEMS technology

– No MR image artifact

– Relatively low hermeticity than metals

10 mm

S. K. An et al., IEEE-TBME 2007.

Neural Prosthetic Engineering

Liquid Crystal Polymer (LCP)

34

• Two widely used polymers

– Polyimide, parylene-C

• Relatively high water absorption rate

• Lack of long-term reliable

• Liquid Crystal Polymer:A New Biomaterial for Implantable Devices

– Biocompatible

– Chemically inert and mechanically stable

– Flexible

– Compatible with MEMS technologies

– RF transparent

– Very Low water absorption rate (<0.04%)

– Fusion-bondable

– Deformable

LCP [1]

(Vecstar)

Polyimide [2]

(PI2525)

Parylene-C [3]

(GALXYL)

Melting Temp. (°C) 280~335 >400 290

Tensile Strength

(MPa)270~500 128 69

Young’s Modulus

(GPa)2~10 2.4 3.2

Water absorption (%)

< 0.04 2.8 0.06 ~ 0.6

Dielectric Constant

(@1MHz)2.9 3.3 2.95

[1] Kuraray group, http://www.kuraray.co.jp/en/[2] HDMicroSystems, http://hdmicrosystems.com/HDMicroSystems/en_US/[3] V&P Scientific, Inc., http://vp-scientific.com/parylene_properties.htm

Neural Prosthetic Engineering

LCP-Based Cochlear Implant

35

Neural Prosthetic Engineering

Electronics Design

36

• Simplified circuit block diagram

Contact pad sideCoil side

Fabricated electronic board

for cochlear implant

[1] S. K. An et al., IEEE TBME 2007.

Current stimulator chip

0.8 μm high voltage CMOS process (AMS)

CIS strategy

16 ch. mono- & 15 ch. bi-polar stimulation

Stimulation rate : 1,000 pps/channel

Duration : 0 μs ~ 56 μs

(7 levels, 8 μs step)

Amplitude : 0 μA ~ 1.8 mA

(255 levels, 7.3 μA step )

[2] J. Jeong et al., Conf Proc IEEE Eng Med Biol Soc 2011.

• Fabricated on Copper clad LCP film

– Planar type receiving coil integration for low-cost & miniaturized cochlear implant

Neural Prosthetic Engineering

• LCP-based Cochlear Implant System

Small & Light Package

37

[1] S. K. An et al., IEEE-TBME 2007.

Parameter Value

Carrier Frequency 2.5 MHz

Operating Distance 12 mm

Current Amplitude 8 μA~1.8 mA

Pulse Duration* 0~56 μs

Pulse Rate* Total 8,000 Hz

Parameter *LCP-based CI Titanium-based CI[1]

Package Size 20 x 28 mm2 65.7 x 33.3 mm2

Package

Max. thickness

(w/o magnet)

1.2 mm 8.2 mm

Weight

(w/o magnet)0.45 g 10.4 g

1. Physical Dimension (LCP vs. Ti-based CI)

2. Wireless & Stimulation* LCP-based

cochlear

implant

Metal-based

cochlear implant [1]

Neural Prosthetic Engineering

Flexible LCP-Based Cochlear Electrode Array

38

• LCP-based cochlear electrode array

– Multi-layered & tapered LCP film structure

– Sawtooth-like structure

– Silicone elastomer encapsulation• Minimal insertion force

• Atraumatic insertion

Cross Section of

16-Channel Interconnection

`

`

`

`

300

35

0

25

50

75

750

57

5

LCP Substrate

First LCP Cover

Second LCP Cover

Multi-layered Lead Wires

Via

Unit: μm

[1] K. S. Min et al., Otology & Neurotology 2014.[2] T.M. Gwon et al., Biomedical Microdevices 2015.

SitesE-Gun Deposited/Electroplated

Blind Via

Side Via Opening

Lead Wires38

Neural Prosthetic Engineering

Evaluation – Human Temporal Bone Insertion Study

39

• Safety validation of the electrode

– Human temporal bone insertion test

– Insertion depth measurement from CT scan image of the temporal bone

• No trauma at basal turn

• Dislocation into scala vestibule at middle turn in 630°insertion trial

• Max. insertion depth: ~630°

Cochlear electrode

[1] T.M. Gwon et al., Biomedical Microdevices 2015.

CT scan images (▲ )and

cross-section (▼) of the temporal bones

(round window approach:

insertion depth ~ 630°)

CT scan images (▲ )and

cross-section (▼) of the temporal bones

(cochleostomy approach:

insertion depth ~ 500°)

SV: Scala vestibuliST: Scala tympani

SV

SVSV

STST

ST

ST

STST

SV

SVSV

Neural Prosthetic Engineering

MRI Compatibility and In Vivo Functionality Testing

40

• Results of the 3.0 T MRI experiments

– Metal-based CI: Severe MR image artifact

– LCP-based CI: Little MR image artifact

50 mm

Axial Coronal

* *

3.0 T MR images of the head

[1] J. H. Kim et al., Clin Exp Otorhinolaryngol 2010.

• LCP-based Cochlear Implant

– EABR measurement

0 2 4 6 8-6

-4

-2

0

2

4

6

Vo

ltag

e (

V)

Time (ms)

Measured eABR

Wave V

Stimulus artifact

Package

Electrodes

Stimulation:

800 μA, 32 μs

(EABR: electrically evoked auditory

brainstem response)

Neural Prosthetic Engineering

Long-Term Reliability of LCP

41

• Accelerate Soak Test in 75℃ Phosphate Buffered

Saline (PBS) Solution

• Multi-Interdigitated Electrode (IDE) array

• Long-term leakage current measurements of polyimide, parylene-C, and LCP.

[1] SW Lee et al., IEEE-TBME 20110 15 30 45 60 75 90 105 120 135 15010

-13

10-12

10-11

10-10

10-9

10-8

10-7

10-6

10-5

10-4

10-3

10-2

Soak Time (days)

Cu

rren

t (A

)

Polyimide

Ch1

Ch2

Ch3

Ch4

Ch5

Ch6

0 15 30 45 60 75 90 105 120 135 15010

-13

10-12

10-11

10-10

10-9

10-8

10-7

10-6

10-5

10-4

10-3

10-2

Soak Time (days)

Cu

rren

t (A

)

Parylene-C

Ch1

Ch2

Ch3

Ch4

Ch5

Ch6

0 30 60 90 120 150 180 210 240 270 300 330 360 390 420 450 48010

-13

10-12

10-11

10-10

10-9

10-8

10-7

10-6

10-5

10-4

10-3

10-2

Soak Time (days)C

urr

en

t (A

)

LCP

Ch1

Ch2

Ch3

Ch4

Ch5

Ch6

Mean time to failure

> 1 year (380 days)

0 15 30 45 60 75 90 105 120 135 15010

-13

10-12

10-11

10-10

10-9

10-8

10-7

10-6

10-5

10-4

10-3

10-2

Soak Time (days)

Cu

rren

t (A

)

Polyimide

Ch1

Ch2

Ch3

Ch4

Ch5

Ch6

0 15 30 45 60 75 90 105 120 135 15010

-13

10-12

10-11

10-10

10-9

10-8

10-7

10-6

10-5

10-4

10-3

10-2

Soak Time (days)

Cu

rren

t (A

)

Parylene-C

Ch1

Ch2

Ch3

Ch4

Ch5

Ch6

0 30 60 90 120 150 180 210 240 270 300 330 360 390 420 450 48010

-13

10-12

10-11

10-10

10-9

10-8

10-7

10-6

10-5

10-4

10-3

10-2

Soak Time (days)

Cu

rren

t (A

)

LCP

Ch1

Ch2

Ch3

Ch4

Ch5

Ch6

0 15 30 45 60 75 90 105 120 135 15010

-13

10-12

10-11

10-10

10-9

10-8

10-7

10-6

10-5

10-4

10-3

10-2

Soak Time (days)

Cu

rren

t (A

)

Polyimide

Ch1

Ch2

Ch3

Ch4

Ch5

Ch6

0 15 30 45 60 75 90 105 120 135 15010

-13

10-12

10-11

10-10

10-9

10-8

10-7

10-6

10-5

10-4

10-3

10-2

Soak Time (days)

Cu

rren

t (A

)

Parylene-C

Ch1

Ch2

Ch3

Ch4

Ch5

Ch6

0 30 60 90 120 150 180 210 240 270 300 330 360 390 420 450 48010

-13

10-12

10-11

10-10

10-9

10-8

10-7

10-6

10-5

10-4

10-3

10-2

Soak Time (days)

Cu

rren

t (A

)

LCP

Ch1

Ch2

Ch3

Ch4

Ch5

Ch6

Mean time to failure

: 75 days

Mean time to failure

: 115 days

Polyimide Parylene-C LCP

Neural Prosthetic Engineering

Optical Stimulation

42

• Introducing Optics in cochlear implantation

• Multichannel Optical Cochlear Implantation

– Highly local stimulation

– Fine frequency resolution

– Increasing the number of effective channels

– Limited by line-of-sight property of light, and added complexity in optical instrumentation

Optical stimulation of auditory neurons: effects of acute and chronic deafening.

Richter CP, Bayon R, Izzo AD, Otting M, Suh E, Goyal S, Hotaling J, Walsh JT

Hearing research 2008 Aug; 242(1-2):42-51

http://openoptogeneticsblog.org/?p=682

Neural Prosthetic Engineering

Reference

• J. Jeong et al., Conf Proc IEEE Eng Med Biol Soc 2011.

• S. K. An et al., IEEE-TBME 2007.

• T.M. Gwon et al., Biomedical Microdevices 2015.

• K. S. Min et al., Otology & Neurotology 2014.

• A.A.Eshraghi (The Anatomical Record, 2012)

• B. S. Wilson & M. Dorman (Hearing Research, 2008)

• Ducci et al., Otology & Neurotology, 2010

• J.Wang and K.D.Wise, J.MEMS 2008.

• C.-P. Richter et al., Hearing Research, 2008

• L.E.Moreno et al., Hearing Research 2011.

• M. Jeschcke et al., Hearing Research 2015.

43