Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 1 / 35

Simulating Hearing LossDelft University of Technology

Leo Koop

April 9, 2015

Outline1 Introductions and Motivation

IntroductionMotivationThe Ear and Cochlea

2 Modeling the CochleaThe General ModelLinear and Non Linear Models

3 Simulating Hearing LossApproachThe Linear Case; A Sound FilterResults

4 Testing and OutlookTestingOutlook

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 2 / 35

Next Subsection1 Introductions and Motivation

IntroductionMotivationThe Ear and Cochlea

2 Modeling the CochleaThe General ModelLinear and Non Linear Models

3 Simulating Hearing LossApproachThe Linear Case; A Sound FilterResults

4 Testing and OutlookTestingOutlook

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 3 / 35

Simulating Hearing Loss

• Applied Mathematics Department at TU Delft

• Supervisor: Kees Vuik

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 4 / 35

Simulating Hearing Loss

• An incas3 project

• Supervisor: Peter van Hengel

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 5 / 35

Next Subsection1 Introductions and Motivation

IntroductionMotivationThe Ear and Cochlea

2 Modeling the CochleaThe General ModelLinear and Non Linear Models

3 Simulating Hearing LossApproachThe Linear Case; A Sound FilterResults

4 Testing and OutlookTestingOutlook

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 6 / 35

Motivation - Hearing Loss, Becoming morePrevalent

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 7 / 35

Motivation - Uses of a Hearing Loss Simulator

• Education

• Hearing loss prevention

• Simplifying hearing aid development and testing

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 8 / 35

Motivation - Uses of a Hearing Loss Simulator

• Education

• Hearing loss prevention

• Simplifying hearing aid development and testing

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 8 / 35

Motivation - Uses of a Hearing Loss Simulator

• Education

• Hearing loss prevention

• Simplifying hearing aid development and testing

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 8 / 35

Next Subsection1 Introductions and Motivation

IntroductionMotivationThe Ear and Cochlea

2 Modeling the CochleaThe General ModelLinear and Non Linear Models

3 Simulating Hearing LossApproachThe Linear Case; A Sound FilterResults

4 Testing and OutlookTestingOutlook

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 9 / 35

The Ear

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 10 / 35

The Cochlea, a Cross Section

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 11 / 35

The Cochlea

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 12 / 35

Next Subsection1 Introductions and Motivation

IntroductionMotivationThe Ear and Cochlea

2 Modeling the CochleaThe General ModelLinear and Non Linear Models

3 Simulating Hearing LossApproachThe Linear Case; A Sound FilterResults

4 Testing and OutlookTestingOutlook

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 13 / 35

Modeling the Cochlea

The Main Equation

∂2p

∂x2(x , t) − 2ρ∂2y

h∂t2(x , t) = 0, 0 ≤ x ≤ L, t ≥ 0 (1)

• y(x , t): The excitation of the oscillator

• p(x , t): Pressure on the cochlear partition

• ρ: Density of the cochlear fluid

• h: Height of the scala

• L: Length of the cochlea

• t: Time

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 14 / 35

Modeling the Cochlea

The Main Equation

∂2p

∂x2(x , t) − 2ρ∂2y

h∂t2(x , t) = 0, 0 ≤ x ≤ L, t ≥ 0 (1)

• y(x , t): The excitation of the oscillator

• p(x , t): Pressure on the cochlear partition

• ρ: Density of the cochlear fluid

• h: Height of the scala

• L: Length of the cochlea

• t: Time

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 14 / 35

Modeling the Cochlea

The Main Equation

∂2p

∂x2(x , t) − 2ρ∂2y

h∂t2(x , t) = 0, 0 ≤ x ≤ L, t ≥ 0 (1)

• y(x , t): The excitation of the oscillator

• p(x , t): Pressure on the cochlear partition

• ρ: Density of the cochlear fluid

• h: Height of the scala

• L: Length of the cochlea

• t: Time

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 14 / 35

Modeling the Cochlea

The Main Equation

∂2p

∂x2(x , t) − 2ρ∂2y

h∂t2(x , t) = 0, 0 ≤ x ≤ L, t ≥ 0 (1)

• y(x , t): The excitation of the oscillator

• p(x , t): Pressure on the cochlear partition

• ρ: Density of the cochlear fluid

• h: Height of the scala

• L: Length of the cochlea

• t: Time

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 14 / 35

Modeling the Cochlea

The Main Equation

∂2p

∂x2(x , t) − 2ρ∂2y

h∂t2(x , t) = 0, 0 ≤ x ≤ L, t ≥ 0 (1)

• y(x , t): The excitation of the oscillator

• p(x , t): Pressure on the cochlear partition

• ρ: Density of the cochlear fluid

• h: Height of the scala

• L: Length of the cochlea

• t: Time

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 14 / 35

Modeling the Cochlea

The pressure term

p(x , t) = my(x , t) + d(x)y(x , t) + s(x)y(x , t) (2)

• m: Mass of the membrane

• d(x): Position dependent damping

• s(x): Position dependent stiffness

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 15 / 35

Modeling the Cochlea

The pressure term

p(x , t) = my(x , t) + d(x)y(x , t) + s(x)y(x , t) (2)

• m: Mass of the membrane

• d(x): Position dependent damping

• s(x): Position dependent stiffness

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 15 / 35

Simplifying the equations

We introduce g(x , t) as follows:

g(x , t) = d(x)y(x , t) + s(x)y(x , t) (3)

Then rewrite my(x , t) as:

my(x , t) = p(x , t) − g(x , t) (4)

This allows us to then rewrite Equation (1) as:

∂2p

∂x2(x , t) − κp(x , t) = −κg(x , t) (5)

Where κ = 2ρhm .

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 16 / 35

Simplifying the equations

Equation 5 can be further reduced to 2 first degree ODEs byintroducing v as follows:

∂y

∂t(x , t) = v(x , t) (6)

∂v

∂t(x , t) =

p(x , t) − g(x , t)

m(7)

Which can then be solved with a numerical integrator, say ClassicalRunge Kutta 4

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 17 / 35

Next Subsection1 Introductions and Motivation

IntroductionMotivationThe Ear and Cochlea

2 Modeling the CochleaThe General ModelLinear and Non Linear Models

3 Simulating Hearing LossApproachThe Linear Case; A Sound FilterResults

4 Testing and OutlookTestingOutlook

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 18 / 35

Damping and Stiffness Terms

The pressure term

p(x , t) = my(x , t) + d(x)y(x , t) + s(x)y(x , t)

Linear damping term

d(x) = ε√m s(x)

Non-linear damping term

d(x , y) =

{(1 − γ)(δsat − δneg )

[1 − 1

1 + e(Λ−α)/µα

]+ γ(δsat − δneg )

[1 − 1

1 + eΛ−β/µβ

]+ δneg

}√m s(x)

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 19 / 35

Damping and Stiffness Terms

The pressure term

p(x , t) = my(x , t) + d(x)y(x , t) + s(x)y(x , t)

Linear damping term

d(x) = ε√m s(x)

Non-linear damping term

d(x , y) =

{(1 − γ)(δsat − δneg )

[1 − 1

1 + e(Λ−α)/µα

]+ γ(δsat − δneg )

[1 − 1

1 + eΛ−β/µβ

]+ δneg

}√m s(x)

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 19 / 35

Damping and Stiffness Terms

The pressure term

p(x , t) = my(x , t) + d(x)y(x , t) + s(x)y(x , t)

Linear damping term

d(x) = ε√m s(x)

Non-linear damping term

d(x , y) =

{(1 − γ)(δsat − δneg )

[1 − 1

1 + e(Λ−α)/µα

]+ γ(δsat − δneg )

[1 − 1

1 + eΛ−β/µβ

]+ δneg

}√m s(x)

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 19 / 35

Damping and Stiffness Terms

The pressure term

p(x , t) = my(x , t) + d(x)y(x , t) + s(x)y(x , t)

Linear stiffness term

s(x) = s0e−λx

Non-linear case; an additional delayed feedback stiffness term

c(x , y) =

{(1 − γ)(σzweig )

[1

1 + e(Λ−α)/µα

]+ γ(σzweig )

[1

1 + eΛ−β/µβ

]}(8)

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 20 / 35

Damping and Stiffness Terms

The pressure term

p(x , t) = my(x , t) + d(x)y(x , t) + s(x)y(x , t)

Linear stiffness term

s(x) = s0e−λx

Non-linear case; an additional delayed feedback stiffness term

c(x , y) =

{(1 − γ)(σzweig )

[1

1 + e(Λ−α)/µα

]+ γ(σzweig )

[1

1 + eΛ−β/µβ

]}(8)

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 20 / 35

Damping and Stiffness Terms

The pressure term

p(x , t) = my(x , t) + d(x)y(x , t) + s(x)y(x , t)

Linear stiffness term

s(x) = s0e−λx

Non-linear case; an additional delayed feedback stiffness term

c(x , y) =

{(1 − γ)(σzweig )

[1

1 + e(Λ−α)/µα

]+ γ(σzweig )

[1

1 + eΛ−β/µβ

]}(8)

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 20 / 35

A Sample Model Output

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−0.2

−0.15

−0.1

−0.05

0

0.05

0.1

0.15

0.2progress 50000.00 steps, t−tilde = 10.00

Oscillators in % of Cochlea Length

Oscill

ato

r D

ispla

cem

ent / V

elo

city (

nm

/ n

m/m

s)

osc. velocity

osc. displacement

Figure: Modeling the response of the basilar membrane to a tone sound

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 21 / 35

Next Subsection1 Introductions and Motivation

IntroductionMotivationThe Ear and Cochlea

2 Modeling the CochleaThe General ModelLinear and Non Linear Models

3 Simulating Hearing LossApproachThe Linear Case; A Sound FilterResults

4 Testing and OutlookTestingOutlook

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 22 / 35

Simulation Approach

• We have a good model of the normal cochlea

• Can we find the Cause given the Effect?

Idea

Find a method to get back the original sound from the model of ahealthy ear. Use this method on a model of a damaged ear tosimulate hearing loss

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 23 / 35

Simulation Approach

• We have a good model of the normal cochlea

• Can we find the Cause given the Effect?

Idea

Find a method to get back the original sound from the model of ahealthy ear. Use this method on a model of a damaged ear tosimulate hearing loss

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 23 / 35

Next Subsection1 Introductions and Motivation

IntroductionMotivationThe Ear and Cochlea

2 Modeling the CochleaThe General ModelLinear and Non Linear Models

3 Simulating Hearing LossApproachThe Linear Case; A Sound FilterResults

4 Testing and OutlookTestingOutlook

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 24 / 35

The Model Viewed as a Convolution

0 2 4 6 8 10 12 14 16 18 20 22−1

−0.5

0

0.5

1Sound: A cappella singing

Time in seconds

Am

plit

ude (

dB

FS

)

0 2 4 6 8 10 12 14 16 18 20 22−2

−1

0

1

2x 10

−7 Oscillator trace

Time in seconds

Oscill

ato

r d

isp

lace

me

nt

(nm

/ms)

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 25 / 35

Finding the Inverse Filter

If the original sound is x and the result from the model is y then themodel is h

y(t) = x(t) ∗ h(t)

Performing a Fourier Transformation results in:

Y (f ) = X (f ) · H(f )

A change in basic operation from convolution to multiplication

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 26 / 35

Finding the Inverse FilterThen H and H−1 are:

H(f ) =Y (f )

X (f )

H−1(f ) =1

H(f )

X (f ) Can then easily be found

X (f ) =Y (f )

H(f )

And then:

x(t) = Inverse Fourier Transform of (X (f ))

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 27 / 35

Next Subsection1 Introductions and Motivation

IntroductionMotivationThe Ear and Cochlea

2 Modeling the CochleaThe General ModelLinear and Non Linear Models

3 Simulating Hearing LossApproachThe Linear Case; A Sound FilterResults

4 Testing and OutlookTestingOutlook

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 28 / 35

Some Results

0 2 4 6 8 10 12 14 16 18 20 22−1

−0.5

0

0.5

1Sound: A cappella singing

Time in seconds

Am

plit

ude (

dB

FS

)

0 2 4 6 8 10 12 14 16 18 20 22−1

−0.5

0

0.5

1Resynthesized sound

Time in seconds

Am

plit

ude (

dB

FS

)

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 29 / 35

What Affects Sound Reproduction Quality

• Length of impulse response!!

• Type of window used, Gaussian seems to be a good choice

• Which oscillator is used

• Integration time step used (affects sound quality)

• Number of oscillators in the Model

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 30 / 35

What Affects Sound Reproduction Quality

• Length of impulse response!!

• Type of window used, Gaussian seems to be a good choice

• Which oscillator is used

• Integration time step used (affects sound quality)

• Number of oscillators in the Model

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 30 / 35

What Affects Sound Reproduction Quality

• Length of impulse response!!

• Type of window used, Gaussian seems to be a good choice

• Which oscillator is used

• Integration time step used (affects sound quality)

• Number of oscillators in the Model

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 30 / 35

What Affects Sound Reproduction Quality

• Length of impulse response!!

• Type of window used, Gaussian seems to be a good choice

• Which oscillator is used

• Integration time step used (affects sound quality)

• Number of oscillators in the Model

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 30 / 35

What Affects Sound Reproduction Quality

• Length of impulse response!!

• Type of window used, Gaussian seems to be a good choice

• Which oscillator is used

• Integration time step used (affects sound quality)

• Number of oscillators in the Model

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 30 / 35

Next Subsection1 Introductions and Motivation

IntroductionMotivationThe Ear and Cochlea

2 Modeling the CochleaThe General ModelLinear and Non Linear Models

3 Simulating Hearing LossApproachThe Linear Case; A Sound FilterResults

4 Testing and OutlookTestingOutlook

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 31 / 35

Testing

• Objective testing may be limited

• Subjective testing• Comparing two sounds on Right and Left channel does not work• Consecutive listening tends to work better• System limitations should be considered

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 32 / 35

Testing and Optimization, a Possibility

0 2 4 6 8 10 12 14 16 18 20 22−1

−0.5

0

0.5

1Resynthesized sound

Time in seconds

Am

plit

ude (

dB

FS

)

0 2 4 6 8 10 12 14 16 18 20 22−1

−0.5

0

0.5

1Resynthesized sound

Time in seconds

Am

plit

ude (

dB

FS

)

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 33 / 35

Next Subsection1 Introductions and Motivation

IntroductionMotivationThe Ear and Cochlea

2 Modeling the CochleaThe General ModelLinear and Non Linear Models

3 Simulating Hearing LossApproachThe Linear Case; A Sound FilterResults

4 Testing and OutlookTestingOutlook

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 34 / 35

Outlook

• We have a good model of the ear

• Can we get good sound back from the model?

• Improving the sample rate

• Using multiple oscillators

• Look for optimality using the settings required for the Non-linearmodel

• Resynthesizing sound from the non-linear model

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 35 / 35

Outlook

• We have a good model of the ear

• Can we get good sound back from the model?

• Improving the sample rate

• Using multiple oscillators

• Look for optimality using the settings required for the Non-linearmodel

• Resynthesizing sound from the non-linear model

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 35 / 35

Outlook

• We have a good model of the ear

• Can we get good sound back from the model?

• Improving the sample rate

• Using multiple oscillators

• Look for optimality using the settings required for the Non-linearmodel

• Resynthesizing sound from the non-linear model

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 35 / 35

Outlook

• We have a good model of the ear

• Can we get good sound back from the model?

• Improving the sample rate

• Using multiple oscillators

• Look for optimality using the settings required for the Non-linearmodel

• Resynthesizing sound from the non-linear model

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 35 / 35

Outlook

• We have a good model of the ear

• Can we get good sound back from the model?

• Improving the sample rate

• Using multiple oscillators

• Look for optimality using the settings required for the Non-linearmodel

• Resynthesizing sound from the non-linear model

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 35 / 35

Outlook

• We have a good model of the ear

• Can we get good sound back from the model?

• Improving the sample rate

• Using multiple oscillators

• Look for optimality using the settings required for the Non-linearmodel

• Resynthesizing sound from the non-linear model

Leo Koop (TU Delft) Simulating Hearing Loss April 9, 2015 35 / 35