Fast Maximum Likelihood Estimation methodusing efficient MCMC proposalB. Karimi1,2, M. Lavielle1,2, E. Moulines2

INRIA1, CMAP École Polytechnique2

belhal.karimi@polytechnique.edu

1 Problem statement

Population models are widely used in domains like pharmacometrics where we need tomodel phenomena observed in each set of individuals. The population approach can beformulated in statistical terms using mixed effect models. When the conditional expecta-tion of the complete log likelihood is hard to compute, the Maximum Likelihood estimatesare obtained using a stochastic version of the EM algorithm. Yet, this method implies beingable to sample from the posterior distribution of the parameters given the observed data.A Markov Chain Monte Carlo procedure can be used to perform this simulation.Our contribution consists in accelerating this posterior sampling in order to improve the over-all parameter estimation algorithm convergence properties.

Notations and Models• Population approach. We denote by N the number of individuals in the population

and n

i

the number of observations per individual i. Let us define the observed datay = (y

i

, 1 i N) where y

i

= (y

ij

, 1 j n

i

) is the vector of observations y

ij

that taketheir values in a subset of Rl.

• A natural decomposition of the joint distribution consists in writing:

p(yi

,

i

; ✓) = p(yi

| i

; ✓)p( i

; ✓) (1)

• p( i

; ✓) is the so-called population distribution used to describe the distribution of theindividual parameters within the population.

• Incomplete log likelihood L(✓; y)

L(✓; y) , p(y; ✓) =NY

i=1

p(yi

; ✓)

(2)

• The ML estimate of ✓ is thus defined by:

ˆ

✓

ML

= argmax

✓2⇥L(✓, y) (3)

• Mixed Effect models. Describing each individual parameters i

as a composition offixed effects, common to the whole population, and random effects as follows:

u(

i

) = u(

pop

) + C

i

� + ⌘

i

(4)

With � a new vector of fixed effects and C

i

a matrix of individual covariates.

2 Maximum Likelihood Estimation

2.1 SAEM Algorithm coupled with MCMC procedure

In this incomplete data model context, the estimation algorithm consists in:Algorithm 1 SAEM algorithmInitialisation: sample latent data

0

i

⇠ p( i

|yi

; ✓

0

) under a given model estimate ✓

0,Iteration k: given the current estimate ✓

k�1:1. Sampling latent data

k

i

⇠ p( i

|yi

; ✓

k�1

) under the current model parameter estimate ✓

k�1

for i 2 [[1, N ]] using an MCMC algorithm.

2. Updating the stochastic approximation Q

k

(✓) of the quantity Ehlog p(y, ; ✓)|y, ✓k�1

i:

Q

k

(✓) = Q

k�1

(✓) + �

k

2

4NX

i=1

log p(yi

,

k

i

; ✓)�Q

k�1

(✓)

3

5 (5)

Where {�k

}k>0

is a sequence of positive stepsize with �

1

= 1.3. Set ✓k = argmax

✓2⇥Q

k

(✓)

Theorem 2.1: Convergence of the SAEM coupled with MCMCWith certain assumptions of ergodicity and smoothness of the transition kernel used in theMCMC:1. if the complete model belongs to the exponential family and its sufficient statistics stay

in a compact, then the results of convergence of [B. Delyon and Moulines(1999)] holdsw.p.1.

2.2 Posterior sampling - Metropolis Hastings Algorithm

2.2.1 Continuous models

In the case where the outcomes are continuous and the individual parameters i

are nor-mally distributed the model is defined by:

y

ij

= f (t

ij

,

i

) + ✏

ij

(6)

and our new method is based on the linearisation of the structural model around the MAPdefined as ˆ

i

= argmax

i

p( i

|yi

, ✓).

Gaussian proposal for continuous models• Taylor expansion of the structural

model around the MAP:

f (t

i

,

i

) ⇡ f (t

i

,

ˆ

i

)+r

i

f (t

i

,

ˆ

i

).

>(

i

� ˆ

i

)

(7)• Resulting linear model between y

i

and

i

:

y

i

� f (t

i

,

ˆ

i

) +r

i

f (t

i

,

ˆ

i

).

>ˆ

i

= r

i

f (t

i

,

ˆ

i

).

>

i

+ ✏

i

(8)

• Tractable conditional distribution i

|yi

:Gaussian distribution N (µ

i

,�

i

) with pa-rameters:

µ

i

=

ˆ

i

�

i

=

"r

i

f (

ˆ

i

).

>r

i

f (

ˆ

i

)

�

2

+ ⌦

�1

#�1

(9)

2.2.2 Non Continuous models

As far as non continuous outcomes, there is no analytical relationship between the obser-vations and the individual parameters and thus no linearisation can be applied. Here, thestrategy to build an efficient proposal consists in using a Laplace approximation of the jointmodel as described in [Wolfinger(2017)] or [Y.(2007)].Define l(

i

) , p(yi

| i

). We can derive the following Gaussian proposal:Gaussian proposal for non continuous models• Laplace approximation, around the

MAP, of:

p(yi

, ✓) =

Ze

log p(yi

,

i

,✓)d i

(10)

• We obtain:

�2 log p(yi

) ⇡ �2 log l(

ˆ

i

)� 2 log p( ˆ i

)

� p log 2⇡ + log |�r2

log p(yi

,

ˆ

i

)|(11)

• Gaussian proposal:

µ

i

=

ˆ

i

�

i

=

h�r2

log l(

ˆ

i

) + ⌦

�1

i�1

(12)

• Fisher Approximation:

�r2

log l(

ˆ

i

) ⇡ Ey

i

| ˆ i

hr2

log l(

ˆ

i

)

i(13)

• Combined to the Fisher identity, weobtain:

�r2

log l(

ˆ

i

) ⇡ rl(

ˆ

i

).rl(

ˆ

i

)

>

l

2

(

ˆ

i

)

(14)

3 Numerical Application: Warfarin dataset

Warfarin is an anticoagulant normally used in the prevention of thrombosis and thromboem-bolism, the formation of blood clots in the blood vessels and their migration elsewhere in thebody, respectively. In [RA. O’reilly(1968)], O’Reilly provide set of plasma warfarin concen-trations and Prothrombin Complex Response in thirty normal subjects after a single loadingdose. A single large loading dose of warfarin sodium, 1.5 mg/kg of body weight, was ad-ministered orally to all 32 subjects. Measurements were made each 12 or 24h. The datasetcan be found in Monolix and simulx R package.

0

5

10

15

0 12 24 36 48 60 72 84 96 108 120Time

Concentration

Figure 1: Warfarin concentration over time for 32 subjects.

PK model

y

ij

=

Dka

V (ka� k)

(e

�kat � e

�kt

) + ✏

ij

(15)

Where ka is he absorption rate constant, k is the elimination rate constant, V is the volumeof distribution and D is the dose administered.In our notation, the complete model is p(y

i

,

i

, ✓) where

i

= (ka

i

, V

i

, k

i

) is the vector of in-dividual parameters. We apply a log transformation to each of the three variables. Then,u(

i

) = (log(ka

i

), log(V

i

), log(k

i

)).

• Fast MCMC Convergence

-0.2

0.0

0.2

0.4

100 1000 10000

ka

-0.15

-0.10

-0.05

0.00

100 1000 10000

V

-0.1

0.0

0.1

0.2

100 1000 10000

k

Figure 2: Convergence of quantiles (0.05, 0.5, 0.95).

• Fast SAEM Convergence

2.5

5.0

7.5

10.0

12.5

10 100 200

V

0.0

0.5

1.0

10 100 200

ω.V

0

5

10

10 100 200

V

0.0

0.2

0.4

0.6

10 100 200

ω.V

Figure 3: Runs on Warfarin dataset (Left) and average error on 100 synthetic datasets (Right) Fast methodin red and reference in blue.

References[B. Delyon and Moulines(1999)] M. Lavielle B. Delyon and E. Moulines. Convergence of a stochastic approxi-

mation version of the EM algorithm. The Annals of Statistics, 1999.[E.Kuhn(2015)] M. Lavielle E.Kuhn. Coupling a Stochastic Approximation version of EM with an MCMC Proce-

dure. ESAIM: Probability and Statistics, (8), 2015.[RA. O’reilly(1968)] PM Aggeler RA. O’reilly. Studies on coumarin anticoagulant drugs. Initiation of warfarin

therapy without a loading dose. 1968.[Wolfinger(2017)] R. Wolfinger. Laplace’s Approximation for Nonlinear Mixed Models. Biometrika, 80, 2017.[Y.(2007)] Wang Y. Derivation of various NONMEM estimation methods.. Journal of Pharmacokinetics and

Pharmacodynamics, 2007.