Appointment Scheduling in Healthcare

Appointment scheduling in healthcare

Alex Kuiper

December 9, 2012

Master Thesis

Supervisors:

prof.dr. Michel Mandjes

dr. Benjamin Kemper

IBIS UvA

Faculty of Economics and Business

Faculty of Science

University of Amsterdam

AbstractPurpose: Appointment scheduling has been studied since the mid-20th century.Literature on appointment scheduling often models an appointment schedulingsystem as a queueing system with deterministic arrivals and random servicetimes. In order to derive an optimal schedule, it is common to minimize thesystems loss in terms of patients waiting times and idle time of the server byusing a loss function. This approach is translated to a D/G/1 queue, which iscomplex for a broad range of settings. Therefore many studies assume servicetime distributions that lead to tractable solutions, which oversimplifies the prob-lem. Also, many studies overcome the complexity through simulation studies,which are often case specific. The purpose of this thesis is to offer an approachto deal with arbitrary service times and to give guidance for practitioners infinding optimal schedules.Approach: First, we approximate service time distributions by a phase-typefit. Second, we compute the waiting and idle times per patient. Finally, werun algorithms that minimize, simultaneously or sequentially, the systems loss.This approach enables us to find optimal schedules for different loss functionsin both the transient and steady-state case.Findings: Our approach is an explicit and effective procedure to find optimalschedules for arbitrary service times. Optimal schedules are derived for dif-ferent scenarios; i.e. for sequential and simultaneous optimization, linear andquadratic loss functions and a broad range of service time distributions.Practical implications: The procedure can be used to compute optimal sched-ules for many practical scheduling issues that can be modeled as a D/G/1queue.Value: We present a guideline for optimal schedules that is of value to practi-tioners in services and healthcare. For researchers on appointment schedulingwe present a novel approach to the classic problem of scheduling clients on aserver.

InformationTitle: Appointment scheduling in healthcareAuthor: Alex Kuiper, [email protected], 5647169Supervisors: prof.dr. Michel Mandjes, dr. Benjamin KemperSecond readers: dr.ir. Koen de Turck, dr. Maurice KosterDate: December 9, 2012

IBIS UvAPlantage Muidergracht 121018 TV Amsterdamhttp://www.ibisuva.nl

Contents

1 Introduction 4

2 Background of appointment scheduling 62.1 Literature review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.1.1 Dynamic versus static scheduling . . . . . . . . . . . . . . . . . . . . . 62.1.2 The D/G/1 model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.1.3 The arrival process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.1.4 The service time distribution and queue discipline . . . . . . . . . . . 82.1.5 Some remarks on scheduling . . . . . . . . . . . . . . . . . . . . . . . . 8

2.2 Model description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3 Approximation by a phase-type distribution 153.1 Phase-type distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153.2 Phase-type approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193.3 Phase-type fit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233.4 Recursive procedure for computing sojourn times . . . . . . . . . . . . . . . . 24

3.4.1 Exponentially distributed service times . . . . . . . . . . . . . . . . . 243.4.2 Phase-type distributed service times . . . . . . . . . . . . . . . . . . . 26

4 Optimization methods 304.1 Simultaneous optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304.2 Sequential optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

4.2.1 Quadratic loss . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324.2.2 Absolute loss . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

4.3 Lag-order method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334.4 Computational results for transient cases . . . . . . . . . . . . . . . . . . . . 34

5 Limiting distributions 395.1 The D/M/1 queue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

5.1.1 Limit solutions in the sequential case . . . . . . . . . . . . . . . . . . . 405.1.2 Limit solutions in the simultaneous case . . . . . . . . . . . . . . . . . 40

5.2 The D/EK1,K/1 queue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425.2.1 The limiting probabilities . . . . . . . . . . . . . . . . . . . . . . . . . 435.2.2 The sojourn time distribution . . . . . . . . . . . . . . . . . . . . . . . 44

5.3 The D/H2/1 queue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455.3.1 The limit probabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . 465.3.2 The sojourn time distribution . . . . . . . . . . . . . . . . . . . . . . . 49

2

CONTENTS 3

5.4 Computational results in steady-state . . . . . . . . . . . . . . . . . . . . . . 52

6 Optimal schedules in healthcare 556.1 Performance under Weibull distributed service times . . . . . . . . . . . . . . 566.2 Performance under log-normal distributed service times . . . . . . . . . . . . 58

7 Summary and suggestions for future work 59

Chapter 1

Introduction

In this thesis we study the classic appointment scheduling problem that practitioners oftenencounter in processes in services or healthcare. In such a setting an appointment refers to anepoch that sets the moment of the clients or patients arrival in time. Next, the client receivesa service from the service provider. For example, a doctor sees several patients during a clinicsession in a hospital. A patient arrives exactly on the priorly appointed time epoch. Uponarrival, the patient either waits for the previous patient to be served, or is directly seen bythe doctor. In the latter, the doctor was idle for a certain time period. Ideally the scheduleis such that the patient has no waiting time and the doctor has no idle time. Unfortunately,this case is never realized due to the fact that e.g. the treatment time for every patient isnot constant, but a random variable. Therefore, we must find the best appointment schedulesuch that the expected waiting and idle times are minimized for all patients. In this thesis wewill study different approaches to achieve this.

Above we gave an example of an appointment scheduling problem in a healthcare setting,but there are many more, such as scheduling of MRI and CT patients. MRI and CT scannersare expensive devices and therefore it is crucial to maximize their utilities, i.e. minimizeidle times. So it seems optimal to have a tight schedule for these scanners. But in case ofcomplications during scans, too tight a schedule will result in high waiting times for patients.Another typical example is scheduling the usage of operating rooms in a hospital or clinic.There are only a small number of these special rooms, where various surgeries have to bescheduled on. Therefore, the utility of each room should be maximized, i.e. minimizing theidle time at the expense of the patients (waiting) time. But it is known that poor schedulingperformance, which results in high waiting times, lead to patients choosing other hospitals orclinics for surgeries.

In addition, there are numerous examples outside the healthcare setting. For instance,ship-to-shore container cranes, where the ships can be seen as clients, the service time is thetotal time of unloading, and the cranes must be seen as the service provider. Too tight aschedule results in waiting ships, which incurs extra costs for the ships. On the other hand,unused cranes do not make profit. Furthermore, there is a risk that ships choose for competingharbors with lower waiting times. These examples show that the cost is twofold: we haveboth waiting times for clients and idle time for the server. Finding an optimal schedule whichminimizes both costs is our aim.

Now we have a feeling in how we can translate some practical problems to an appointmentscheduling problem. Assume for now that there is a finite number of patients to be sched-

4

5uled, say N , where the service times Bi for i {1, . . . , N} are random variables which areindependent, and in most cases also identically distributed. The waiting and idle times perpatient is denoted by Wi respectively Ii. The waiting and idle times are random variables aswell, since they depend on the service times of previous scheduled patients. Our goal is tominimize the sum of all waiting and idle times over all possible schedules. A naive approachwould be to schedule all patients equidistantly based on a patients average service time. Thisschedule, denoted by T , can be written as t1 = 0 and ti =

i1j=1 E[Bj ]. However, a session

could take longer than the expected service time. When this happens all subsequent patientswill have positive waiting times in expectation. We will see in Chapter 2 that this scheduleis far from optimal, since it leads to infinite waiting times when N tends to infinity.

An important factor which affects the optimal appointment schedule is the trade-off be-tween Wi and Ii in the cost function Ri. If the time of the patient is relatively more expensive,more weight is put on Wi and the schedule will become less tight. Visa versa, if the doctorstime is considered to be relatively more expensive, more weight is put on Ii and the schedulebecomes more tight.

The outline of this thesis is as follows. In the upcoming chapter we will discuss the back-ground in the form of a literature review and some preliminaries. Because we are interestedin a healthcare setting, we consult related literature. This gives us a proper framework toderive a mathematical model, with relevant cost functions. In the next chapter, Chapter 3,we will introduce phase-type distributions, which can be used to approximate any positivedistribution arbitrary accurately. We will give a solid theoretical basis to sustain this claim.Furthermore, they exhibit a specific property which allows us to use a recursive procedure.

This procedure will be used to compute optimal schedules for various cost functions forfinite amounts of patients, i.e. the transient case in Chapter 4, first. We will use two differentoptimization approaches: simultaneous and sequential. Simultaneous optimization is findingan optimal schedule for all patients jointly, while sequential optimization is a recursive ap-proach in which you optimize patient by patient. The latter has the advantage that it reducesthe scheduling problem to finding the optimal schedule for one patient each time. At the endof this chapter we will compare our findings with relevant literature. Secondly, in Chapter 5,we will derive a method to compute the optimal schedule in its steady-state for differentcoefficients of variation, which is a measure of the variability of the service times. Setting theexpectation equal to one will allow us to compare the performance of optimal schedules underchanges in the coefficient of variation. We will compute the optimal schedule in steady-statefor both the sequential and simultaneous approach and for different cost functions, which willbe used in Chapter 4.

In Chapter 6 we check the performance of our approach in simulated examples basedon real-life settings. We model these examples by data generated from either the Weibullor log-normal distribution. The choice of these particular distributions originates from thehealthcare setting as explained in Chapter 2. In Chapter 7 we summarize our results andsuggest topics for further research.

Finally I am thankful to my supervisors Benjamin Kemper and Michel Mandjes for theirscientific support and guidance during my six-month internship at the Institute for Businessand Industrial Statistics of the University of Amsterdam (IBIS UvA). Also, I would like tothank the second readers Maurice Koster and Koen de Turck who made the effort to readand comment on my work.

Chapter 2

Background of appointmentscheduling

In this chapter we perform a background study on appointment scheduling. In the upcomingsection we review literature on appointment scheduling with a focus on healthcare. Further,we derive assumptions for a model. So that at the end of this chapter we have a well definedoptimization problem in a properly justified model.

2.1 Literature review

The research on appointment scheduling dates back to the work of Bailey (1952,1954) [6, 7],Welch and Bailey (1952) [5] and Welch (1964) [28]. The authors study the phenomenonof scheduled patients who are to be treated in a typical healthcare delivery process. Thisphenomenon of priorly scheduled arrivals that are treated by a service provider is often studiedas a queueing system in which jobs arrive following a deterministic arrival process and receivea service which varies in duration.

After this pioneering work on appointment scheduling the subject has been extensivelyresearched in both services and healthcare. The article by Cayirly and Veral (2003) [10] givesa good overview on the state of the art in appointment scheduling. We use this article tohighlight special features for appointment scheduling in healthcare.

2.1.1 Dynamic versus static scheduling

To begin with the objective of outpatient scheduling is to find an appointment system forwhich we optimize over the systems loss, which is the sum of the expected losses incurred bywaiting and idle times. Literature on appointment scheduling can be divided into two cate-gories with respect to outpatient scheduling: static and dynamic. In the static environmentthe appointment system is completely determined in advance, in other words off line schedul-ing. This is in contrast to the dynamic case in which changes in the schedule are permitted,so called online scheduling. Most literature focuses on the static case; only a few papers suchas Fries and Marathe (1981) [14] and Liao et al. (1998b) [18] consider the dynamic case inwhich the schedule of future arrivals are revised continuously.

6

2.1. LITERATURE REVIEW 7

2.1.2 The D/G/1 model

The outpatient scheduling problem can be modeled by a queueing model. The so calledD/G/1 queueing model, by Kendalls three-factor (A/B/C) classification (1953) [17]. Thethree parameters are:

A, the arrival process, in our case the schedule, is deterministic, denoted by D.

B, the service time distribution is general, denoted by G. This means explicitly that anypositive distribution can be taken to model the service times.

C, the number of servers is set to 1, since we have a single doctor or practitioner.

An optimal schedule in this context is the deterministic arrival process of the patients, whichminimizes the sum of Ri for all i. A few studies in healthcare investigated so called multi-stage models in which patients have to go through several facilities, such as in Rising et al.(1973) [20], who performed an extensive case study and Swisher et al. (2001) [21], who dida broad simulation study. We will consider practitioners and doctors as independent queues,because there is a doctor-patient relationship, often seen in literature: Rising et al. (1973) andCox et al. (1985) [11], but also justified by contemporary medical ethics. It is a mathematicalfact that when there are multiple doctors it is better to have a single flow of patients, whoare scheduled to the first available doctor.

2.1.3 The arrival process

The arrival process itself can come in many forms. First important factor is the unpunctualityof patients, which is the difference between the time the patient arrives and his appointmenttime. Generally patients arrive more early than late, which is showed in many studies thatcan be found in Cayirly and Veral (2003). On the other hand the doctors unpunctuality canbe measured as lateness to the first appointment. This kind of lateness is only considered insome studies. A second important factor, also addressed in Cayirly and Verals paper is thepresence of no-shows, that is that a patient does not show up at all. Empirical studies showthat the probability of no-shows ranges from 5% to even 30%. Simulation studies show thatthe no show probability has a greater impact on the appointment schedule than the coefficientof variation and the number of patients per clinic session, see the extensive simulation studyby Ho and Lau (1992) [9].

On the other hand patients can also drop by, not having a priorly scheduled appointment.These are called walk-in patients, the presence of these kind of patients is not often incorpo-rated in studies. This is of course in line with the static approach of appointment scheduling,since scheduling these walk-in patients on-the-fly will result in a modification to the schedule.In case there are walk-ins this should not automatically harm the staticity of the appointmentschedule since one can take walk-ins into account in the schedule by seeing walk-in patientsonly at instances when the doctor is idle. Another factor on the arrival process is the presenceof companions. The D/G/1 queueing model has infinite waiting capacity and therefore thepresence of companions is not taken into account. Moreover, there is no restriction on thenumber of waiting patients. However as remarked in Cayirly and Veral, for hospitals it isimportant to know how many people are likely to use the waiting room, since hospitals haveto facilitate all the waiting patients and their companions.

8 CHAPTER 2. BACKGROUND OF APPOINTMENT SCHEDULING

2.1.4 The service time distribution and queue discipline

So far we discussed the arrival process and the number of service providers (doctors). Re-maining is the service time and queue discipline. The queue discipline is in all studies on afirst-come, first-served (FCFS) basis. In case of punctual patients this discipline is the sameas serving patients in order of arrival. But, if patients are unpunctual doctors can choose tosee the next patient when the next patients is already dropped in. This reduces his idle time.However, assuming that lateness is always less than the scheduled interarrival times, we canassume that the order of arrival will be equal to the scheduled order.

Now, we discuss the most important factor for this thesis, that is the service time (perpatient). The total service time is defined to be the sum of all the time a patient is claimingthe doctors attention, preventing him or her seeing other patients, i.e. the service timesper patient, see Bailey (1952). An important quantity in queuing theory is the (squared)coefficient of variation, denoted by (S)CV , of a random variable Bi for patient i

CV =

, and SCV =

2

2

where = E[Bi] and 2 = Var[Bi]. Many analytical studies assume Bi to be exponential,obviously because their analytical approach will be intractable otherwise, see Wang [27], Friesand Marathe [14], and Liao et al. [18]. However, the one-parameter exponential distributionsets the CV = 1, which is too restrictive and not seen in practice. More common in healthcareis data with 0.35 < CV < 0.85. Furthermore empirical data collected from clinics andfrequency distributions of observed service times display forms that are unimodal and right-skewed, Welch and Bailey (1952), Rising et al. (1973), Cox et al. (1985), Babes and Sarma(1991) [4] and Brahimi and Worthington (1991) [8]. Examples of such distributions are thelog-normal or (highly flexible) Weibull distribution.

In 1952 Bailey already observed that the performance of the system is very sensitive tosmall changes in appointment intervals. Since then many studies have reported that anincrease of the variability in service times, i.e. the SCV s, lead to an increase of both thepatients waiting times and the doctors idle times, and thus incur extra costs. Examples ofthis phenomenon can be found in Rising et al. (1973), Cox et al. (1985), Ho and Lau (1992),Wang (1997), and in Denton and Gupta (2003) [12]. The choice of an optimal appointmentschedule depends mainly on the mean and variance, see Ho and Lau (1992) and Denton andGupta (2003). Hence it is important to model the observed data well, that is by matchingthe first two moments, a possible way to do so is by phase-type fit, see Adan and Resing(2002) [1]. Wang showed already in (1997) [26] how one can numerically compute an optimalappointment schedule for general phase-type distributions.

2.1.5 Some remarks on scheduling

We finish with some remarks. The appointment scheduling problem can also be considered tobe discrete so that one can only set schedules at certain times. This is the approach followedby Kaandorp and Koole (2007) [15]. The main advantage of this approach is that there isjust a finite number of combinations possible, so that they can use a faster minimizationalgorithm. We consider the continuous time case, which can be seen as the limit case of thediscrete problem of Kaandorp and Koole, which is with their methodology impossible to solvedue to dimension issues.

2.2. MODEL DESCRIPTION 9

Based on experience we assume that one patient who has to wait 20 minutes is worse thanthat 10 clients have to wait only 2 minutes. We can incorporate this effect with cost functions,which penalizes the loss incurred by one long waiting times more than the sum of the shortwaiting times, a common choice for this purpose is a quadratic cost function. Another choiceis a linear cost function, which is more appropriate in a production environment, when wedo not have to incorporate peoples experiences. We can reason along the same lines for thedoctor (or server). We then implement a cost function, either linear or quadratic, on thedoctors idle times.

Since we deal with randomness, driven by the variable service times, we look at these lossesin expectations, which we call risk. A more general discussion of choices for cost functionscan be found in Cayirly and Veral (2003). Herein they define the expected total loss (risk)as a weighted sum of expected waiting times, idle times and overtime. We only focus on thelinear and quadratic cost functions used in Wang (1997) and Kemper et al. (2011). In thenext section we present our model, settings and cost functions in more detail.

2.2 Model description

In this section we present our model. The optimization problem as stated in the introductioncan be written as a minimization of the expected patients waiting times and servers idletimes over the arrival epochs:

mint1,...,tN+1

N+1i=1

(E[Ii] + E[Wi]) (2.1)

in which Wi refers to the patient is waiting time and Ii refers to the servers idle time beforeit serves the i-th patient. This problem is defined in the following setting:

There are N + 1 patients to be priorly scheduled in one clinic session. So there are nowalk-in patients.

Let B1, . . . , BN+1 the service times of N + 1 patients.

The Bis are independent and identically distributed.

The ti are the scheduled times i {1, . . . , N+1}, and let xi = ti+1ti, for i {1, . . . , N}the interarrival times.

Scheduled patients are punctual, always show up and are served in order of arrival.

One wants to minimize a convex cost function Ri depending on the expected waitingtime of patient i and the expected idle time of the doctor.

We schedule N+1 patients, so that we end up with exactly N interarrival times. The relationof the interarrival times xis and the arrival epochs tis is showed in Figure 2.1. We considernow the naive schedule of the introduction. We propose it as an example, which proves thateven a simple, but naive, heuristic can have major impact on (expected) waiting times. Itshows the relevancy of optimal appointment scheduling.


Figure 2.1: the relation between N + 1 arrival epochs ti and N interarrival times xi.

Example 2.1 (A naive appointment schedule). Consider again the appointment schedule Tby setting the interarrival times xi equal to the expected service time of patient i

t1 = 0 and ti =i1j=1

E[Bj ]

for i = 2, . . . , N,N + 1. This is a very simple approach, but in fact the service load is equalto 1. It means that per unit time the in- and outflow of patients is equal to 1, which will leadto infinite waiting times by the following proposition, see Kemper et al. (2011).

Proposition 2.2. In a D/G/1 queue with load 1 starting empty, with the service times Bihaving variance 2 y

]dy.

By Chebyshevs inequality the integrand is bounded

P

[1k

ki=1

Bi Ai

> y

] min

{1,

1

y2Var

[1k

ki=1

Bi Ai

]},

so that

Ik

0

(1 1

y2

)dy =

10

dy +

1

1

y2dy = 2.


This gives us a majorant so that we can apply both the Dominated Convergence and theCentral Limit Theorem

limk

P

[1k

ki=1

Bi Ai

> y

]=

0

(1 (y)) dy = 12pi.

Furthermore

1N

N1k=1

1kIk =

10

1[x1N1]dNxe/N IdNxe/N dx.This integrand is again bounded, knowing that IdNxe/n < 2, by 2x1/2 so

10 2x

1/2 dx = 4.Using Dominated Convergence Theorem gives us pointwise convergence of

limN

1N

N1k=1

1kIk =

10

12pi

1xdx =

2

pi.

We finish the proof by multiplying the latter expression by which gives us the right-handside of equation (2.2).

What happens in the last example is that the occupation rate, defined as

=E[Bi]E[Ai]

,

is equal to 1. The occupation rate is also known as the service load. To ensure stability, i.e.with probability zero we have infinite patients waiting if t, we need that E[Bi] < E[Ai],i.e. the service load should be less than 1.

Let Ri = Eg(Ii) +Eh(Wi) the risk per patient, where g, h : R0 R0 are cost functions.Also, we demand that g, h are convex and satisfy g(0) = h(0) = 0, so that the problem is

mint1,...,tN+1

R = mint1,...,tN+1

N+1i=1

Ri = mint1,...,tN+1

E

[N+1i=1

(g(Ii) + h(Wi))

], (2.3)

c.f. equation (2.1). The risk R can also be thought of the systems loss, since it captures allthe loss incurred by idle and waiting times.

Proposition 2.3. It is always optimal to schedule the first patient at time zero.

Proof. Suppose one has an optimal schedule V where t1 = a > 0 and the alternative (shifted)schedule W where t1 = 0 and all ti = ti a then expected idle time of the first patient islarger than 0, because by punctuality

E[g(I1)] = g(a) + E[g(I1 )] = g(a) and E[h(W1)] = E[h(W 1 )] = 0.

Furthermore, for all i {2, . . . , N + 1}

E[g(Ii)] = E[g(Ii )] and E[h(Wi)] = E[h(W i )]

as patients arrive on their arrival epochs ti or ti = tia. Observe that schedule RV = RW+a,

which shows that schedule W is the optimal schedule.


By Lindleys recursion formulas, see [19], which are graphically demonstrated in figure Fig-ure 2.2, we have

Ii = max{(ti ti1)Wi1 Bi1, 0} = max{xi1 Si1, 0}, (2.4)Wi = max{Wi1 +Bi1 (ti ti1), 0} = max{Si1 xi1, 0}, (2.5)

Define now the loss function l(x)

Figure 2.2: the relations between interarrival, idle, sojourn and waiting times. We see thatthe idle time Ii can be written as the interarrival time xi1 minus the sojourn time Si1. Thewaiting time Wi can be written as the sojourn time Si1 minus the interarrival time xi1.The figure originates from Vink (2012) [24].

l(x) = g(x)1x0.

The function l(x) is also convex which can be deduced easily by the fact that g(x) and h(x)are convex functions satisfying g(0) = h(0) = 0. Proposition 2.3 and the fact that eitheridle time is zero and waiting time is positive or visa versa, one can reduce the optimizationproblem to minimizing over N arrival epochs or interarrival times:

R =

N+1i=1

Ri = E

[N+1i=2

(g(Ii) + h(Wi))

]= E

[Ni=1

l (Si xi))].

There are many loss functions possible, but often one considers absolute loss l(x) = |x|,which corresponds with linear costs or quadratic loss l(x) = x2. The latter loss functioncorresponds with quadratic costs of waiting and idle times. We can generalize these lossfunctions to weighted versions, i.e. let (0, 1):

R() =N+1i=2

E[ |xi1 Si1|1Si1xi10

]=

N+1i=2

E[ |Si1 xi1|+ (1 2) |Si1 xi1|1Si1xi1>0

](2.6)

= E

[N+1i=1

Ii + (1 )Wi],

(2.7)


and

R() =

N+1i=2

E[ (xi1 Si1)2 1Si1xi10

]=

N+1i=2

E[ (Si1 xi1)2 + (1 2) (Si1 xi1)2 1Si1xi1>0

](2.8)

= E

[N+1i=1

I2i + (1 )W 2i].

(2.9)

Furthermore, there are other variables which can incorporate costs, for example the approachby Wang [26] [27] where he minimizes the expected sojourn times per patient

N+1i=1

E[Si] =N+1i=1

E[Wi +Bi]

where E[W1] = 0. However, E[Bi]s do not depend on the schedule choice, so that it isequivalent with minimizing waiting time only. But minimizing waiting time only will takeinterarrival times xi so we add the expected systems completion time. The expectedsystems completion time is defined as the sum of arrival time of the last patient tN+1 andhis expected sojourn time E[SN+1]. We can generalize this also to weighted version by givingweights to the sojourn and completion time.

Since the system completion time can also be seen as the sum of all idle times plus servicetimes. It is equivalent with minimizing idle time only. So Wangs choice of cost functions isequivalent with linear costs. An equivalent definition is the sum of all the interarrival timesplus the expected sojourn time of the last patient. We summarize, let (0, 1)

R() = N+1i=1

E[Si] + (1 ) (tN+1 + E[SN+1])

= N+1i=1

(E[Wi] + E[Bi]) + (1 )N+1i=1

(E[Ii] + E[Bi])

= N+1i=1

E[Wi] + (1 )N+1i=1

E[Ii] +N+1i=1

E[Bi]

=

N+1i=1

E[Si] + (1 )(E[SN+1] +

Ni=1

xi

). (2.10)


So we distinguished four different costs in terms of time:

Completion time: the time when the doctor (server) finishes seeing the last patient. Idle time: the time that a server is idle before the next patient comes in. Waiting time: the time that a patient has to wait before he or she is seen (served). Sojourn time: the sum of waiting and service time.

Moreover, we observed that

R() = R() +N+1i=1

E[Bi], (2.11)

which shows the relation between the systems loss of Kemper and Wang. Minimizing absoluteloss, equation (2.6), is equivalent with minimizing Wangs R. This allows us to compare theirmethods at the end of Chapter 4.

In this chapter we proposed and motivated our model. In addition, we showed somepreliminary results such as finding an optimal appointment schedule is equivalent with findingoptimal interarrival times. Moreover, we reduced the problem of minimizing functions of idleand waiting times to minimizing a function of the sojourn time only. In the next chapterwe focus on a rich class of distributions which lie dense in the class of all distributions andhave appropriate properties. These are called phase-type distributions. We will use thesedistributions in our minimization problem to compute optimal schedules for transient andthe steady-state cases.

Chapter 3

Approximation by a phase-typedistribution

In this chapter we will start with an overview of phase-type distributions. The reason whywe will focus on phase-type distributions is that they are very flexible in capturing datastructures, for example via moment matching [1] or via the EM algorithm [3]. In the nextsection we will give a theoretical framework, see [23] and [13], which proves that phase-typedistributions can approximate any positive distribution arbitrarily accurately. After showingthis result we will give explicit formulas of how we can fit phase-type distributions based on itsfirst two moments. In the final section we show a recursive procedure of computing the sojourntime distribution of patients in the D/PH/1 setting when their service time distribution isphase-type.

3.1 Phase-type distribution

The assumption of exponential service times is often used, because of its simplicity. Thememoryless property is what makes it so attractive to make this assumption, that is

P [X > t+ s] = P [X > t]P [X > s] .

The reason why we focus on phase-type distributions is that they can be seen as a generaliza-tion of exponential service times. They do not satisfy the condition of above, except of coursethe special case of exponential service times. They have another attractive property that is,loosely speaking, that when the process is stopped at an arrival time ti the probability isdistributed over the possible number of phases in the system. This distribution of probabilityover the phases can be taken as a new probability vector, which can be used as a start vectorin the same phase-type form, but time is then starting at t = 0 instead of ti. This propertycan be exploited to (numerically) compute optimal arrival times.

First, we start with the precise definition of a phase-type distribution and give some keyexamples. Consider a Markov process Jt on a finite state space {0, 1, . . . , p}, where 0 isabsorbing and the other states are transient. The infinitesimal generator Q is given as

Q =

(0 0S0 S

),

15

16 CHAPTER 3. APPROXIMATION BY A PHASE-TYPE DISTRIBUTION

where S is an mm-matrix and S0 = S1, which is the so called exit vector. The vector 1is a column vector of ones of length m, so that each row in Q sums to zero.

Definition 3.1. A random variable X distributed as the absorption time inf {t > 0 : Jt = 0}with initial distribution (0,) (row vector of length m + 1) is said to be phase-type dis-tributed. In phase-type representation we say X PH(,S), since and S define thecharacteristics of the phase-type distribution completely.

The transition matrices

Pt = exp (Qt) =

n=0

Qntn

n!

of the Markov process Jt can also be written down in block partitions

Pt =

(1 0

1 eSt1 eSt),

which gives an expression for the distribution function

F (t) = 1eSt1.

Also, other basic characteristics of phase-type distributions can be derived:

1. The density: f(t) = eStS0 = eStS1.

2. The Laplace-Stieltjes transform:

0 est F (ds) = (sI S)1S0.

3. The n-th moment: E [Xn] = (1)nn!Sn1.Phase-type distributions do not have unique representations, see Example 3.2.

Example 3.2 (Erlang distribution). This distribution is denoted by EK() and is a specialcase of the gamma distribution in which the shape parameter is a natural number. Itsprobability density function is given by

f(t) = (t)K1

(K 1)!et.

The interpretation is that a random variable has to go through K exponential phases withsame scale parameter. So that we can also write this in phase-type representation (,S),namely = (1, 0, . . . , 0) of dimension 1K and a matrix S of dimension K K

S =

0 . . . 00 . . . ......

. . .. . .

. . . 00 . . . 0 0 . . . 0 0

.

Its moments are given by

E[Xn] =(K + n 1)!

(K 1)!1

n

3.1. PHASE-TYPE DISTRIBUTION 17

so that its SCV = 1K .

A further generalization is to take a mixture of two Erlang distributions with same scaleparameter, denoted by EK1,K(). This distribution is of special interest to us, since we willuse it to approximate distributions with. Let EK() with probability 1p and EK1() withprobability p, so = (1 p, 0, . . . , 0, p, 0, . . . , 0) (dimension 1 (K + (K 1)) and

S =

0 . . . 0 0 0 . . . 0 00 . . . ... 0 0 . . . 0 0...

. . .. . .

. . . 0...

... . . ....

...

0. . . 0 0 0 . . . 0 0

0 0 . . . 0 0 0 . . . 0 00 0 . . . 0 0 0 . . . 00 0 . . . 0 0 0 . . . ......

... . . ....

......

. . .. . .

. . . 0

0 0 . . . 0 0 0. . . 0

0 0 . . . 0 0 0 0 . . . 0

,

where S is a K+ (K 1)K+ (K 1)-matrix. (Upper left block has dimension KK andlower right block K 1K 1.) This is equivalent with the following, more parsimonious,representation, = (1, 0, . . . , 0) and matrix

S =

0 . . . 00 . . . ......

. . .. . .

. . . 0

0. . . 0 (1 p)

0 0 . . . 0

with dimension K K. Its moments are given by

E[EnK1,K ] = p(K + n 1)!

(K 1)!1

n+ (1 p)(K + n 2)!

(K 2)!1

n

and so its SCV = K(p1)2

(K+p1)2 , which is in the interval[

1K ,

1K1

]when p varies between [0, 1]. So

we can use a mixture Erlang distributions with K {2, 3, . . .} to approximate a distributionswith a SCV < 1, which we will do so in Section 3.3. In fact by Theorem 3.5 we knowthat a mixture of Erlang distributions can approximate any distribution arbitrary accurately.Finally this example demonstrated that phase-type representations PH(,S) are not unique.

Example 3.3 (Hyperexponential distribution). The hyperexponential distribution can beseen as a mixture of exponential random variables, with different parameters 1, . . . , n > 0and with i > 0 and

ni=1 i = 1, such that its density is

f(t) =ni=1

iieit, t > 0.


This distribution is often denoted as Hn(1, . . . , n;1, . . . , n). In phase-type representationthe probability vector = (1, . . . , n) and the matrix

S =

1 0 . . . 0

0 2 . . ....

.... . .

. . . 00 . . . 0 n

.Consider the special case where n = 2, such that = (p1, p2) with pi > 0 and p1 + p2 = 1.

S =

(1 00 2

).

Then we have

F (t) = 12i=1

pieit and f(t) =

2i=1

piieit

and its moments are given by

E[Hn2 ] = p1n!

n1+ p2

n!

n2,

so that the SCV = 2p121+p2

22

(p11+p22)2 1 1, because

p121 + p2

22 = p1(p1

21 + p2

22) + p2(p1

21 + p2

22)

= p2121 + p1p2(

21 +

22) + p

22

22

p2121 + 2p1p212 + p2222 = (p11 + p22)2.

Hence we can use this distribution to approximate distributions with a SCV 1, which wewill show in Section 3.3 and the theoretical basis for this is given by Theorem 3.10.

Example 3.4 (Coxian distribution). The Coxian distribution, notation CK , is a wide classin which the mixture Erlang is a special case in which the service times are equal. Thephase-type representation is given by a vector = (1, 0, . . . , 0) and the matrix

S =

1 p11 0 . . . 00 2 p22 . . .

......

. . .. . .

. . . 0

0. . . 0 K1 pK1K1

0 0 . . . 0 K

.

We restrict ourselves to the Coxian-2 since this distribution is sometimes used in approxima-tions whenever SCV > 12 , since by straightforward computations we have that

E [C2] =1

1+

p

2,

E[C22]

=2

21+

2p

12+

2p

2,

3.2. PHASE-TYPE APPROXIMATION 19

so that we have E[C22] 32E[C2]2. This gives SCV = E[C22 ]E[C2]2 1 12 . In general it holds that

the SCV of CK is greater or equal to1K , since the minimum SCV is obtained when we set

p1 = p2 = . . . = pK1 = 1 and 1 = 2 = . . . = K i.e. an Erlang K distribution for whichthe SCV = 1K .

In general the Coxian distribution is extremely important as any acyclic phase-type distri-bution has an equivalent Coxian representation, but this lies outside the scope of this thesis.

3.2 Phase-type approximation

Before we fit distributions by phase-type distributions we have to prove the validity of thesefits. In this section we prove that we can approximate any distribution with positive supportarbitrarily accurately by a mixture of Erlang distributions, see Tijms (1994) [23]. Furthermore,we prove that a certain type of distributions, i.e. with a completely monotone density, canalso be approximated arbitrarily accurately by a mixture of exponential distributions, the socalled hyperexponential distribution. The main idea of this proof can be found in Feller [13],however, here we modified the proof to get the required result.

Theorem 3.5. Let F (t) be the cumulative distribution function of a positive random variablewith possibly positive mass at t = 0 i.e. F (0) > 0. For fixed > 0 define the probabilitydistribution function

F(t) =K=1

pK()

1K1j=0

et

(t

)jj!

+ F (0), t 0,where pK() = F (K) F ((K 1)),K = 1, 2, . . . . Then

lim

F(x) = F (x)

for any continuity point x of F (x), (i.e. pointwise convergence).

Proof. Let , x > 0 fixed and U,x be a Poisson distributed random variable with

P [U,x = j] = ex

(x

)jj!

, j = 0, 1, . . . .

We have

E[U,x] =j=0

ex

(x

)jj!

j = xj=1

ex

(x

)j1(j 1)! = x,

Var[U,x] = E[U2,x] x2 = xj=1

ex

(x

)j1(j 1)!j x

2

= x2j=2

ex

(x

)j2(j 2)! + x

j=1

ex

(x

)j1(j 1)! x

2 = x.

3.2. PHASE-TYPE APPROXIMATION 21

Proof. The statements follows from the fact that mixture Erlang distribution is a special caseof a Coxian distribution, which we prove now. Let n N and Z = nK=1 pKEK with pK > 0and

nK=1 pK = 1 in which EK has an Erlang-K distribution with parameter . The unique

Laplace-Stieltjes transform of the Erlang mixture Z is given by

Z(s) =n

K=1

pKEK(s),

where EK is the Laplace-Stieltjes transform of EK which is(

+s

)K. On the other hand we

have a Coxian distribution Z defined as

Z =

Y1 w.p. (1 q1)Y1 + Y2 w.p. (1 q2)q1Y1 + Y2 + Y3 w.p. (1 q3)q2q1

...n1i=1 Yi w.p. (1 qn1)

n2i=1 qin

i=1 Yi w.p.n1i=1 qi

in which Yi Exp(i) and the probabilities sum up to 1. Hence Z is a mixture of randomvariables and define

EK =Ki=1

Yi with Laplace-Stieltjes transform EK(s) =

Ki=1

ii + s

.

So that

Z (s) =n1K=1

(K2i=1

qi(1 qK)EK(s))

+n1i=1

qjEn(s).

Now, we set i = for all i and find

Z (s) =n1K=1

K2i=1

qi(1 qK) pK

EK(s)

+n1i=1

qj pn

En(s) =

nK=1

pKEK(s) = Z(s).

By uniqueness of the Laplace-Stieltjes transform the statement is proven.

The hyperexponential distribution can also be used to approximate arbitrarily accuratelya specific class of distributions. This class should have a probability density function whichis completely monotone. There are several distributions, which are completely monotone,such as the exponential, hyperexponential, Weibull and Pareto distribution. The Weibulldistribution is seen in some healthcare settings, see Babes and Sarma (1991) [4]. Before weprove the statement above in a theorem we present the Extended Continuity Theorem, whichrelates Laplace-Stieltjes transforms to measures in limits. First, we give the definition of acompletely monotone function.

Definition 3.7. A probability density function f is completely monotone if all derivativesof f exist and satisfy

(1)nf (n)(t) 0 for all t > 0 and n 1.


Theorem 3.8. If f(t) and g(t) are completely monotone functions then the positive linearproduct of these two is functions is completely monotone. Furthermore the product of twocompletely monotone functions is completely monotone.

Proof. Let a, b > 0 then h(t) = af(t) + bg(t) is completely monotone by linearity. For thesecond statement let h(t) = f(t)g(t), so that by Leibniz rule we have

nh(t)

tn=

ni=0

(n

i

)f (i)(t)g(ni)(t),

it follows then by completely monotonicity of functions f and g.

Theorem 3.9 (Extended Continuity Theorem). For n = 1, 2, . . . let Gn be a measure withLaplace-Stieltjes transform Fn. If Fn(x) F (x) for x > x0, then F is the Laplace-Stieltjestransform of a measure G and Gn G. Conversely, if Gn G and the sequence {Fn(x0)}is bounded, then Fn(x) F (x) for x > x0.Proof. See Feller (1967) [13].

Theorem 3.10. Let F (t) be a probability distribution function with a completely mono-tone probability density function f(t) then there are hyperexponential distribution functionsFm, m 1 of the form

Fm(t) = 1Mmn=1

pmnemn t, t 0,

with ni andMm

n=1 pmn = 1, where pmn > 0 for all i, such that

limmFm(t) = F (t), for all t > 0,

i.e. uniform convergence.

Proof. Let F (t) the probability distribution function and consider F (aay) for fixed a > 0 andvariable y (0, 1). By Taylor expansion around y = 0, by completely monotonicity of f allthe derivatives exist for all y [0, 1), we have

F (a ay) =n=0

(a)nFn(a)n!

yn = F (a)n=0

a

n+ 1

(a)nfn(a)(n!

yn+1

which holds for y [0, 1). We change our variable y to x (0,) by y = ex/a, so that

Fa(x) = F (a aexa ) = F (a)n=0

a(a)nfn(a)

(n+ 1)!

n+1ax

= F (a)n=1

Ca(n)enax,

which is the Laplace transform of an arithmetic measure G(z) giving mass of pn = Ca(n) > 0(by completely monotonicity) to the points z = na for n = 1, 2, . . .. In detail, we have that

1 Fa(x) = F (a)

0Ca(n)e

zx dGa(z) = F (a)n=1

pa(n)enax 0.

3.3. PHASE-TYPE FIT 23

We observe that for any x

Fa(x) F (x).Applying the Extended Continuity Theorem 3.9 gives the existence of a measure G(z)

Ga(z) G(z)

with the cumulative distribution function F (x) =

0 ezx dG(z) as it is Laplace-Stieltjes

transform.

So, if F (t) is a cumulative distribution function with a completely monotone probabilitydensity function f(t) then there are hyperexponential cumulative distribution functions Fm(t)of the form:

Fm(t) = 1Mmn=1

pmnemn t,

which converge to F (t) for all t > 0 if m and Mn tend to infinity. Observe that Fm is indeeda hyperexponential distribution function for m N.

3.3 Phase-type fit

In practice it often occurs that the only information of random variables that is availableis their mean and standard deviation, based on data only. On basis of these two quantitiesone can fit (approximate) its underlying distribution by a phase-type distribution. The onlycondition is that the random variable for which we approximate its distribution must be pos-itive, sometimes completely monotone. The syllabus by Adan and Resing (2002) [1] suggestsspecific distributions for this fitting purpose.

Let X be a positive random variable and (S)CV its (squared) coefficient of variation. Incase 0 < CV < 1 one fits an Erlang mixture distribution that is with probability p it is anErlang K1 and with 1p an Erlang K, in shorthand notationEK1,K(; p). The parametersare given by

1

K SCV 1

K 1 ,

for K = 2, 3, . . . . Secondly we choose p and such that

p =1

1 + SCV

(K SCV

K(1 + SCV )K2SCV

), =

K pE[X]

.

then the EK1,K distribution matches its expectation E[X] and coefficient of variation CV .Because of the fact that we use an Erlang mixture model with the same scale parameter,

but different shape parameters, we have that the coefficient of variation is always less thanone. To get a coefficient of variation greater than 1 we have to vary the scale parameter aswell. The hyperexponential distribution is the simplest case with different scale parameters.So in case CV 1 we choose to fit a hyperexponential distribution with parameters p1, p2, 1and 2 in shorthand notation H2(p1, p2;1, 2). We have four parameters to be estimated, sowe set p2 = (1 p1), so that we do not have an atom at zero. Furthermore, we use balancedmeans

p11

=p22


then the probabilities are given by

p1 =1

2

(1 +

SCV 1SCV + 1

)and p2 =

1

2

(1

SCV 1SCV + 1

)

with rates

1 =2p1E[X]

and 2 =2p2E[X]

.

If SCV = 1 then it reduces p1 =12 = p2 and 1 = 2, which is equivalent with the

exponential distribution. And when SCV then p1 1.In case that SCV 12 also a Coxian-2 distribution can be used with the following param-

eters

1 =2

E[X], p =

1

2SCV, 2 = 1p.

Remark that we use only two moments to fit the data, so that we do not match skewnessand kurtosis of the particular distribution. On the other hand there is enough freedom tomatch more moments when we use more general phase-type distributions for fitting. However,choosing a parsimonious model is more practical and in most cases sufficient.

3.4 Recursive procedure for computing sojourn times

Up till now we have studied phase-type distributions as an approximation tool. The reasonfor this is that in the general D/G/1 case we cannot compute the waiting and idle times. Thesolution for this to translate the D/G/1 setting to a D/PH/1 setting by approximating thegeneral service time distributions by phase-type distributions.

Wang introduced in 1997 [27] a recursive procedure on the calculation of the sojourn timeof customers on a stochastic server. We can translate this to our scheduling problem in whicha doctor is seeing patients. Wangs procedure makes use of an attractive property of phase-type distributions mentioned in Section 3.1. In this section we will explain Wangs approachin detail. First, we show the approach for the specific case of exponentially distributed servicetimes. Second, we explain it for phase-type distributed service times, in which the phase-typedistributions are allowed to differ among patients, which is a further generalization of Wangsresult.

3.4.1 Exponentially distributed service times

In this section we present the iterative procedure for exponentially distributed service times,where the service times are not necessarily identical. The procedure gives a good idea for thenext section where the service times are phase-type distributed. From now on, 1 denotes acolumn vector of ones of appropriate size.

Suppose we have a service order 1, . . . , N + 1. We are interested in the sojourn timedistribution

FSi(t) = P [Si t] , t 0for all (patients) i {1, . . . , N + 1}. Let pi,k(t) the probability that patient i sees k patients infront of him after t units of time after his arrival ti, so that the case where k = 0 corresponds

3.4. RECURSIVE PROCEDURE FOR COMPUTING SOJOURN TIMES 25

to that the patient i is served, so that

P [Si t] = 1n1k=0

pi,k(t).

Define pi = (pi,i1, pi,i2, . . . , pi,0) a row vector of dimension i so that

P [Si t] = 1 pi1.

The first patient, who will be served directly, has infinitesimal generator

Q1 =

(0 01 1

)and submatrix

S1 = 1.Then we have, by definition of phase-type distributions,

p1 = p1,0 = e1t and FS1(t) = 1 e1t, t 0.

The second patient, who arrives at time x1, will find either no patient in the system withprobability p1(x1) or the first patient is still in the system with probability 1p1(x1). The firstpatient follows an exponential distribution with parameter 1. Because of the memorylessproperty of exponential random variables, the waiting time of the second patient is alsoexponentially distributed with parameter 1. So that its sojourn time is governed by thecontinuous-time Markov chain with infinitesimal generator

Q2 =

0 0 00 1 12 0 2

.Let S2 be the submatrix

S2 =

(1 10 2

),

then p2(t) satisfies the following system of differential equations

dp2(t)

dt= p2(t)S2 with p2(0) = (p1(t), 1 p1(t)1)

for which the solution is given by

p2(t) = (p1(x1), 1 p1(x1)1)eS2t = (e1x1 , 1 e1x1)eS2t t 0.

Since the Markov chain has an acyclic structure the transient solution of pi can be derivedby a system of differential equations. In general, for the i-th patient

Qi =

(0 0S0i Si

),


where Si is the submatrix

Si =

1 1 0 . . . 0

0 2 2 . . ....

.... . .

. . .. . . 0

0 0 n1n10 . . . 0 0 b

,

then pi(t) = (pi1(xi1), 1 pi1(xi1)1)eSit for t 0, since it is the solution ofdpi(t)

dt= pi(t)Si with pi(0) = (pi1(xi1), 1 pi1(xi1)1).

We point out that the time for the i-th patient starts running when he arrives. Then heeither waits or is served directly. The corresponding interarrival time xi is used in the initialcondition for the subsequent patient. We summarize the above procedure in a proposition.

Proposition 3.11. If the interarrival times are xi for i = 1, 2, . . . , N then the sojourn timedistribution of the i-th (i = 1, 2, . . . , N + 1) patient is given by

Fi(t) = P [Si(t) t] = 1 pn1,

where

p1 = eS1t,

pi =(pi1(xi1), 1 pi1(xi1)

)eSit for i = 2, 3, . . . , N + 1.

Furthermore, we have

E[S1] =1

1,

E[Si] =(pi1(xi1), 1 pi1(xi1)

)i1

j=01

iji2j=0

1ij

...1i

for i = 2, 3, . . . , N + 1.This proposition is proved by induction. The expression for mean sojourn times follows

from the properties of phase-type distributions. Furthermore, we observe that each iterationthe dimension of pi(t) increases by 1. The underlying continuous-time Markov chain is ob-served at the epochs of arrival times, ti. At these points the states of the Markov chain Yn(t)are defined as the number of patients waiting in the system. In case there are no patientswaiting the arriving patient is in service. We remark that only the first i 1 patients affectthe sojourn time of patient i.

3.4.2 Phase-type distributed service times

In this section we generalize the recursive procedure from the latter section to phase-typedistributed service times. We use the article by Wang (1997) to describe the procedure.


He assumes independent and identical phase-type distributed service times, where S of thephase-type representation has an acyclic structure. We extend on this by varying the (acyclic)phase-type distributions among patients. In detail, let patient i have a phase-type distributedservice time distribution, with probability vector i (dimension mi1) and mimi-matrix Si.

Define now the bivariate process {Yi(t),Ki(t), t 0} for patient i = 1, . . . , N + 1, whereYi(t) N is representing the number of patients in front of the i-th patient and Ki(t) Nis the particular phase in which the service is in if the server is busy, otherwise Ki(t) = 0.Furthermore, the state (0, 0) is the absorbing state and all other states are transient for every

patient i. Let p(i)j,k(t) the probability that {Yi(t),Ki(t), t 0} is in state (j, k) (j patients

before him and the server is in phase k)

p(i)j,k(t) = P [(Yi(t),Ki(t)) = (j, k)] .

pi(t) =(p

(i)i1,1(t), . . . , p

(i)i1,mi(t), p

(i)i2,1(t), . . . , p

(i)i2,mi1(t), . . . , p

(i)0,1(t), . . . , p

(i)0,m1

(t))

i = (i,1, i,2, . . . , i,mi)

For the first patient, at t1 = 0, it holds that there is no patient in the system, so that thefirst patient is phase-type distributed with transition matrix S1 and 1

p1(t) = (p10,1, . . . , p

10,m1) = 1e

S1t F1(t) = 1 p1(t)1.Hence P[Si t] = 1pi(t)1. Then for the next patient, who is phase-type distributed (2,S2)and arrives at t2 = x1, there are two cases:

The process {Y2(t),K2(t), t 0} will start at state (1, k) with probability p10,k(x1),where k = 1, 2, . . . ,m1.

The process {Y2(t),K2(t), t 0} will start at state (0, k) with probability 2,kF1(x1),where k = 1, 2, . . . ,m2.

So that the sojourn time of the 2-nd patient is governed by a continuous-time Markov chainwith infinitesimal generator

Q2 =

0 0 00 S1 S012S02 0 S2

,with initial-state distribution (0,p1(x1),2F1(x1)). So that if we let

S2 =(S1 S

012

0 S2

)then the sojourn time of the 2-nd patient is given by the following system of differentialequations

dp2(t)

dt= p2(t)S2,

with initial condition p2(0) = (p1(x1),2F1(x1)). The unique solution is

p2(t) = (p1(x1),2F1(x1))eS2t, t 0.

In general, for patient i, who arrives at ti (just after the interarrival time xi1):


The process {Yi(t),Ki(t), t 0} will start at state (i1, k) with probability pi1i2,k(xi1),where k = 1, 2, . . . ,m1

The process {Yi(t),Ki(t), t 0} will start at state (i2, k) with probability pi1i3,k(xi1),where k = 1, 2, . . . ,m2

...

The process {Yi(t),Ki(t), t 0} will start at state (1, k) with probability pi10,k (xi1),where k = 1, 2, . . . ,mi1

The process {Yi(t),Ki(t), t 0} will start at state (0, k) with probability pi10,k (xi1) =i,kFi1(xi1), where k = 1, 2, . . . ,mi

So that the sojourn time of i-th patient is governed by a continuous-time Markov chain withinfinitesimal generator

Qi =

0 0 0 0 . . . 00 S1 S

012 0 . . . 0

0 0 S2. . . . . . 0

......

. . .. . . S0i2i1 0

0 0 . . . 0 Si1 S0i1iS0n 0 . . . 0 0 Si

,

with initial-state distribution (0,pi1(xi1),iFi1(xi1)) and submatrix

Si =

S1 S

012 0 . . . 0

0 S2. . . . . . 0

.... . .

. . . S0i2i1 00 . . . 0 Si1 S0i1i0 . . . 0 0 Si

.

So that the sojourn time of the i-th patient is given by the following system of differentialequations

dpi(t)

dt= pi(t)Si

with initial condition pi(0) = (pi1(xi1),iFi1(xi1)). The solution is given by

pi(t) = (pi1(xi1),iFi1(xi1))eSit, for t 0.

So for finite number of patients we can compute the individual sojourn time distributions.We summarize this recursive procedure and state it as a proposition.

Proposition 3.12. If the interarrival times are xi for i = 1, 2, . . . , N then the sojourn timedistributions Fi(t) for (patient) i = 1, 2, . . . , N + 1 are given by

Fi(t) = P[Si t] = 1 pi(t)1


where

p1(t) = 1eS1t, S1 = S1

pi(t) = (pi1(xi1),iFi1(xi1))eSit for i = 2, 3, . . . , N + 1.

The mean sojourn time is E[Si] = (pi1(xi1),iFi1(xi1))S1i 1.This proposition can be proved by induction. The expression for the mean sojourn times

follow directly from the properties of phase-type distributions, see Section 3.1. Observe thatpi relies completely on pi1 and its dimension is expanded from

i1j=1mj to

ij=1mj , the

phases of the new patient are added to continuous-time Markov chain. Also, the distributionof the i-th patient is a function of x1, . . . , xi1 only, because of the first-come, first-serveddiscipline. When all phase-type distributions are exponential distributions we are in thesimple case described extensively in the previous subsection.

In this chapter we introduced phase-type distributions and proved their ability to approxi-mate distributions with a positive support arbitrarily accurately. After which we gave explicitand easy-to-use formulas to fit distributions roughly. In order to do so, we divided distributionfunctions into two categories:

Distribution functions with a SCV 1 are fitted by a EK1,K(; p) distribution. Distribution functions with a SCV 1 are fitted by a H2(1, 2; p1, p2) distribution.

Furthermore, we demonstrated a recursive procedure for phase-type distributions to computethe patients individual sojourn time distributions. This procedure will be used in the nextchapter for optimization.

Chapter 4

Optimization methods

In this chapter we obtain optimal schedules in different settings. The fact that the distributionfunctions can be approximated by phase-type distributions gives us the opportunity to usethe recursive procedure described in the latter chapter. So the first step is to approximate theservice times by an appropriate phase-type distribution. The second step is to implement therecursive procedure on these phase-type distributed service times to find the sojourn times.Third, we optimize over these sojourn times to find optimal schedules.

Since optimizing simultaneously for all patients is highly complex, we use numerical meth-ods to compute optimal interarrival times. Further, we compare the simultaneous approachwith the sequential counterpart introduced by Kemper et al. (2011). This approach aroseas a trade-off between computational time and a sufficiently close-to-optimal schedule. Vinket al. (2012) introduced the so called lag-order method, which spans all trade-offs betweenthe sequential and simultaneous approach. We will describe the idea behind the lag-ordermethod for the sake of completeness in Section 4.3.

Observe that the optimization problem is over t2, . . . , tN+1. By the relation between tisand xis defined by ti =

i1j=1 xj , see Chapter 2, the problem is equivalent with minimiz-

ing over the interarrival times x1, . . . , xN only. Let us first describe different optimizationmethods.

4.1 Simultaneous optimization

In the classical case one minimizes the systems loss over all possible schedules. This meansthat all patients are scheduled simultaneously, so that we minimize as follows:

mint2,...,tN+1

R(t2, . . . , tN+1) = minx1,...,xN

R(x1, . . . , xN )

= minx1,...,xN

N+1i=2

E [l(Si1 xi1)]

= minx1,...,xN

Ni=1

E [l(Si xi)] , (4.1)

where l(x) is convex function. To optimize simultaneously there is almost no tractable deriva-tion possible. Only the exponential case has a tractable solution, see Wang (1997). Thedifficulty is that the xis are too much interlinked. The choice of an optimal xi depends

30

4.2. SEQUENTIAL OPTIMIZATION 31

implicitly on the previous optimal interarrival times xi1, . . . , x1. This is because the arrivalsof preceding patients influence the waiting and idle times of patient i.

Moreover, the fact that phase-type distributions have no nice representations makes com-putations intractable. Therefore in this case one wants to uses numerical algorithms to findthe optimum interarrival times, see the appendix for such an outline of such an algorithm. Asolution which leads to tractable solution is to consider the problem sequentially as we willdo so in the next section.

4.2 Sequential optimization

A solution to the complexity of optimizing simultaneously is to consider the optimizationproblem sequentially. This approach is introduced by Kemper et al. (2011) [16]. If oneoptimizes the schedule sequentially, one minimizes for all i {2, . . . N + 1}, given that youknow ti1, ti2 . . . , t1

mintiR(ti, ti1, . . . , t1) = min

tiE [g(Ii) + h(Ii)]

= minxi1

E [l(Si1 xi1)] . (4.2)

In this formula Si1 depends implicitly on the previous arrival epochs ti1, . . . , t1 or equiv-alently xi2, . . . , x1, but these are fixed, since we consider the optimization problem sequen-tially. This reduces the problem drastically, such that there is an optimal schedule undersome conditions, see Theorem 4.1 below, from Kemper (2011) [16].

Theorem 4.1. Let l : R R0 be a convex function with l(0) = 0. Let B1, . . . , BN+1 beindependent non-negative random variables such that

E

[l

(Ni=1

Bi + y

)]

32 CHAPTER 4. OPTIMIZATION METHODS

Because of convexity, l(xj) is monotone and the non-negativity of l(xj) imply that for alla b b

a

l(xj) dxj l(b) + l(a).By Fubinis Theorem we have b

aE[l(Sj xj)] dxj = E [ b

a

l(Sj xj) dxj] ,combining this with the results above we find b

aE[l(Sj xj)] dxj E [l(Sj b)] + E [l(Sj a)]

so that E[l(Sj xj)] is absolutely continuous with derivative E[l(Sj xj)] and thereforeconvex. Hence there exists a minimum, since l(x) R0. Moreover, E[l(Sj xj)] is non-increasing in xj and non-negative at xj = 0 i.e. E[l(Sj)] 0 always. Therefore xj is non-negative j.

Since weighted linear and quadratic cost functions satisfy the conditions of Theorem 4.1,we can derive optimal interarrival times, which we present here as examples.

4.2.1 Quadratic loss

Consider Ri the weighted quadratic cost function, c.f. equation (2.8). We optimize sequen-tially, so one has to minimize the following function over xi given xi1, . . . , x1

Ri(xi, . . . , x1) = E [l(Si xi )] = E[ (Si xi)2 1{Sixi

4.3. LAG-ORDER METHOD 33

Suppose the special case that = 12 (waiting and idle times are equally weighted) then thesolution reduces to

xi = ESi.

Thus the optimal interarrival times are equal to the mean of the (corresponding) sojourntimes.

4.2.2 Absolute loss

Consider Ri the weighted absolute loss function, c.f. equation (2.6). Optimizing sequentiallymeans that we minimize the following function over xi given xi1, . . . , x1

Ri(xi, . . . , x1) = E [l(Si xi )] = E [ |Si xi|1Sixi0]

By Fubinis Theorem the latter can be rewritten

E [l(Si xi )] = xi

0

xis

dtdFSi(s) +

xi

sxi

(1 ) dtdFSi(s)

(Fubini) =

xi0

t0 dFSi(s)dt+

xi

t

(1 ) dFSi(s)dt

=

xi0

FSi(t) dt+ (1 ) xi

(1 FSi(t)) dt.

Now, using Theorem 4.1 one obtains xi by solving

E[l(Si xi )

]= FSi(x

i ) (1 ) (1 FSi(xi )) = 0 xi = F1Si (1 ) .

So in this example the optimal interarrival times are quantiles of the (corresponding) sojourntime distributions.

4.3 Lag-order method

In this section we discuss the lag-order method introduced by Vink et al. (2012) [25]. Thekey observation is that patient is waiting time depends on all preceding interarrival times.However, we can restrict the influence of preceding interarrival times which are far away frompatient is, since interarrival times further away from xi affect the xi less. Hence, we considerfor all i = 1, . . . , N

minxi

Ri(xi, xi1, . . . , xik) = minxiE [l(Si(xi1, . . . , xik) xi)] , (4.3)

instead of

minxi

Ri(xi, xi1, . . . , x1) = minxiE [l(Si(xi1, . . . , x1) xi)] .

This procedure is an optimization method which contains every compromise between op-timizing sequentially, k = 1, to optimizing simultaneously, k = N 1. For the practicalconsideration of this approach we refer to the paper by Vink et al.


4.4 Computational results for transient cases

In this section we evaluate the simultaneous and sequential optimization for three differ-ent SCV values where we set the mean equal to one. We choose similar SCV values asin Wang (1997) [26]. Herein, Wang chooses (as an example) a Coxian-3, exponential andCoxian-2 distribution to model a SCV = 0.7186, SCV = 1 and a SCV = 1.6036. We use ourmoment matching procedure, see Section 3.3, to match mean one and the above SCV values.We summarize our parameters.

To model a SCV = 0.7186 Wang uses a Coxian distribution with three phases, namely,(1, 2, 3) = (

43 ,

83 , 4) and (p1, p2) = (

12 ,

12) so that = (1, 0, 0) and

S =

43 23 00 83 430 0 4

We model the same SCV by an EK1,K(; p) distribution with parameters K = 2, = 1.60026 and p = 0.399744. I.e = (1, 0) and

S =

(1.60026 0.9605630 1.60026

).

SCV = 1 Wang uses an exponential distribution with = 1 to match this SCV , byusing the phase-type fit we get exactly the same distribution.

To model a SCV = 1.6036 Wang uses a Coxian distribution with only two phases,(1, 2) = (1.3, 0.4333) and p1 = 0.1, so that = (1, 0) and

S =

(1.3 0.130 0.4333

)We model this SCV by a H2(1, 2; p1, p2) distribution with parameters chosen by ourmatching method. In phase-type notation = (p1, p2) = (0.740745, 0.259255) and

S =

(1.48149 00 0.51851

).

In the following figures we present our computations of optimal schedules in terms of inter-arrival times xi for

different number of patients N + 1, where N = 5, 10, 15, 20, 25, different SCV s: SCV = 0.7186, 1, and 1.6036, to match the paper of [26] the chosen phase-type distributions of Wang and our phase-type fit to match the cor-

responding SCV s,

equally weighted linear and quadratic cost functions, simultaneous (Figures 4.1, 4.2 and 4.3) and sequential optimization (Figure 4.4).

4.4. COMPUTATIONAL RESULTS FOR TRANSIENT CASES 35

Wang computed in his article optimal schedules for N = 10 by simultaneous optimizationover a linear cost function. Kemper (2011) computed the optimal schedule for exponentiallydistributed service times, i.e. SCV = 1. We extend this by also computing optimal schedulesfor various SCV s. We find optimal values in the simultaneous case by the constraint optimizerin MatLab, i.e. fmincon. In case of sequential optimization we use the command fsolve.In Figures 4.1, 4.2 and 4.3 optimal schedules for different number of patients N+1 are plotted.We observe the following pattern in the interarrival times: they are low in the beginning ofthe schedule; then they seem to converge to a (maximum) value attained somewhere in themiddle, and then they seem to decrease in exact the opposite way as how they increased.These patterns are known as dome shapes, which are also observed in [25] and in [15].

In Figure 4.2(b) we compute the optimal interarrival times for simultaneous optimizationwith the quadratic costs. This can be compared with the paper by Vink [25], where hecomputed in the exact same setting the optimal schedule for N = 10.

If N increases we see that the middle parts of the schedules are converging to a maximumvalue. We will compute these maxima in Chapter 5, since they are the solution of the steady-state. In all figures we see that quadratic costs lead to higher interarrival times, i.e. less tightschedules. In all figures we observe that the difference between the optimal schedule withWangs parameter choice and the phase-type fit of Section 3.3 is extremely small. This givesus an indication that the phase-type fitting procedure grasps the general characteristics well.

In Figure 4.4 we compute optimal schedules by sequential optimization. We see a hugedifference between these schedules and the optimal schedules derived with simultaneous opti-mization. This seems reasonable since the optimization method is completely different. Thedome shape pattern is replaced by a single increase in interarrival times. The interarrivaltimes seem to converge to a maximum as the slope is decreasing.

Comparing the schedules in Figure 4.4 for different SCV s, we see that in the linear case, (a)and (c), higher SCV s lead to lower interarrival times in the beginning and greater interarrivaltimes at the end of the schedule, so that the graphs intersect. In Figure 4.4(b) and (d) wesee that the graphs with higher SCV s have greater interarrival times always.


5 10 15 20 25

1

1.2

1.4

1.6

1.8

2

i

xi

(a) Linear costs, phase-type fit.

5 10 15 20 25

1

1.2

1.4

1.6

1.8

2

i

xi

(b) Quadratic costs, phase-type fit.

5 10 15 20 25

1

1.2

1.4

1.6

1.8

2

i

xi

(c) Linear costs, Wangs parameters.

5 10 15 20 25

1

1.2

1.4

1.6

1.8

2

i

xi

(d) Quadratic costs, Wangs parameters.

Figure 4.1: the optimal schedules in xis by simultaneous optimization for SCV = 0.7186 < 1.

5 10 15 20 25

1

1.2

1.4

1.6

1.8

2

i

xi

(a) Linear costs.

5 10 15 20 25

1

1.2

1.4

1.6

1.8

2

i

xi

(b) Quadratic costs.

Figure 4.2: the optimal schedules in xis by simultaneous optimization for SCV = 1. Theparameter choice for both Wangs method as our phase-type fit is the same.

4.4. COMPUTATIONAL RESULTS FOR TRANSIENT CASES 37

5 10 15 20 25

1

1.2

1.4

1.6

1.8

2

i

xi


5 10 15 20 251.2

1.4

1.6

1.8

2

2.2

i

xi


5 10 15 20 25

1

1.2

1.4

1.6

1.8

2

i

xi


5 10 15 20 251.2

1.4

1.6

1.8

2

2.2

i

xi


Figure 4.3: the optimal schedules in xis by simultaneous optimization for SCV = 1.6036 > 1.


2 4 6 8 10 12 14

0.6

0.8

1

1.2

1.4

1.6

i

xi

SCV = 1.6036SCV = 1SCV = 0.7186


2 4 6 8 10 12 14

1

1.2

1.4

1.6

1.8

2

i

xi

SCV = 1.6036SCV = 1SCV = 0.7186


2 4 6 8 10 12 14

0.6

0.8

1

1.2

1.4

1.6

i

xi

SCV = 1.6036SCV = 1SCV = 0.7186


2 4 6 8 10 12 14

1

1.2

1.4

1.6

1.8

2

i

xi

SCV = 1.6036SCV = 1SCV = 0.7186


Figure 4.4: the optimal schedule in xis by sequential optimization for different SCV s.

Chapter 5

Limiting distributions

In Chapter 4 we observed some convergence of the optimal interarrival times in sequential andsimultaneous optimization seem to converge, see Section 4.4. In particular, in simultaneousoptimization we saw the optimal times in the middle of the schedule converge to the samevalue when N increased. In sequential optimization we noticed the convergence of the (right)tail of the optimal times. This indicates that in both cases there is some underlying limit xto which the optimal interarrival times xi converge as N .

An approach to find these limits is to let the number of patients tend to infinity, butnumerical optimization is not doable in this setting, since the computation time will takeinfinitely long. That is why we focus on other ways to derive the limit interarrival times inthis chapter.

We start off with the derivation of interarrival times in case of exponential service times.More specifically we look at the steady-state of the D/M/1 queue under sequential and simul-taneous optimization. For this case in particular we can derive the limit interarrival times xanalytically. For hyperexponential and mixture Erlang service times we are unable to deriveanalytic results. For these phase-type distributed service times we present a method basedon the embedded Markov chain.

In sequential optimization we solve

E[l(S x)]x

= 0,

where we have found explicit formulas for (equally weighted) linear and quadratic cost func-tions in Chapter 4, that is

for linear costs (absolute loss) x = F1S(

12

);

for quadratic costs (quadratic loss) x = E[S].

For the limit interarrival times in the setting of simultaneous optimization, we argue thatthese times are equidistant in the limit. Furthermore, the sojourn time distribution dependson x, so that for N large

R =N+1i=1

Ri (N + 1)Ri = (N + 1)E [l(S(x) x)] .

39

40 CHAPTER 5. LIMITING DISTRIBUTIONS

Hence, minimizing R is equivalent with, see [25]

E [l(S(x) x)]x

= 0.

We found that we have to minimize

for linear costs (absolute loss) Ri = E[|S(x) x|] or Ri = E[S(x)] + x of (2.10); for quadratic costs (quadratic loss) Ri = E[(S(x) x)2] = E[S(x)2] + 2xE[S(x)] + x2.

By equation (2.11) we know that minimizing Ri or Ri result in the same optimal schedule

for linear costs, which naturally also holds in steady-state.

5.1 The D/M/1 queue

Suppose the service times are exponentially distributed with parameter . In the syllabusQueueing Theory by Adan and Resing (2002), and Tijms (1986) [1, 22] the limit distributionof the sojourn time is derived for the G/M/1 queue

P[Si t] = 1 e(1)t, t 0, where solves = e()x.We point out that is not equal to the occupation rate , since we do not have a Poissonarrival process.

5.1.1 Limit solutions in the sequential case

When we look at the sequential optimization case, we have derived expression for both theabsolute and the quadratic cost functions, which can be used to find the limit interarrivaltimes analytically. First, using the fact that in the linear costs case we have xi = F

1Si

(12).We find

x = F1Si

(1

2

)=

log 2

(1 ) and FSi(x) =1

2= 1 .

So that for absolute cost we have

x =2 ln(2)

1.3862

.

In case of quadratic cost, we have that x = E[Si] = 1(1) and = e()x, therefore

x =e

(e 1) 1.5820

.

5.1.2 Limit solutions in the simultaneous case

For the other case we minimize xi also over the xi-dependent sojourn time distributions. Todo so we observe that in the transient case the number of interarrival times with the samesize increases, which can be seen in the figures in Section 4.4. So that, when n, we havethat xi = x and Si = S for all patients i so for the linear cost case

limN

Ni=1

E [|Si(xi) xi|] = limN

NE[|S(x) x|],

5.1. THE D/M/1 QUEUE 41

and for the quadratic cost case

limN

Ni=1

E[(Si(xi) xi)2

]= lim

NNE[(S(x) x)2].

We want to minimize both expressions, this is done by taking the derivative and setting itequal to zero

d

dxE[|S(x) x|] = d

dx

( x

(t x)fS(x)(t) dt+ x

0(x t)fS(x)(t) dt

)= 0, (5.1)

d

dxE[(S(x) x)2] = d

dx

(E[S(x)2] 2xE[S(x)] + x2) = 0. (5.2)

It is known for the G/M/1 queue that

FSi(x)(t) = 1 e(1x)t,fSi(x)(t) = (1 x)e(1x)t.

Using this we find for equation (5.3) that

d

dx

( x

(t x)fS(x)(t) dt+ x

0(x t)fS(x)(t) dt

)=

d

dx

1 2e(1x)x (1 x)x(x 1)

=d

dx

1 2x + log x(x 1)

= x1 + (log x 2)x(x 1)2x

=1 + (log x 2)x

(1 x + x log x)= 0,

where we used for x that

x =x(x 1)

1 xx , (5.3)

which we found by implicit differentiation of the unique solution equation: x = e(1x)x

and equation (5.5). The numerator should equal zero, so

1 + (log x 2)x = 0 x 0.3178. (5.4)

Via the unique solution equation we can also express x in terms of x:

x =log x

(x 1) . (5.5)

Implementing the solution of equation (5.4) into (5.5) one finds for the absolute cost theoptimal interarrival times in the limit case

x 1.6803

.


Now we focus on quadratic costs, see equation (5.2), in order to find its limit interarrivaltimes, we compute

E [S(x)] = 1(1x) so thatdE[S(x)]

dx =x

(1x)2 ,

E[S2(x)

]= 2

2(1x)2 so thatdE[S2(x)]

dx =4x

2(1x)3 .

We fill in equation (5.2) with the derived results and equations (5.3) and (5.5)

(E[S(x)2] 2xE[S(x)] + x2) = 4x

2(1 x)3 21

(1 x 2xx

(1 x)2 + 2x

=4x(x 1)

2(1 xx)(1 x)3 +2 log x + 2

(x 1)+

2x log x(1 x + x log x)(x 1)

= 2

(1 + log x + x(1 + log x + (log )

2)

(1 x + x log x)(x 1))

= 0,

which is equivalent to setting the numerator equal to zero resulting in x 0.25. Implement-ing the solution for x in (5.5) gives us the limit interarrival time

x 1.8466

.

For phase-type distribution there is no clear derivation to find the limit distribution, sincethere is no similar limit representation of the sojourn time. However, we will focus on twophase-type distributions, proven to be important for fitting in Section 3.3, for which wederive a method to compute their limit distribution function. This enables us to computelimit interarrival times also for cases where SCV 6= 1.

5.2 The D/EK1,K/1 queue

The hyperexponential (H2) and mixture Erlang (EK1,K) distributions are often used inpractice to fit unknown distributions by means of their mean and standard deviation. In casethe data has a (S)CV 1 it is common to fit an EK1,K(; p) distribution. Its density isgiven by:

fB(t) = p(t)K2

(K 2)!et + (1 p) (t)

K1

(K 1)!et. (5.6)

In this section we study the D/EK1,K/1 queue. In this system patients arrive one byone with interarrival times which are identically and independently distributed with densityfA(t) = x(t), because arrivals are deterministic. The service times are mixture Erlangdistributed with density defined in equation (5.6). For stability we require that for theoccupation rate , it holds that

=1

E[A]E[B] =

1

x

(pK 1

+ (1 p)K

)< 1.

5.2. THE D/EK1,K/1 QUEUE 43

So that x is larger than the average service requirement.The state of the D/EK1,K/1 queue can be described by the pair (n, t), where n is the

total number of phases to be completed and t the elapsed time since the last arrival. To solvethis problem we need a two dimensional state description, where t is continuous. The amountof phases in the system upon arrival is exactly how much work is left before the arrivingpatient can be served. So we look at the system just before arrivals to make computationseasier. Let Nk the total number of phases in the system which are not yet completed justbefore the k-th arrival. The relation between Nk+1 and Nk is then given by

Nk+1 =

{Nk D1,k+1 +K 1 with probability pNk D2,k+1 +K with probability 1 p

where D1,k+1 and D2,k+1 are the number of phases completed between the arrival of the k-thand k + 1-th patient. There is a distinction between these two as in the latter case there isone more phase in the system. The sequence {Nk}k=0 forms a Markov chain, for which wewill compute the limit probabilities in the next section. These limit probabilities can be usedto derive the steady-state sojourn time distribution see Section 5.2.2.

5.2.1 The limiting probabilities

In this section we determine the limit distribution

am = limk

P [Nk = m] .

The limit probabilities am solve the equation

a = aP, (5.7)

where a = (a0, a1, a2, . . .). The transition probabilities from one state to another are givenby

pm,n = P [Nk+1 = n|Nk = m] ,which is a mixture of two Poisson processes:

pm,n = pP [Pois(x) = m n+K 1] + (1 p)P [Pois(x) = m n+K] . (5.8)

So that we distinguish four cases. First the transition probability to a state more than Kphases higher than the state you are in is 0. The state m + K from m can only be reachedif you are in the Pois(x) case which has probability (1 p). Thirdly, the states betweenthe 0 phase and K phases higher than your starting state can be reached by both Poissonprocesses. However, the latter process added K phases to the system instead of K1, so thatit has to complete one more phase. Finally, the probabilities should add up to one, so that theprobability of jumping to zero phases in the system is equal to 1 minus all the probabilitiesof moving to nonzero phases. We summarize

For n > m+K we have: pm,n = 0.For n = m+K we have: pm,n+K = (1 p)ex =: 0.For 0 < n < m+K we have: pm,n = p

(x)mn+K1(mn+K1)! e

x + (1 p) (x)mn+Kmn+K! ex =: mn+K .For n = 0 we have: pm,0 = 1

m+Kn=1 pm,n.


0 5 10 15 20 250

0.1

0.2

0.3

0.4

0.5

m

am

(a) Lin-lin scale.

0 5 10 15 20 25106

104

102

100

m

am

(b) Log-lin scale.

Figure 5.1: The am, probability of finding m phases to be completed, for m [0, 1, . . . , 25]on two different scales. Here, the mixture Erlang distribution is found by a phase-type fitof Section 3.3 where the SCV = 0.7186 and E[X] = 1, the interarrival time x = 1.3.

Now we write the matrix P of transition probabilities as

P =

p0,0 K K1 . . . 1 0 0 . . .p1,0 K+1 K K1 . . . 1 0 0 . . .p2,0 K+2 K+1 K K1 . . . 1 0 0p3,0 K+3 K+2 K+1 K K1 . . . 1 0

.... . .

.

Next we use equation (5.7) to compute the limit probabilities a. Since the dimension ofa is infinite, we are obliged to choose a certain cutoff point N , we solve(a0, a1, . . . , aN ) =(a0, a1, . . . , aN )P. Only in the case of exponential service times one can put in solutions ofthe form am =

m, see Adan and Resing [1]. Still, cutting off at a certain point seems legible,because by stability the limit probabilities am exponentially decay to zero for m > M , seeTijms [23] (page 288) and graphically in Figure 5.1. The kink in the end is because of cuttingthe probability vector off at N = 25.

5.2.2 The sojourn time distribution

In the last section we calculated the limit probabilities. These probabilities give exactly howmuch work there is left in the system upon arrival, i.e. the waiting time. So one can find thesteady-state sojourn time by the convolution formula

P [S t] = P [W +B t] = tu=0

P [W t u|B = u]P [B = u] du = tu=0

FW (tu)fB(u) dt,

where fW (tu) = a0 +N

m=1 am((tu))m1

(m1)! e(tu) and fB(u) is of course the service time

defined in equation (5.6). So that the expressions that are needed for computing the optimal

5.3. THE D/H2/1 QUEUE 45

interarrival times numerically are given by:

P [S t] = a0FB(t) +Nm=1

am

t0

((t u))m1(m 1)! e

(tu)fB(u) du,

E[S] = E[W ] + E[B]

=

Nm=0

am

(pm+K 1

+ (1 p)m+K

),

E[S2] = E[(W +B)2]= E[W 2] + 2E[W ]E[B] + E[B2]

=

Nm=1

amm(m+ 1)

2+ 2

Nm=1

am

(pm

K 1

+ (1 p)m+ 1

K

)+

(pK(K 1)

2+ (1 p)(K + 1)K

2

).

These expressions converge to the true values when N . We will see in Section 5.4 thatin our optimizations N = 10 +K is already enough.

5.3 The D/H2/1 queue

In this section we study the D/H2/1 queue. In this system patients arrive one by one withidentically and independently distributed interarrival times with density fA(t) = x(t), be-cause arrivals are deterministic. The service times are hyperexponentially distributed, whichare used when the data has (S)CV 1. It is common to fit an H2(p1, p2;1, 2) distributionin these cases. Its density is given by:

fB(t) = p11e1t + p22e2t, (5.9)

where p1 + p2 = 1 c.f. Example 3.3 An hyperexponential variable can be seen as a drawingbetween the random variables X1 Exp(1) and X2 Exp(2) with probabilities p1 and p2.For stability we require that for the occupation rate it holds that

=1

E[A]E[B] =

1

x

(p11

+p22

)< 1.

So that the interarrival time x is larger than the average service requirement.The state of the D/H2/1 queue can be described by the triplet (i,m, t), where i denotes

the number of patients in the system who

Appointment Scheduling in Healthcare

Documents

Transcript of Appointment Scheduling in Healthcare