.IJQUEUEING SYSTEMSVOLUME J: THEORYLeonard KleinrockProf essorComput er Science DepartmentSchool of Engineering and Applied Sci enceUniversity of California, Los AngelesA Wiley-Interscience PublicationJohn Wiley & SonsNew York Chiches te r > Brisbane Toronto" Ah, ' All thinas come to those who wait.'They come, but cjten come too late."From Lady Mary M. Curr ie: Tout Vient aQui Sait Attendre (1890)27947Copyr ight 1975, by John Wiley & Sons, Inc.All rights reserved. Publ ished simulta neously in Canada.Reproduction or translation of any part of this work beyond thatpermitted by Sections 107or 108of 'he 1976 United St ates Copy-ri ght Ac t wi thout the permission of the copyright owne r is unlaw-ful. Requests for permission or furthe r information should beaddressed to the Permissions Depart ment , John wi ley & Sons. Inc.Library of Congress Cat aloging in Publication Data:K leinrock, Leonard.Queueing systems."A Wiley-Iruerscience pu blicat ion."CONTE NTS: v . I. Th eor y.I. Que ueing theory. I. Title.T57.9. K6 5 19.8'2ISBN 0-47 1-49110- 113 141574-9846PrefaceHow much time did you waste waiting in line this week? It seems we cann otescape frequent delays, and they are getting progressively worse ! In thi s textwe study the phenomena of standing, waiting, and serving, and we call thisstudy queueing theory.Any system in which arrivals place demands upon a finite-capacity resourcemay be termed a queueing system. In particular, if the arri val times of thesedemand s are unpredictable , or if the size of these demands is unpredictable,then conflicts for the use of the resource will ari se and queu es of waitingcustomers will form. The lengths of these queue s depend upon two aspects ofthe flow pattern : first, they depend upon the average rate at which demandsare placed upon the resource; and second, they depend upon the statisticalflu ctuations of this rate. Certainly, when the average rate exceeds the capacity,then the system breaks down and unbounded queues will begin to form; it isthe effect of this average overload which then dominates the growth of queue s.However, even if the average rate is less than the system capacity, then here,too, we have the formation of queues due to the statistical fluctuati ons andspurts of arrivals that may occur; the effect of these var iations is greatlymagnified when the average load approaches (but does not necessarilyexceed) that of the system capacity. The simplicity of these queueing struc-tures is decepti ve, and in our studies we will often find ourselves in deepanalytic waters. Fortunately, a familiar and fundamental law of sciencepermeates our queueing investigations . This law is the conservation of flow,which states that the rate at which flow increases within a system is equal tothe difference between the flow rat e int o and the flow rate out of that system.Thi s observation permits us to write down the basic system equations forrath er complex structures in a relativel y easy fashion.The pu rpose of this book , then , is to present the theory of queue s at thefirst-year graduate level. It is assumed that the student has been exposed to afirst course in probabi lity theory; however, in Appendi x II of thi s text wegive a pr obability theory refresher and state the basic pr inciples that we shallneed. It is also helpful (but not necessary) if the student has had someexposure to transforms, alth ough in thi s case we present a rat her completeviiviii PREFACEtransform theory refresher in Appendix I. The student is advised to read bothappendices before proceeding with the text itself. Whereas our material ispresented in the language of mathematics, we do take great pains to give asinformal a presentation as possible in order to strike a balance between theabstractions usually encountered in such a study and the basic need forunderstanding and applying these tools to practical systems. We feel that asatisfactory middle ground has been established that will neither offend themathematician nor confound the practitioner. At times we have relaxed therigor in proofs of uniqueness , existence, and convergence in order not tocloud the main thrust of a presentation. At such times the reader is referred tosome of the other books on the subject. We have refrained from using thedull "theorem-proof" approach; rather, we lead the reader through a naturalsequence of steps and together we "discover" the result . One finds thatprevious presentations of this material are usually either too elementary andlimited or far too elegant and precise, and almost all of them badly neglect theapplications; we feel that the need for a book such as this, which treads theboundary inbetween, is necessary and useful. This book was written over aperiod of fiveyears while being used as course notes for a one-year (and later atwo-quarter) sequence in queueing systems at the University of California,Los Angeles. The material was developed in the Computer Science Depart-ment within the School of Engineering and Applied Science and has beentested successfully in the most critical and unforgiving of all environments,namely, that of the graduate student. This text is appropriate not only forcomputer science departments , but also for departments of engineering,operations research, mathematics, and many others within science, business ,management and planning schools .In order to describe the contents of this text, we must first describe thevery convenient shorthand notation that has been developed for the specifica-tion of queueing systems. It basically involves the three-part descriptorA/B/m that denotes an m-server queueing system, where A and Bdescribe theinterarrival time distribution and the service time distribution, respectively. Aand B take on values from the following set of symbols whose interpretationis given in terms of distributions within parentheses: M (exponential) ; ET(r-stage Eriangian); HR (R-stage hyperexponential); D (deterministic) ;G (general). Occasionally, some other specially defined symbols are used. Wesometimes need to specify the system's storage capacity (which we denote byK) or perhaps the size of the customer population (which we denote by M) ,and in these cases we adopt the five-part descriptor A/B/m/K/M; if either ofthese last two descriptors is absent, then we assume it takes on the value ofinfinity. Thus, for example, the system D/M/2/20 is a two-server system withconstant (deterministic) interarrival times, with exponentially distributedservice times, and with a system storage capacity of size 20.PREFACE ixThis is Volume I (Theory) of a two-volume series, the second of which isdevoted to computer applications of this theory. The text of Volume I(which consists of four parts) begins in Chapter I with an intr oducti on toqueuein g systems, how they fit into the general scheme of systems of flow,and a discussion of how one specifies and evaluates the performance of aqueueing system. Assuming a knowledge of (or after reviewing) the mater ialin Appendices I and II, the reader may then proceed to Chapter 2, where he iswarned to take care! Section 2.1 is essential and simple. However, Sections2.2, 2.3, and 2.4 are a bit "heavy" for a first reading in queueing systems, andit would be quite reasonable if the reader were to skip these sections at thi spoint, proceeding directly to Section 2.5, in which the fundamental birth-death process is introduced and where we first encounter the use of a-trans-forms and Laplace tran sforms . Once these preliminaries in Part I are estab-lished one may proceed with the elementary queueing theory presented inPar t II. We begin in Chapter 3 with the general equilibrium solut ion to birth-death processes and devote most of the chapter to providing simple yetimportant examples. Chapter 4 generalizes this treatment , and it is herewhere we discuss the method of stages and provide an intr oduction to net-works of Mar kovian queues. Whereas Part II is devoted to algebraic andtr ansform oriented calculati ons , Part III returns us once again to probabilistic(as well as tran sform) agruments. This discussion of intermediate queueingtheory begins with the important M/G/I queue (Chapter 5) and then proceedsto the dual G/M/I queue and its natural generalization to the system G/M/m(Chapter 6). The material on collective marks in Chapter 7 develops theprobabilistic interpretati on of tran sforms . Finally, the advanced mat erial inPart IV leads us to the queue G/G/I in Chapter 8; this difficult system (whosemean wait cannot even be expressed simply in terms of the system parameters)is studied through the use of the spectral solution to Lindley' s integral equa-tion. An approximati on to the precedence structure among chapters in thesetwo volumes is given below. In this diagram we have represented chapters inVolume I as numbers enclosed in circles and have used small squares forVolume II. The shading for the Volume I nodes indicates an appropri ateamount of mater ial for a relatively leisurely first cour se in queueing systemsthat can easily be accompl ished in one semester or can be comfort ably handledin a one-qua rter course. The shading of Chapter 2 is meant to indicate thatSections 2.2- 2.4 may be omitted on a first reading, and the same applies toSections 8.3 and 8.4. A more rapid one-semester pace and a highly acceleratedone-quarter pace would include all of Volume I in a single cour se. We closeVolume I with a summary of import ant equations, developed thr oughout thebook, which are grouped together according to the class of queueing systeminvolved; this list of results then serves as a "handbook" for later use by thereader in concisely summarizing the principal result s of this text. The resultsX PREFACEo Volume Io Volume II25are keyed to the page where they appear in order to simplify the task oflocating the explanatory material associated with each result.Each chapter contains its own list of references keyed alphabetically to theauthor and year; for example, [KLEI 74] would reference this book. Allequations of importance have been marked with the symbol - , and it isthese which are included in the summary of important equations. Each chapterincludes a set of exercises which, in some cases, extend the material in thatchapter ; the reader is urged to work them out.LEONARD KLEI NROCKXII PREFACEthe face of the real world's complicated models, the mathematicians proceededto ad vance the field of queueing theory rapidl y and elegantl y. The frontiersof this research proceeded int o the far reaches of deep and complex mathe-matics. It was soon found that the really intere sting model s did not yield tosolution and the field quieted down considerably. It was mainly with theadvent of digital computers that once again the tools of queueing theory werebrought to bear on a class of practical problems, but thi s time with greatsuccess. The fact is that at present, one of the few tools we have for analyzingthe performance of computer systems is that of queueing theory, and thisexplains its popularity among engineers and scientists today. A wealth ofnew problems are being formulated in terms of this theory and new tools andmeth ods are being developed to meet the challenge of these problems. More-over, the application of digital computers in solving the equati ons of queueingtheory has spawned new interest in the field. It is hoped that thi s two-volumeseries will provide the reader with an appreciation for and competence in themethods of analysis and application as we now see them.I take great pleasure in closing this Preface by acknowledging thoseindi vidual s and instituti ons that made it possible for me to bring this bookint o being. First , I would like to thank all tho se who participated in creatingthe stimulating environment of the Computer Science Department at UCLA,which encouraged and fostered my effort in this directi on. Acknowledgmentis due the Advanced Research Projects Agency of the Department of Defense ,which enabled me to participate in some of the most exciting and ad vancedcomputer systems and networks ever developed . Furthermore , the JohnSimon Guggenheim Foundation provided me with a Fellowship for theacademic year 1971-1 972, during which time I was able to further pursue myinvestigati ons. Hundreds of students who have passed through my queueing-systems courses have in major and minor ways contributed to the creationof this book, and I am happy to ackn owledge the special help offered byArne Nilsson, Johnny Wong, Simon Lam, Fouad Tobagi , Farouk Kamoun,Robert Rice, and Th omas Sikes. My academic and profes sional colleagueshave all been very support ive of this endeavour. To the typi sts l owe all. Byfar the lar gest port ion of this book was typed by Charlotte La Roche , and Iwill be fore ver in her debt. To Diana Skocypec and Cynthia Ellman I give mydeepest thanks for carrying out the enormous task of. proofreading andcorrection-making in a rapid , enthusiastic, and support ive fashion. Others whocontributed in maj or ways are Barbara Warren, Jean Dubinsky, Jean D'Fucci ,and Gloria Roy. l owe a great debt of thanks to my family (and especially tomy wife, Stella) who have stood by me and supported me well beyond thecall of duty or marriage contract. Lastl y, I would certainly be remiss inomitting an acknowledgement to my ever-faithful dict atin g machine, whichwas constantly talking back to me.March, 1974ContentsVOLUME IPART I: PRELIMINARIESChapter 1 Queueing Sys tems1.1. Systems of Flow .1.2. The Specification and Measure of Queueing Systems338Chapter 2 Some Impor tant Random Processes 102. 1. Notation and Structure for Basic Queueing Systems 102.2. Definition and Classification of Stochastic Processes 192.3 . Discrete-Time Markov Chains 262.4 . Co nti nuo us-Time Mar kov Chain s . 442.5 . Birth-Death Processes. 53PART II: ELEMENTARY QUEUEING THEORYChapter 3 Birth-Death Queueing Sys tems in Equili brium 893.1. Gener al Eq ui libri um Solution 903.2. M/M/I: The Classical Queueing System . 943.3. Discouraged Arrivals 993.4. M/ M/ ro: Responsive Servers (Infinite Number of Server s) 1013.5. M/M/m: The m-Server Case. 1023.6. M/M/I /K : Finite Storage 1033.7. M/ M/m/m: m-Server Loss Syste ms . 1053.8. M/M/I IIM: Finite Customer Population-Single Server 1063.9. M/M/ roIIM: Finite Cu stomer Population- " Infinite"Number of Servers 1073.10. M/M/m/K/M : Fi nite Population, m-Server Case , FiniteStorage 108XIIIxiv CONTENTSChapter 4 Markovian Queues in Equilibrium I 154.1. The Equilibrium Equ at ions . 11 54.2. The Method of Stages- Erlangian Distribution E. 1194. 3. The Queue M/Er/1 1264.4. The Queue ErlM/I 1304.5. Bulk Arri val Systems 1344.6. Bulk Service Systems 1374.7. Series-Parallel Stages : Generalizations 1394.8. Networks of Markovian Queues 147PART III: INTERMEDIATE QUEUEING THEORYChapter 5 The Queue M/G/I 1675. 1. The M/G/I System 1685.2. The Paradox of Residual Life: A Bit of Renewal Theory . 1695.3. The Imbedded Markov Chain 1745.4. The Transition Probabilities . 1775.5. The Mean Queue Length . 1805.6. Distributi on of Number in System . 1915.7. Distribution of Waiti ng Time 1965.8. The Busy Peri od and Its Durat ion . 2065.9. The Number Served in a Busy Period 2165.10. From Busy Periods t o Waiting Times 2195. 11. Combinat orial Methods 2235.12. The Tables Integrodifferential Equation . 226Chapter 6 The Queue G/M/m 2416. 1. Transition Probabilit ies for the Imbedded Markov Chain(G/M/m) 2416.2. Conditi onal Distributi on of Queue Size . 2466.3. Cond itional Distribut ion of Waiting Time 2506.4. The Queue G/M/I 2516.5. The Queue G/M/m 2536.6. The Queue G/M/2 256Chapter 7 The Method of Collective Marks 2617. 1. The Mar king of Customers 26I7.2. The Catastrophe Process 267CONTDITS XVPART IV: ADVANCED MATERIALChapter 8 The Queue GIGII8. 1. Lindley's I ntegral Equat ion8.2. Spect ra l Sol ution to Lindley' s In tegra! Eq uation8.3. Ki ngman ' s Algebra for Queues8.4. The Idle Tim e and Duali tyEpilogueAppendix I : Transform Theory Refresher: z-Transforrn and LaplaceTransform..2752752832993043191.1. Why Transforms ? 3211.2. The z-Transform . 3271.3. Th e Laplace Transform 3381.4. Use of Transforms in the Soluti on of Difference and Dif-feren tia l Equa tions 355Appendix II: Probability Theory RefresherII. I. Rules of the Game 36311.2. Random Variables 368I1.3. Expectation 37711.4. Transfo rms, Generating Funct ion s, and CharacteristicFunctions . 38111.5. Inequal it ies and Limit Theorems 38811.6. St ochastic Processes 393Glossary of Notation 396Summary of Important Results 400Index 411xvi CONTENTSVOLUME 1/Chapter I A Queueing The ory PrimerI. Notation2. Gene ral Results3. Markov, Birth-Death, and Poisson Processes4. The M /M / l Que ue5. The MI Ml m Queueing System6. Markovian Que ueing Networks7. The M / G/l Queue8. The GIMII Queue9. The GI Mlm Queue10. The G/G /l Que ueChapter 2 Bounds, Inequalities and ApproximationsI. The Heavy-Traffic Approximation2. An Upper Bound for the Average Wait3. Lower Bounds for the Average Wait4. Bounds on the Tail of the Waiting Time Distribution5. Some Remarks for GIGlm6. A Discrete Approximation7. The Fluid Approximation for Queues8. Diffusion Processes9. Diffusion Approximation for MIGII10. The Rush-Hour ApproximationChapter 3 Priority QueueingI . The Model2. An Approach for Calculating Average Waiting Times3. The Delay Cycle, Generalized Busy Periods, and WaitingTime Distributions4. Conservation Laws5. The Last-Come- First-Serve Queueing Discipline-.CONTENTS xvii6. Head- of-the-Line Priorities7. Ti me-Dependent Prior ities8. Opt imal Bribing for Queue Position9. Service-Time-Dependent DisciplinesChapter 4 Computer Time-Sharing and Multiaccess Systems1. Definitions and Models2. Distribution of Att ained Service3. The Batch Processing Algorithm4. The Round-Robin Scheduling Algorithm5. The Last-Come-First-Serve Schedul ing Algorithm6. The FB Schedul ing Algorithm7. The Mul tilevel Processor Sharing Scheduling Algor ithm8. Selfish Scheduling Algo rithms9. A Conservation Law for Time-Shared Systems10. Ti ght Bounds on the Mean Response Time11. Finite Popul ation Models12. Mult iple-Resour ce Models13. Models for Multiprogramming14. Remote Terminal Access to ComputersChapter 5 Computer-Communication NetworksI. Resource Sharing2. Some Contrasts and Trade-Off's3. Networ k Structures and Packet Switching4. The ARPANET-An Operational Descripti on of anExisting Network5. Definitions, the Model, and the Problem Statement s6. Delay Analysis7. The Capacity Assignment Problem8. The Traffic Fl ow Assignment Problem9. The Capacity and Flow Assignment Probl em10. Some Topological Considerations-Applications to theARPANETII . Satellite Packet Switching12. Grou nd Radio Packet Switchi ngxvi ii CONTENTSChapter 6 Computer-Communication NetworksMeasurement, Flow Control and ARP ANET Traps1. Simulation and Routing2. Early ARPANET Measur ements3. Flow Control4. Lockups, Degradations and Traps5. Network Throughput6. One Week of ARPANET Data7. Line Overhead in the ARPANET8. Recent Changes to the Flow Cont rol Procedure9. The Cha llenge of the FutureGlossarySummary of ResultsIndexQUEUEING SYSTEMSVOLUME I: THEORYPART IPRELIMINARIESIt is difficult to see the forest for the trees (especially if one is in a mobrather than in a well-ordered queue). Likewise, it is often difficult to see theimpact of a collection of mathematical results as you try to master them; it isonly after one gains the understanding and appreciation for their applicationto real-world problems that one can say with confidence that he understandsthe use of a set of tools .The two chapters contained in this preliminary part are each extreme inopposite directions. The first chapter gives a global picture of where queueingsystems arise and why they are important. Entertaining examples are providedas we lure the reader on. In the second chapter, on random processes, weplunge deeply into mathematical definitions and techniques (quickly losingsight of our long-range goals); the reader is urged not to falter under thissiege since it is perhaps the worst he will meet in passing through the text.Specifically, Chapter 2 begins with some very useful graphical means fordisplaying the dynamics of customer behavior in a queueing system. We thenintroduce stochastic processes through the study of customer arrival, be-havior, and backlog in a very general queueing system and carefully lead thereader to one of the most significant results in queueing theory, namely,Little's result, using very simple arguments. Having thus introduced theconcept of a stochastic process we then offer a rather compact treatmentwhich compares many well-known (but not well-distinguished) processes andcasts them in a common terminology and notation, leading finally to Figure2.4 in which we see the basic relationships among these processes; the readeris quickly brought to realize the central role played by the Poisson processbecause of its position as the common intersection of all the stochasticprocesses considered in this chapter. We then give a treatment of Markovchains in discrete and continuous time; these sections are perhaps the tough-est sledding for the novice, and it is perfectly acceptable ifhe passes over someof this material on a first reading. At the conclusion of Section 2.4 we findourselves face to face with the important birth-death processes and it is here2 PRELIMINARIESwhere things begin to take on a relationship to physical systems once again.In fact , it is not unreasonable for the reader to begin with Section 2.5 of thi schapter since the treatment following is (almost) self-contained from therethroughout the rest of the text. Only occasionally do we find a need for themore detailed material in Sections 2.3 and 2.4. If the reader perseveresthrough Chapter 2 he will have set the stage for the balance of the textbook.IQueuei ng SystemsOne of life' s more disagreeable act ivities, namel y, waiting in line, is thedelightful subject of thi s book. One might reasonably ask, " What does itprofit a man to st udy such unpleasant phenomena1" The answer , of course,is that through understanding we gain compassion, and it is exactl y thiswhich we need since people will be waiting in longer and longer queues ascivilizat ion progresses, and we must find ways to toler ate these unpleasantsitua tions. Think for a moment how much time is spent in one's dailyacti vities waiting in some form of a queue: waiting for breakfast ; stopped at atraffic light ; slowed down on the highways and freeways ; del ayed at theentrance to one's parking facility; queued for access to an elevat or ; sta ndi ngin line for the morn ing coffee; holding the telephone as it rings, and so on.The list is endless, and too often also are the queues.The orderliness of queues varies from place to place ar ound the world.Fo r example, the English are terribly susceptible to formation of orderlyqueues, whereas some of the Mediterranean peopl es consider the idealudicrous (have you ever tried clearing the embarkation pr ocedure at thePort of Brindisi 1). A common sloga n in the U.S. Army is, "Hurry up andwait." Such is the nature of the phenomena we wish to study.1.1. SYSTEMS OF FLOWQueueing systems represent an example of a much broader class ofint erest ing dynamic systems, which, for convenience, we refer to as " systemsof flow." A flow system is one in which some commodity flows, moves, or istran sferred through one or more finite-capacity channels in order to go fromone point to another. For example, consider the flow of automobi le traffi ct hr ough a road network, or the transfer of good s in a railway system, or thest reami ng of water th rough a dam, or the tr ansmission of telephone ortelegraph messages, or the passage of customers through a supermarketcheckout counter, or t he flow of computer pr ograms t hrough a ti me-shar ingcomputer system. In these examples the commodities are the automobiles,the goods, the water, the telephone or telegraph messages, the customers, andthe programs, respecti vely; the channel or channels are the road network,34 QUEUEING SYSTEMSthe rail way net wor k, the dam, the telephone or telegraph networ k, thesupermarket checkout counter, and the computer processing system, re-spectively. The " finite capacity" refers to the fact that the channel can satisfythe demands (placed upon it by the commodity) at a finite rate only. It isclear that the analyses of many of these systems require analytic tools drawnfrom a variety of disciplines and, as we shall see, queueing theory is just onesuch discipline.When one analyzes systems of flow, they naturally break int o two classes :steady and unsteady flow. The first class consists of those systems in which theflow proceeds in a predictable fashion. That is, the quantity of flow isexactly known and is const ant over the int erval of interest; the time when tha tflow appears at the channel, and how much of a demand that flow places uponthe channel is known and consta nt. These systems are trivial to analyze in thecase of a single channel. For example, consider a pineapple fact ory in whichempty tin cans are being transported along a conveyor belt to a point atwhich they must be filled with pineapple slices and must then proceed furtherdown the conveyor belt for addi tional operations. In thi s case, assume thatthe cans arrive at a constant rate of one can per second and that the pine-apple-filling operation takes nine-tenths of one second per can. These numbersare constant for all cans and all filling operations. Clearl y thi s system willfunct ion in a reliable and smooth fashion as long as the assumpti ons statedabove continue to exist. We may say that the arrival rate R is one can persecond and the maximum service rate (or capacity) Cis 1/0.9 = 1.11111 . ..filling operations per second . The example above is for the case R < c.However , if we have the condition R > C, we all know what happens : cansand/ or pineapple slices begin to inundat e and overflow in the fact ory! Thuswe see that the mean capacity of the system must exceed the average flowrequirements if chaotic congest ion is to be avoided ; this is true for all systemsof flow. Th is simple observation tells most of the story. Such systems are oflittl e interest theoretically.The more interesting case of steady flow is that of a net work of channels.For stable flow, we obviously require that R < C on each channel in thenet wor k. However we now run int o some serious combinat orial problems.For example, let us consider a rail way net wor k in the fictitious land ofHatafla. See Figure 1.1. The scenario here is that figs grown in the city ofAbra must be transported to the destinati on city of Cadabra, makin g useof the rail way network shown. The numbers on each chann el (section ofrailway) in Figure 1.1 refer to the maximum number of bushels of figs whichthat cha nnel can handle per day. We are now confronted with the followingfig flow probl em: How many bushels of figs per day can be sent from Abra toCadabra and in what fashion shall this flow of figs take place ? The answer tosuch questions of maximal " traffic" flow in a variety of networ ks is nicelyZeus 81.1. SYSTEMS OF FLOWNonabel5Abra CadabraSucsamad 6 OriacFigure 1.1 Maximal flow problem.sett led by a well-known result in net work flow theory referred to as themax-flow-min-cut theorem. To state this theo rem, we first define a cut as aset of channel s which, once removed from the network, will separate allpossible flow from the origin (Abra) to the destination (Cadabra). We definethe capacity of such a cut to be the total fig flowthat can travel across that cutin the direct ion from origin to destination. For example, one cut consists ofthe bran ches from Abra to Zeus, Sucsamad to Zeus , and Sucsamad to Oriac ;the capacit y of thi s cut is clearl y 23 bushels of figs per day. The max-flow-min-cut theorem states that the maximum flow that can pass bet ween anorigin and a destination is the minimum capacity of all cuts. In our exampleit can be seen that the maximum flow is therefore 21 bushels of figs per day(work it out). In general, one must consider all cut s that separate a givenorigin and destination. This computation can be enormously time consuming.Fortunately, there exists an extremel y powerful method for finding not onlywhat is the maximum flow, but also which flow pattern achieves this maxi-mum flow. Thi s procedure is known as the labeling algorithm (due to Fordand Fulkerson [FORD 62]) and is efficient in that the computational require-ment grows as a small power of the number of nodes ; we present the algor ithmin Volume II , Chapt er 5.In addition to maximal flow problems, one can pose nume rous otherinteresting and worthwhile questions regarding flow in such networks. Forexample , one might inquire int o the minimal cost network which will supporta given flow if we assign costs to each of the channels. Also, one might askthe same questions in networks when more than one origin and dest inati onexist. Complicating matters further, we might insist that a given networksupport flow of various kinds. for example, bushels of figs, cartons ofcartridges and barrel s of oil. Thi s multic ommodity flow problem is anextremely difficult one, and its solution typically requires consi derablecomputati onal effort. These and numerous other significant problems innet wor k flow theory are addressed in the comprehensive text by Frank andFrisch [FRAN 71] and we shall see them again in Volume II , Chapter 5. Net-work flow theory itself requires met hods from graph theory, combinator ial6 QUEUEING SYSTEMSmathematics, optimization theory, mathematical programming, and heuristicprogramming.The second class into which systems of flow may be divided is the class ofrandom or stochastic flow problems. By this we mean that the times at whichdemands for service (use of the channel) arrive are uncertain or unpredict-able, and also that the size of the demands themselves that are placed uponthe channel are unpredictable. The randomness, unpredictability, or unsteadynature of this flow lends considerable complexity to the solution and under-standing of such problems. Furthermore, it is clear that most real-worldsystems fall into this category. Again, the simplest case is that of random flowthrough a single channel; whereas in the case of deterministic or steady flowdiscussed earlier in which the single-channel problems were trivial, we havenow a case where these single-ehannel problems are extremely challengingand, in fact , techniques for solution to the single-channel or single-serverproblem comprise much of modern queueing theory .For example, consider the case of a computer center in which computationrequests are served making use of a batch service system. In such a system,requests for computation arrive at unpredictable times, and when they doarrive, they may well find the computer busy servicing other demands. If, infact, the computer is idle, then typically a new demand will begin serviceand will be r u ~ until it is completed. On the other hand, if the system is busy,then this job will wait on a queue until it is selected for service from amongthose that are waiting. Until that job is carried to completion, it is usually thecase that neither the computation center nor the individual who has submittedthe program knows the extent of the demand in terms of computational effortthat this program will place upon the system; in this sense the service require-ment is indeed unpredictable.A variety of natural questions present themselves to which we would likeintelligent and complete answers . How long, for example , maya job expect towait on queue before entering service? How many jobs will be serviced beforethe one just submitted? For what fraction of the day will the computationcenter be busy? How long will the intervals of continual busy work extend?Such questions require answers regarding the probability of certain periodsand numbers or perhaps merely the average values for these quantities.Additional considerations, such as machine breakdown (a not uncommoncondition), complicate the issue further; in this case it is clear that some pre-emptive event prevents the completion of the job currently in service. Otherinteresting effects can take place where jobs are not serviced according to theirorder of arrival. Time-shared computer systems, for example, employ rathercomplex scheduling and servicing algorithms, which, in fact , we explore inVolume II, Chapter 4.The tools necessary for solving single-channel random-flow problems are1.1. SYSTEMS OF FLOW 7contained and described withi n queue ing theory, to which much of th is textdevote s itself. Th is requires a back ground in pr obability th eory as well as anunderst anding of complex variables and so me of the usual tr ansform-calculus methods ; th is material is reviewed in Appendices I and II.As in the case of deterministic flow, we may enlarge our scope of probl emsto that of networks of channels in which random flow is encountered. Anexample of such a system would be that of a computer network. Such asystem consists of computers connected together by a set of communicationlines where the capacity of these lines for carrying information is finite. Let usreturn to the fictiti ous land of Hatafla and assume that the railway net workconsidered earli er is now in fact a computer net work. Assume that user slocated at Abra require computational effort on the facility at Cadabra. Theparticular times at which these requests are made are themsel ves unpredict-able, and the commands or inst ructions that describe these request s are al soof unpredictable length . It is these commands which must be transmitted toCadabra over our communication net as messages. When a message isinserted int o the network at Abra, and after an appropriate deci sion rule(referred to as a routing procedure) is accessed, then the message proceedsthrough the network along so me path. If a port ion of this path is busy, andit may well be, then the message must queue up in front of the busy channeland wait for it to become free. Const ant decisions must be made regardingthe flow of messages "and routing procedures. Hopefully, the message willeventually emerge at Cadabra, the computation will be performed, and theresults will then be inserted into the network for delivery back at Abra.It is clear that the problems exemplified by our computer net wor k involve avariety of extremely complex queueing problems, as well as networ k flowand deci sion problems. In an earlier work [KLEI 64] the author addressedhimself to certain as pects of these questions. We develop the analysis of thesesyst ems lat er in Volume II , Chapter 5.Having thus classified *systems of flow, we hope that the reader underst andswhere in the general scheme of things the field of queueing theory may bepl aced. The methods from thi s the ory a re central to analyzing most stochas ticflow problems, and it is clear from a n examination of the current literat urethat the field and in particular its applications are growing in a viable andpurposeful fashion. The classification described above places queueing systems within the class of systems offlow. This approach identifies and emphasizes the fields of applicatio n for queueing theory.An alterna tive approach would have been to place queueing theory as belongi ng to thefield of app lied stochastic processes ; this classification would have emphasized the mat he-ma tical structure of queueing theory ra ther than its applications. The point of view takenin this two-volume book is the former one, namely. with application of the theory as itsmajor goal rat her than extension of the math emat ical for mal ism and result s.8 QUEUEING SYSTEMS1.2. THE SPECIFICATION AND MEASUREOF QUEUEING SYSTEMSIn order to completely specify a queueing system, one must iden tify thestochas tic processes that describe the arriving stream as well as thestructureand di sciplin e of the service facility. Generally, the arri val pr ocess is describedin terms of the probability di stribution of the interarrical times of customersand is denoted A(t) , where*A(t ) = P[time between arrivals ~ t] (I.I )The assumption in most of queueing theory is that these interarrival times areindependent , identically distributed random variables (and, therefore, thest rea m of arrivals forms a stationary renewal process ; see Chapter 2). Thus,onl y the di stribution A(t) , which describes the time between a rrivals, is usuallyof significa nce. The second statistical quantity that must be described is theamount of demand these arrivals place upon the channel; thi s is usuall yreferred to as the service time whose probability distributi on is den oted byB(x) , that is,B(x) = P[service time ~ x ] ( 1.2)Here service time refers to the length of time that a cust omer spends in theser vice facility.Now regarding the st ruct ure and discipline of the service facility, one mustspec ify a variety of additiona l quantities. One of these is the extent ofstorage capacity available to hold waiting customers and typically thi s quan-tit y is described in terms of the variable K ; often K is taken to be infinite. Anadditional specification involves the number of service stations available, andif more than one is available, then perhaps the di stribution of service timewill differ for each, in which case the distribution B(x) will include a subscriptto indicate that fact. On the other hand, it is sometimes the case that thearriving st ream con sist s of more than one identifiable class of customers ; insuch a case the interarrival distributi on A (t ) as well as the service di str ibut ionB(x) may each be characteri stic of each class and will be identified again byuse of a subscript on these distr ibutions. An other importa nt st ruct uraldescripti on of a queueing system is that of the queueing discipline; thi sdescribes the order in which cust omers are taken from the queue and allowedint o service. For example, some standa rd queueing di sciplines are first-co me-first-serve (FCFS), last-come-first-serve (LCFS), and random order ofservice. When the arriving customers are distin guishable according to gro ups,then we encounter the case of priority queueing disciplines in which priority The notat ion P[A] denotes, as usual. the " pro bability of the event A,"1.2. THE SPECIFICATION AND ~ I E A S U R E OF QUEUEING SYSTntS 9among groups may be established. A further sta tement regarding the avail-ability of the service facility is also necessary in case t he service faci lity isoccasionally requ ired to pay attention to other ta sks (as, for example, itsown breakdown). Beyond this, queue ing systems may enjoy custo merbehavior in the form of defections from the queue, j ockey ing among the manyqu eues, balking before ent ering a queue, bribing for queue positi on , cheatingfor queue po sition, a nd a variety of othe r interesting and not-unexpectedhumanlike cha racterist ics. We will encounter these as we move th rough t hetext in an orderly fashion (first-come-fi rst-serve according to page nu mber).No w that we have indi cated how one must specify a queueing system, it isappropriate t hat we ide nti fy the meas ures of performance and effectivenessthat we sha ll obtai n by analysis. Basicall y, we are int erested in the waiting timefor a custo mer, the number of customers in the system, the length of a busyperiod (the continuous interval during which the server is busy), the length ofan idle period, a nd the current 1I'0rk backlog expressed in units of time. Allt hese quant ities a re ra ndom variables and thus we seek their complet eprob abilistic description (i.e., their probability dist ribu tion function).Us ually, however , to give the distributi on functio n is to give more thanone can easi ly make use of. Consequently, we often settle for the first fewmoments (mean, var iance, etc.).Happily, we shall begin with simple co nsiderations and develop the toolsin a st raightforward fashio n, paying a tte ntion to the essential det ails ofa nalysis. In t he followi ng pages we will encounter a variety of simple queueingproblems, simple at least in the sense of description and usually rathersophistica ted in terms of solution. However , in orde r to do t his pr operl y, wefirst devote our efforts in the following chapter to describing some of t heimportant ra ndom processes that ma ke up the arriva l a nd service processesin our queueing systems.REFERENCESFORD 62 Ford, L. R. and D. R. Fulkerson, Flows in Networks, PrincetonUniversity Press (Princeton, N.J.), 1962.FRAN 71 Frank. H. and I. T. Frisch, Communication. Transmission , andTransportation Network s, Addison-Wesley (Reading, Mass.), 1971.KLEI 64 Kleinrock, L. . Communication Nets ; Stochastic Message Flow andDelay . McGraw-Hili (New York), 1964, out of print. Reprinted byDover (New York), 1972.2Some Important Random Processes*We assume that the reader is familiar with the basic elementary notions,terminology, and concepts of probability theory. Th e particular aspects ofthat theory which we require are presented in summary fashion in AppendixII to serve as a review for those readers desi ring a quick refresher andreminder; it is recommended that the material therein be reviewed, especiallySection 11.4on transforms, generating functions , and characteri stic functions.Included in Appendix " are the following important definitions, concep ts,and results :Sample space, events , and probabi lity. Conditional probability, statistical inde pendence, the law of totalprobabi lity, and Bayes' theorem.A real random variable, its pro babili ty dist ribution function (PDF),its probability density function (pdf), and their simple properties. Events related to random variables and their probabilities. Joint dist ribu tion functions.Functions of a random variable and t heir density functions. Expectati on. Laplace transforms , generating functions, and characteri stic functi onsand their relationships and propertics.t Inequalities and limit theorems .Definition of a stochast ic process.2.1. NOTATI ON AND STRUCTURE FOR BASICQUEUEING SYSTEMSBefore we plunge headlong into a step-by-step development of queueingtheory from its elementary not ions to its inte rmediate and then finally tosome ad vanced material , it is important first that we understand the basic Sections 2.2, 2.3, and 2.4 may be skipped on a first read ing.t Appendix [ is a transform theor y refresher. This material is also essential to the properunder standing of this text.102.1. NOTAn ON AND STRUCTURE FOR BASIC QUEUEING SYSTEMS IIst ruct ure of queues. Also, we wish to provide the read er a glimpse as towhere we a re head ing in th is journ ey.It is our purpose in thi s sectio n to define some notation, both symbolicand graphic, and then to introduce one of the basic stochast ic pr ocesses thatwe find in queueing systems. Further , we will deri ve a simple but significa ntresult , which relates some first moments of importance in t hese systems. Inso doing, we will be in a positi on to define the quantities and processes thatwe will spend many pages studying later in the text.The system we co nsider is the very general queueing system G/G/m ; recall(from the Preface) that thi s is a system whose interarrival time di stributi onA (I) is completel y arbit ra ry a nd who se service time di stributi on B(x) is alsocompletely arbitrary (all interar rival times and service time s are assumed tobe inde pendent of each ot her). The system ha s m servers and order of serviceis also quite arbit ra ry (in particular, it need not be first-come-first-serve).We focus attentio n on the flow of customers as they arri ve, pass through , a ndeventuall y lea ve thi s system: as such, we choose to number the customers withthe subsc ript n a nd define C,. as foll ows :Cn denotes the nth customer to enter the system (2. 1)Thus, we may portray our system as in Figure 2.1 in which the box representst he queueing system and the flow of cust omers both in and out of the systemis shown. One can immediately define some random processes of int erest.For example, we are int erested in N( I) where *N(I) ~ number of cust omers in the system at time I (2.2)Another stochastic process of interest is the unfinished work V( I) that existsin the system a t time I , that is,V( I) ~ the unfinished work in the system a t time I~ the remainin g t ime required to empty the system of allcustomers present at time I (2.3)Whenever V( I ) > 0, then the system is said to be busy, and only whenV( I) = 0 is the syste m sai d to be idle. The durati on and location of the se busyand idl e peri ods a rc al so quantiti es of int erest.Oueoei nqsystem.. ,J; ~; I- - - - - - - ----'Figure 2. 1 A general queueing system. The notation ~ is to be read as "equals by defi nition."12 SOME IMPORTANT RANDOM PROCESSESwhich is independent of n. Similarly, we define the service time for COl asThus we have defined for the nth customer his a rrival time, " his" intera rri v.time, his service time, his waiting time, a nd his system t ime. We find(2. 5)(2.6)(2.9'(2.8;(2.7)(2. 1(interarrival times are drawn from the dis-P[t" ~ t] = A(t)P[Xn ~ x] = B(x)Xn ,;; service time for CnII' n ~ waiting ti me (in queue) for C;= II' n + x"and from our assumptions we haveThe det ail s of the se stochastic processes may be observed first by definingthe foll owing variables and then by di spl aying these va riab les on an appro-priate time di agram to be discussed below. We begin with the definitions.Recalling that the nth cust omer is den oted by Cn. we define his arriva l timeto the queueing system asT n ~ a rriva l time for Cn (2.4)The sequences {tn } a nd {xn } may be thought of as input va riables for OUIqueueing system; the way in which the system handles these cust omers give:rise to queues and waiting times that we must now define. Thus, we define t h.waiting time (time spent in the queue) * as .The total time spent in the system by COl is the sum of his wai ti ng time an,service time, which we denote bys; ~ system time (queue plus service) for COl* T he terms " waiting ti me" and " queueing time" have conflicting defini tions within tlbody of queueing-theory literatu re. The former sometimes refers to the total time spentsystem. and the latter then refers to the total time spent on queue ; however . these tvdefinit ions are occasionally reversed. We attempt to remove that confusion by definiwaiting and queueing time to be the same quant ity. namely. the time spent wa iting 'queue (but not being served); a more appropriate term perhaps would be " wasted tim'The tota l time spent in the system will be referred to as "sys tem time" (occasionally kno-as " flow time" ).2.1. NOTATION AND STRUCTURE FOR BASIC QUEUEING SYSTEMS 13expedient at th is point to elaborate somewhat further on notation. Let uscon sider the interarrival time In once again. We will have occasion to referto the limiting random varia ble i defined by- ~ I'I = im 1nn-e cc(2.11)which we den ote by I n -+ i. (We have already requ ired that the interarrivaltimes In have a di stribution independent of n, but this will not necessaril y bethe case with many other random variables of interest.) The typical notationfor the probability distribut ion function (PDF) will beand for the limiting PDFP[i ~ I] = A (I)(2.12)(2. 13)rseI)dThi s we denote by A n(l) -+ A(t ) ; of course, for the interarrival time we haveassumed that A n(l ) = A (I) , which gives rise to Eq. (2.6). Similarly, theprobability den sit y function (pdf) for t n and i will be an(l) and aCt), respec-tively, and will be den oted as an(t) -+ aCt). Finally, the Laplace transform(see Appendix II) of these pdf's will be denoted by An *(s) and A *(s),respecti vely, with the obvious notation An*(s ) -+ A *(s) . The use of the letterA (and a) is meant as a cue to remind the reader that they refer to theinterarrival time . Of .course, the moments of the interarrival time are ofinterest and they will be denoted as follows *:E[l n]';;'i. (2.14)Acc ording to our usual notati on , the mean interarrival time for the limitingrandom variable will be given] by i in the sense that i. -+ i. As it turns outi, which is the average interarri val time between customers, is used sofrequentl y in our equ ati ons that a special notation ha s been adopted asfoll ows :Thus i. represents th e Qt'erage arrical rate of customers to our queueingsystem. Hi gher moments of the interarrival time are also of interest and sowe define the k th moment by The notat ion E[ J denotes the expecta tion of the quant ity within squar e brackets. Asshown, we also adopt the overbar notat ion to denote expect at ion.t Actually, we should use the notation I wit h a tilde and a ba r, but this is excessive and willbe simplified to i. The same simplification will be applied to many of our ot her randomvariables.))alitheinvongon-vn_ - a(t ), An*(s) ->- A*(s)- - I k ktn-+ t = ~ = a1 = a, t ; ->- t = ak (2. 18)In a similar ma nner we identify the notation associated with X n , I\' n, and S nas follo ws :Xn = service time for CnXn-+ X, B.(x) -+ B(x), b. (x) -+ b(x), Bn*(s) - ~ B*(s)- - 1 b bXn -.. x = - = 1 = ,f-l(2. 19)IVn = waiting time for Cn-s; = system time for C,s ; -+ s, Sn(Y) ->- S(y), sn(Y) -+ s(y), S n"(s) -+ S *(s)(2.20)(2.21)All th is notation is self-evident except perhaps for the occas ional specialsymbols used for the first moment and occasionally t he higher moment s ofth e random variables invol ved (t ha t is, the use of t he symbols )" a, Il, b, IV,and T). The reader is, at thi s poin t , directed to t he Gl ossary for a completeset of not ati on used in thi s book.With the above not ation we now suggest a time-diagram notation forqueues, which pe rmits a graphical view of the dynamics of our queueingsystem and a lso provides the det ails of t he underlying stochastic processes.This diagram is shown in Figure 2.2. Thi s particular figure is shown for a2.1. NOTATION A:--ID STRUCTURE FOR BASIC QUEUEI NG SYSTEMS 15s.' ICll _1 C. cr. ,!'. Cn +2ServicerC ~ Cn + 1 C:"'+2 Tlme v-c-e--T.'n +1 T/u 2Queue(11+2Cn en +1 Cn -t2Figure 2.2 Time-diagram notation for queues.first-come-first-serve order of service, but it is e ~ s y to see how the figuremay al so be made to represent any order of service. In this time diagram thelower horizontal time line rep resents the queue and the upper hori zontal timeline represents the service facility; moreover, the diagram shown is for thecase of a single server, although this too is easil y generalized. An arrowapproaching the queue (or service) line from below indicates that an arrivalhas occurred to the queue (or service facility) . Arrows emanating from theline indicate the departure of a customer from the queue (or service facility).In this figure we see that customer Cn+1 arrives before customer Cn entersservice; only when C; departs from service may Cn+l ente r service and , ofcourse, the se two events occur simultaneously. Notice that when Cn+2 entersthe system he finds it empty and so immediately proceeds through an emptyqueue directly int o the service facility . In this di agram we have al so shown thewaiting time and the system time for Cn (note that 1\'n+2 = 0). Thus, as timeproceed s we can identify the number of cust omers in the system N(t), theunfini shed work Vet) , and also the idle and busy period s. We will find muchuse for thi s time-di agram notation in what follows.In a general que ueing system one expects that when the number ofcustomers is lar ge then so is the waiti ng time. One manifestation of thi s is avery simple relati onship between the mean number in the queueing system,the mean a rriva l rate of customers to that system, and the mean systemtime for cust omers. It is our purpose next to deri ve that relati onship andthereby familiarize ourselves a bit further with the underlying behaviorof the se systems. Referring back to Figure 2.1, let us position ourselves atthe input of the queueing system and count how man y cust omers ent er as afunction of time . We denote this by ot (/) whereot (t) ~ number of arri vals in (0, t ) (2.22)16 SOME IMPORTANT RANDOM PROCESSES12,-- - ---------- - ------------,1110~ 9E 8B'5 7c 6'05"~ 4~ 321oL----"'="'"Time lFigure 2.3 Arrivals and departures.Alternat ively, we may position ourselves at the output of the queuei ng systemand count the number of departures that leave; thi s we denot e bybet) ~ number of depar ture s in (0, r) (2.23)Sample functions for these two stochastic processes are shown in Figur e 2.3.Clearly N( t), the number in the system at time t, must be given byN( t ) = (r) - bet)On the other hand, the tot al area bet ween these two curves up to some point ,say t , repr esent s t he total time all customers have spent in the system (meas-ur ed in unit s of customer-seconds) during the int erval (0, t) ; let us denote thiscumulative area by yet ). Moreover, let At be defined as the average arrivalrate (customers per second) during the int erval (0, t); that is,(2.24)We may define T, as the system time per customer averaged over all custome rsin the interval (0, t); since yet ) repre sent s the accumulated customer- secondsup to time t , we may divide by the number of arrivals up to that point toobtainyet)Tt = -(X( t)Lastly, let us define NI as the average number of custome rs in the queueingsystem during the int erval (0, r): this may be obtained by dividing theaccumulated number of customer-seconds by the total interval length t2.1. NOTATION AND STRUCTURE FOR BASIC QUEUEING SYSTEMS 17thusly_ y(t )N,= -tFrom these last three equations we seeN, = A,T,Let us now assume that our queueing system is such that the following limitsexist as t -.. CtJ:A= lim J. ,,-'"T=lim T,,- '"Note that we are using our former definitions for Aand T representing theaverage customer arri val rate and the average system time , respectively. Ifthese last two limits exist, then so will the limit for N" which we denote by Nnow representing the average number of customers in the system; that is,N= AT - (2.25)Thi s last is the result we were seeking and is known as Little's result . It statesthat the average number of customers in a queueing system is equal to theaverage arrical rate of customers to that system, times the average time spentin that system. * The above proof does not depend upon any specific assump-tion s regarding the arrival distribution A (r) or the service time distributionB(x) ; nor does it depend upon the number of servers in the system or upon theparticular queueing discipline within the system. This result existed as a" folk the orem" for many years ; the first to establish its validity in a formalway was J. D. C. Little [LITT 61] with some later simpl ifications by W. 'S.Jewell [JEWE 67) . S. Ei lon [EILO 69 ) and S. Stidham [STID 74). It is im-portant to note that we have not precisely defined the boundary around ourqueueing system. For exampl e, the box in Figure 2.1 could apply to the entiresystem composed of queue and server , in whic h case Nand T as defined referto quant ities for the entire syste m; on the other hand , we could have consid-ered the bound ar y of the queueing system to contai n only the queue itsel f, inwhich case the relationship wou ld have beenNo= AW - (2.26)where No represents the average number of customers in the queue and, asdefined earlier, W refers to the average time spent waiting in the queue. As athird possible alterna tive the queueing system defined could have surrounded An intuiti ve proof of Little' s result depends on the observation that an arriving cus-tomer should find the same average number, N, in the system as he leaves behind uponhis departure. Thi s latter quantity is simply the arri val ra te A times his average time insystem, T.18 SOME I11PORTANT RANDOM PROCESSESonly the server (or servers) itself; in this case our equation would have reducedtoR. = AX (2.27)where R. refers to the average number of customers in the service facility(or facilities) and x, of course, refers to the average time spent in the servicebox. Note that it is always true thatT = x + W _ (2.28)The queueing system could refer to a specific class of customers, per hapsbased on priority or some other attribute of this class, in which case the samerelationship would apply. In other words, the average arri val rate of customersto a "queueing system" times the average time spent by cust omer s in that"system" is equal to the average number of customers in the "system,"regardless of how we define that " system."We now discuss a basic parameter p, which is commonly referred to as theutilization factor. The utilization factor is in a fundamental sense really theratio R/C, which we introduced in Chapter I. It is the rat io of the rate atwhich "work" enter s the system to the maximum rat e (capacity) at which thesystem can perform thi s work; the work an arri ving customer brings into thesystem equals the number of seconds of service he requires. So, in the case ofa single-server system, the definition for p becomesp ,;; (average arrival rate of customers) X (average service time)= AX _ (2.29)Thi s last is true since a single-server system has a maximum capacity fordoing work , which equals I sec/sec and each ar riving customer brings anamount of work equal to x sec; since, on the average, ..1. customers ar rive persecond, then }.x sec of work are brought in by customer s each second thatpasses, on the average. In t he case of mult iple servers (say, III servers) thedefinition remains the same when one considers the ratio R/C , where now thework capacity of the system is III sec/sec; expressed in terms of system param-eters we then havea AXp = - _ (2.30)mEquat ions (2.29) and (2.30) apply in the case when the maximum servicerat e is independent of the system state; if this is not t he case, then a morecareful definition must be provided. The rate at which work enters thesystem is sometimes referred to as the traffi c intensity of the system and isusually expressed in Erlangs ; in single-server systems, the utilizat ion factor isequal to the traffic intensity whereas for (m) mult iple servers, the tr affi cintensity equal s mp. So long as 0 ~ p < I, then p may be interpreted asp = E[fraction of busy servers1 (2.3I)2.2. DEFINITION AND CLASSIFICATION OF STOCHASTIC PROCESSES 19[In the case of an infinite number of servers, t he ut ilizati on fact or p plays noimpor tant part , and instead we are interested in the number of busy servers(and its expectati on).] -Indeed , for the system GIGII to be st able , it must be that R < C, that is,o~ p < I. Occasionally, we permit the case p = 1 with in the ran ge ofsta bility (in particul ar for the system 0 /0/1). Stability here once again refer sto the fact that limiting di stributions for all random vari ables of interestexist , and that all customers are eventually served. In such a case we maycarry out the following simple calcul ation. We let 7 be an arbitrarily longt ime interval ; during this interval we expect (by the law of large numbers)with probability 1 that the number of arrivals will be very nearly equal to .AT.Moreover , let us define Po as the probability that the server is idle at somerandomly selected time . We may, therefore, say that during the interval 7,the server is busy for 7 - TPO sec, and so with pr obability I , the number ofcustomers served during the interval 7 is very nearly (7 - 7po)fx. We maynow equate the number of arri vals to the number served during thi s int erval,which gives, for lar ge 7,Thus, as 7 ->- 00 we have Ax = I - Po; using Definiti on (2.29) we finall yha ve the important conclusion for GfG/lp = 1 - P (2.32)The interpretati on here is that p is merely the fracti on of time the server isbusy; thi s supports the conclusion in Eq. (2.27) in which Ax = p was shownequal to the average number of customers in the service facilit y.Thi s, then, is a rapid look at an overall queueing system in which we ha veexposed some of the basic stochast ic processes, as well as some of theimpo rta nt definiti ons and notation we will encounter. Moreover , we haveestabli shed Little' s result , which permits us to calcul ate the average numberin the system once we have calculated the average time in the system (or viceversa). Now let us move on to a more careful study of the imp ortant stochas ticprocesses in our queueing systems.2.2*. DEFINITION AND CLASSIFICATIONOF STOCHASTIC PROCESSESAt the end of Appendix II a definiti on is given for a stochas tic process,which in essence states that it is a famil y of random vari ables X(t) wher e the The reader may choose to skip Sections 2.2, 2.3, and 2.4 at this point and move directlyto Section 2.5. He may then refer to this materi al only as he feels he needs to in the balanceof the text.20 SOME IMPORTANT RANDOM PROCESSESrandom variables are "indexed" by the time parameter I. For example, t henumber of people .sitting in a movie theater as a funct ion of time is astochastic process, as is also the atmospheric pressure in that movie the ateras a functi on of time (at least those functi ons may be modeled as stoc hasticprocesses). Often we refer to a stochastic process as a random process. Arandom process may be thought of as describing the moti on of a particle insome space. The classification of a random process depends upon threequantities: the slate space; the index (lime) parameter; and the statisticaldependencies among the random variables X(I ) for different values of theindex parameter t. Let us discuss each of these in order to provide the generalframework for random processes.Fir st we consider the state space. The set of possible values (or st ates) thatX(I) may take on is called its state space. Referring to our analogy with regardto the motion of a particle, if the positions th at particle may occupy arefinite or countable, then we say we have a discrete-state process, oftenreferred to as a chain. The state space for a chain is usually the set of inte gers{O, 1,2, .. .}. On the other hand, if the permitted positions of the particleare over a finite or infinite continuous interval (or set of such intervals), thenwe say that we have a cont inuous-state process.Now for the index (lime) parameler. If the permitted times at which changesin positi on may take place are finite or countable, then we say we have adiscrele-(time) parameter process; if these changes in positi on may occuran ywhere within (a set of) finite or infinite intervals on the time axis, then wesay we have a continuous-parameter process. In t he former case we ofte n writeX n rather than X(I) . X n is often referred to as a random or stochas tic sequellcewhereas X(I ) is often referred to as a random or stochas tic process .The truly di stingui shing feature of a stochas tic process is th e rel ati onshipof the random variables X(I) or X n to ot her members of the same famil y. Asdefined in Appendi x II , one must specify the complete j oint dis t rib utionfunction among the random variables (which we may th ink of as vectorsden oted by the use of boldface) X = [X(t l ), X( I.) , . . .J, namel y,Fx(x; t) ~ P[X(t I) ~ Xl ' .. , X( l n) ~ xnl (2.33)for all x = (Xl' X., . . . , Xn) , t = (II> I. , ... , In), and 11. As menti oned there,thi s is a formidable task ; fortunately, many interest ing stochastic processespermit a simpler description. In any case, it is the funct ion Fx(x ; t) that reallydescribes the dependencies among the random variables of the stoc has ticprocess. Below we describe some of the usual types of stochas tic pr ocessesth at are characterized by different kinds of dependency relati ons among t heirrandom variables. We provide thi s classificati on in order to give t he reader aglobal view of this field so that he may better understand in which particular2.2. AND CLASSIFICAT ION OF STOCH ASTI C PROCESSES 21regions he is operating as we proceed with our st udy of queueing theory andit s related stochas t ic pr ocesses.(a) Stationary Processes. As we discuss at the ver y end of Appendix II,a stochas tic process X (I ) is said t o be stat ionary if Fx(x ; t) is invari ant toshifts in time for all values of it s arguments; that is, given any constant Tthe foll owing must hold :FX(x ; t + T) = Fx (x ; t) (2.34)where the notati on t + T is defined as the vector ( 11 + T , 12 + T, . . , I n + T) .An associated noti on , that of wide-sense stationarity, is identified with therandom process X(I) if merely both the first and second moments are inde-pendent of the locati on on the time axis, that is, if E[X(I )] is independent of Iand if E[X(I)X(I + T)] depends only upon T and not upon I. Observe that allst ati onary processes are wide-sense stati onary, but not conversely. Thetheory of stationary random pr ocesses is, as o ne might expect, simpler thanthat for nonstationary processes.(b) Independent Processes. The simplest and most tr ivial stochast icprocess to consider is the random sequence in which {Xn } forms a set ofindependent random variables, that is, the j oint pdf defined for our stochasticprocess in Appendix .II mu st fact or into the product, thusly(2.35)In th is case we are stretching th ings somewhat by calling such a sequence arandom process since there is no st ruct ure or dependence among the randomvariables. I n the case of a continuous random process, such an independentpr ocess may be defined, and it is commonl y referred to as " white noi se"(an example is the time derivative of Brownian motion).(c) Markov Processes. In 1907 A. A. Markov publi shed a paper [MARK07] in which he defined and investigated the properties of what are nowknown as Markov processes. In fact, what he created was a simple andhighly useful form of dependency among the random vari ables forming astochasti c process, which we now describe.A Markov proces s with a di screte state space is referred t o as a Markovchain. The discrete-time Markov chain is the easiest to conceptualize andunderstand. A set of random variables {Xn} forms a Markov chain if thepr obability that the next value (state) is X n+1 depends onl y upon the currentvalue (st ate) X n and not upon any previous values. Thus we have a randomsequence in which the dependency extends backwards one unit in time. That22 SOME IMPORTANT R A N D O ~ I PROCESSESis, the way in which the ent ire past history affects the future of the process iscomplet ely summarized in the current value of the process.In t he case of a discrete-time Markov chain the instants when state changesmay occur are preord ained to be at the integers 0, 1,2, . . . , n, . . . . In thecase of the continuous-t ime Markov chain, however, the transitions bet weenstates may take place at any instant in time. Thu s we are led to consider therand om variable that describes how long the process remains in its curr ent(discrete) state before making a tr ansition to some ot her state . Because theMarkov pr operty insists that the past history be compl etely summarized inthe specification of the current state, then we are not free to require that aspecification also be given as to how long the proce ss has been in its currentstate ! This imposes a heavy constraint on the distributi on of time that theprocess may remain in a given state. In fact , as we shall see in Eq. (2.85),"this state time must be exponent ially distributed. In a real sense, then , theexponential distributi on is a continuous distribution which is "rnemoryless"(we will discuss this not ion at considerable length later in thi s chapter).Similarl y, in the discrete-time Markov chain , the process may remain in thegiven state for a time that must be geometrically distributed ; thi s is the onlydiscrete pr obability mass funct ion that is memoryless. Thi s memorylessproperty is requi red of all Markov chains and restri cts the generality of theprocesses one would like to cons ider .Expressed analytically the Marko v property may be written asP[X(tn+l) = xn+1 1 X(tn) = Xn, X( tn_1) = Xn_l>' . . ,X(t l) = xtl= P[X(t n+l) = xn+1 I X(tn) = xnl (2.36)where t1 < t2 < . .. < I n < tn+1 and X i is included in some discrete statespace.The consideration of Markov processes is central to the study of queueingtheory and much of thi s text is devoted to that study. Therefore, a goodporti on of thi s chapter deals with discrete-and continuous-time Mar kovchains.(d) Birth-death Processes. A very important special class of Mar kovchains has come to be known as the birth-death process. These may be eitherdiscrete-or continuous-time processes in which the defining condit ion is thatstate transiti ons take place between neighboring states only. That is, one maychoose the set of integer s as the discrete state space (with no loss of generality)and then the birth-death process requires that if Xn = i, then Xn+l = i - I,i, or i + I and no other. As we shall see, birth-death processes have played asignificant role in the development of queueing theory. For the moment ,however , let us proceed with our general view of stochastic processes to seehow each fits int o the gener al scheme of things.2.2. DEFINITlO N AND CLASSIFICATI ON OF STOCHASTI C PROCESSES 23(e) Semi-Markov Processes. We begin by discussing discrete-timesemi-Ma rkov processes. The discrete-time Markov chain had the propert ythat at every unit inter val on the time axi s the process was required to make atransition from the current state to some other state (possibly back to thesame state). The transition probabilities were completely arbitrary; however ,the requirement that a transition be made at every unit time (which reallycame about because of the Markov property) leads to the fact that the timespent in a state is geometrically distributed [as we shall see in Eq. (2.66)].As mentioned earlier, this imposes a strong restriction on the kinds ofprocesses we may consider. If we wish to relax that restriction, namel y, topermi t an arbitrary distribution of time the process may remain in a state,then we are led directly into the notion of a discrete-time semi-Markovprocess; specifically, we now permit the times between state transitions toobey an arbitrary probability distribution. Note , however, that at the instantsof state tran sition s, the process behaves just like an ordinary Markov chainand, in fact , at those instants we say we have an imbedded Markov chain.Now the definition of a continuous-time semi-Markov pr ocess followsdirectly. Here we permit state transitions at any instant in time. However, asopposed to the Mar kov process which required an exponentially distributedtime in state, we now permit an arbitrary distribution. Thi s then affords usmuch greater generality, which we are happy to employ in our study ofqueueing systems. Here , again , the imbedded Markov process is defined atthose instants of state transition. Certainly, the class of Markov processes iscontained within the class of semi-Markov processes.(f) Random Walks. In the study of random processes one often en-counters a process referred to as a random walk . A random walk may beth ought of as a particle moving among states in some (say, discrete) statespace. What is of interest is to identify the location of the particle in that statespace. The salient feature cf a rand om walk is that the next position thepr ocess occupies is equal to the previ ous position plus a random variablewhose value is drawn independently from an arbitrary distribution ; thi sdistribution, however, does not change with the state of the process. * Th at is,a sequence of random variables {S n} is referred to as a random walk (sta rtingat the ori gin) ifSn = X, + Xz + . .. + Xn n = I, 2, . . . (2.37)where So = 0 and X" Xz, .. . is a sequence of independent random variableswith a common distributi on. The index n merely counts the number of statetransitions the process goes through ; of course, if the instants of thesetransitions are taken from a discrete set , then we have a discrete-time random* Except perhaps at some boundary states.24 SOME IMPORTANT RANDOM PROCESSESwalk, whereas if they are taken from a continuum, then we have a continu ous-time random walk. In any case , we assume that the interval between thesetr an sitions is di stributed in an arbitrary way and so a random walk is aspecial case of a semi-Ma rkov process. * In the case when the commondi stribution for Xn is a di screte distribution, then we have a di screte- st at erandom wal k; in thi s case the transiti on probabil ity Pi; of goi ng from sta te ito stat e j will depend only up on the difference in indices j - i (which weden ote by q;_;).An exa mple of a continuous-t ime rand om walk is that of Brownian mot ion;in the disc rete-time case an exa mple is the total number of heads observed in asequence of independent coin tosses.A random walk is occasionally referred to as a process with " independentincrements. "(g) Renewal Processes. A renewal process is related] to a random walk.However, the interest is not in following a pa rticle among many states butrather in counting transitions that take place as a functi on of time . That is,we consider the real time axi s on which is laid out a sequence of points; thedistribution of time between adj acent point s is an a rbitrary common distri-bution and each point corresponds to an instant of a state transition. Weass ume that the process begins in sta te 0 [i.e., X(O) = 0] a nd increases byunity at each transiti on epoch ; that is, X(t) equals the number of state tran-siti ons that ha ve taken place by t. In thi s sense it is a special case of a randomwalk in which q, = I and q; = 0 for i ~ I. We may think of Eq. (2.37) asdescribing a rene wal pr ocess in which S; is the random variabl e denot ing thetime at which the nt h tr ansiti on tak es place. As earl ier , the sequence {Xn } is aset of independe nt identicall y distributed random variab les where Xn nowrepresent s the time bet ween the (n - I)th and nth tr ansition. One should be .careful to distinguish the interpret ati on of Eq. (2.37) when it applies torenewal pr ocesses as here and when it applies to a random walk as earlier.The difference is that here in the renewal process the equat ion describes thetime of the nth renewal or transition , whereas in the rand om walk it describesthe state of the pr ocess and the time between sta te tr ansitions is some ot herrand om varia ble.An importa nt example of a renewal process is the set of arrival instantsto the G/G/m queue. In this case, Xn is identi fied with the interarrivaI time. Usually, the distribution of time between intervals is of lillieconcern in a randomwalk;emphasis is placed on the value (position) Sn after n transitions. Often, it is assumed thatthis distributionof interval time is memoryl ess, thereby making the randomwalka specialcase of Markov processes; we are more generous in our defi nition here and permit anarbitrary distribution.t It maybe considered to be a special caseof the randomwalk as defined in (f) above. Arenewal process is occasionally referred to as a recurrent process.2.2. DEFINITION AND CLASSIfiCATION OF STOCHASTIC PROCESSES 25MPPi j arbitraryS ~ l PPrj arbitraryIT arbitrary RWn.: qj - iIT arbi traryRPq, = 1ITarbitraryFigure 2.4 Relationships among the interesting random processes. SMP: Semi-Markov process; MP: Markov process; RW: Random walk; RP: Renewal process;BD: Birth-Death Process.So there we have it-a self-consistent classification of some interestingstoc hastic processes. In order to aid the reader in understanding the relation-ship among Markov .pr ocesses, semi-Markov processes, and their specialcases, we have prepared the diagram of Figure 2.4, which shows thi s relation-ship for discrete-state systems. The figure is in the form of a Venn diagram.Moreover , the symbol Pii denotes the probability of making a transiti on nextto state j given that the process is currently in state i. Also, fr den otes thedistribution of time between transitions; to say that "fr is mernoryless"implies that if it is a discrete-time process, thenfr is a geometric distributi on ,whereas if it is a continuous-time process, then fr is an exponential distri-buti on. Furthermore, it is implied that fr may be a functi on both of thecurrent and the next state for the pr ocess.The figure shows that birth-death processes form a subset of Markovpr ocesses, which themselves form a subset of the class of semi-Markovprocesses. Similarl y, renewal processes form a subset of random walkpr ocesses which also are a subset of semi-Markov processes. Moreover ,there are some renewal processes that may also be classified as birth-death26 SOME IMPORTANT RANDOM PROCESSESprocesses. Simil arl y, those Markov processes for which PH = q j - i (tha t is,where the transit ion probabilities depend only upon the di fference of theindices) overla p those random walks whercj', is memor yless. A rand om walkfor which [, is memo ryless and for which q j- i = 0 when Ij - i/ > I overlapsthe class of birth-death processes. If in addition to thi s last requirement ourrando m walk has q, = I , then we have a process that lies at the intersectionof all five of the processes shown in the figure. This is referred to as a "purebirth" pr ocess ; alt hough /, must be. memoryless, it may be a dist ributionwhich depends upon the state itself, ii ]; is independent of the state (thusgiving a const ant " birth rate" ) then we have a process that is figuratively andliterally at the "center" of the st udy of st ochastic pr ocesses and enjoys thenice properti es of each ! This very special case is referred to as the Poissonprocess and plays a major role in queueing the ory. We shall develop itsproperties later in thi s chapter.So much for the classificati on of stochas tic processes at this point. Let usnow elab orate up on the definition and properties of discret e-state Markovprocesses. Thi s will lead us naturall y int o some of the elementary queueingsystems. Some of the required theory behind the more sop histicated contin-uous-stat e Markov processes will be developed later in this work as the needarises. We begin with the simpler di screte-state , di screte-time Markovchains in the next section and foll ow that with a section on di screte-state,continuous-t ime Markov chains.2.3. DISCRETE-TL\tIE MARKOV CHAINS'As we have said , Markov processes may be used to describe the motion ofa particle in some space. We now consider di screte-t ime Mar kov chai ns,which permit the particle to occupy discrete positions and permi t transiti onsbet ween these positions to take place only at di screte times. We present theelements of t he theor y by carrying along the foll owi ng contemp orary exa mple.Consider the hipp ie who hitchhikes from city to cit y acr oss the country.Let Xn denote the city in which we find our hippie at noon on day n. Whenhe is in some particular city i, he will accept the first ride leavi ng in theevening from tha t city. We assume that the tr avel time betwee n any two cit iesis negligible . Of course, it is possible that no ride comes alo ng, in whichcase he will remain in city i until the next evening. Since vehicles head ing forvarious neighboring cities come along in some unpredi ct able fashion , thehippie' s posi tion at some time in the fut ure is cle