A Modern Tool for Approximative Queueing Analysis: Theory and Practice

ACM Sigmetrics 2010

Jens SchmittTU Kaiserslautern

Florin CiucuT-Labs / TU-Berlin

-A Tutorial-

2

The Problem of QueueingThe Problem of Queueing

… is about analyzing queue size (backlog), loss, delay, e.g.,

3

• Each call is allocated a fixed chunk of network’s capacity

• Goal: size the network so that the queue is empty most of time (almost zero blocking probabilities)

Queueing in Circuit Switching Networks, e.g., Queueing in Circuit Switching Networks, e.g., Telephone NetworkTelephone Network

4

• All flows share the available bandwidth by interleaving packets (statistical multiplexing)

• Goal: analysis of loss, delay (important for voice/video applications)

Queueing in Packet Switching Networks, e.g., Queueing in Packet Switching Networks, e.g., the Internetthe Internet

5

Behind the Title: The Stochastic Network Calculus (SNC)Behind the Title: The Stochastic Network Calculus (SNC)

• Probabilistic extension of the Deterministic Network Calculus (DNC), which− Was conceived by R. Cruz in the early 1990’s − Is regarded as the theory for deterministic queueing analysis

• SNC is a tool (technique) for stochastic queueing analysis− It’s approximative: provides rigorous bounds on queueing

measures but no guarantees on the gap to optimal results− Can solve queueing problems which other alternative tools cannot

• It works like− Deterministic evaluation of sample paths using the DNC− Probabilistic evaluation of sample paths (e.g., using large

deviations arguments)

• Our goal is to convey that SNC is useful (…and approachable)

6

Relationship with Existing Methods for Queueing Relationship with Existing Methods for Queueing AnalysisAnalysis

• Single-queue analysis is broadly understood using Queueing Theory, Matrix Geometric Methods, Effective Bandwidth− Solutions are tailored for each queueing problem

• Queueing network analysis is largely restricted to Poisson arrivals (BCMP, Kelly networks)

• SNC provides a uniform network queueing algebra for broad classes of arrivals/scheduling/service and has two main features1. ‘‘Scheduling Abstraction’’: abstracts away the details of

scheduling by a uniform service representation2. ‘‘Convolution Form Networks”: abstracts the network

service by a single-node view

7

Feature 1: ‘‘Scheduling Abstraction”Feature 1: ‘‘Scheduling Abstraction”

Feature 2: ‘‘Convolution Form Networks”Feature 2: ‘‘Convolution Form Networks”

…

9

Queueing Theory vs. SNC. Queueing Theory vs. SNC. M/M/1 QueueM/M/1 Queue

10

Outline of the TutorialOutline of the Tutorial

• Theory− Arrival/Service processes− Feature 1: ‘‘Scheduling Abstraction”− Single-Node queueing measures (backlog, delay, output)− Feature 2: ‘‘Convolution Form Networks”

• Applications− Reanalysis of classical queueing systems (M/M/1, M/D/1)− End-to-End delays in a packet network with cross traffic− End-to-End analysis of networks with heavy-tailed and

self-similar traffic− Output behavior at overloaded queues

• Insights on the SNC bounds accuracy. Conclusions

11

Arrivals at a Queue Arrivals at a Queue Arrival Process Arrival Process

• … as a Point Process

• … as a Marked Point Process

• … as a Fluid Process

12

Arrival ProcessArrival Process

• Also called Workload Process

• Bivariate extension

• Non-negative, non-decreasing, left-continuous

• Time model: in this tutorial mostly discrete

13

ApproximateApproximate the Arrival Process the Arrival Process

• Estimation of backlog/delay distributions reduces to theTail Estimation Problem, e.g.,

− Closed-form solutions/numerical evaluation is difficult, even for the Poisson process

• One may look for approximations (bounds) of the form

• The tail estimation problem remains

• Basic idea: assume the existence of bounds on either• Tail (distribution) or• MGF

14

Arrival Approximation 1: Tail BoundArrival Approximation 1: Tail Bound

• Formally defined as

• Note that− provides a stationary bound but is not

required to be stationary

− is not necessarily linear in

15

Example 0: Reduction to Deterministic BoundExample 0: Reduction to Deterministic Bound

16

Example 1: Hyperexponential BoundExample 1: Hyperexponential Bound

• Can be constructed for − Markov-Modulated processes, measured data

• Arbitrarily close approximations for Pareto, Weibull distributions

• Classical case (in SNC) with only one exponential (Exponentially Bounded Burstiness – EBB)− Notation simplifies but accuracy lessens

17

Example 2: Heavy-Tailed Self-Similar Example 2: Heavy-Tailed Self-Similar

• Can be constructed for− stable distributions− Compound point processes with Pareto increments− Multiplexed heavy-tailed On-Off− Measured data, e.g., aggregate Internet traffic

18

Arrival Approximation 2: MGF BoundArrival Approximation 2: MGF Bound

• Formally defined as

• Note that− It captures bounds on all the moments of the arrivals

− Inapplicable to heavy-tailed arrivals

• Classical example with only one exponential (EBB)− Notation simplifies but accuracy lessens

19

Relationship Between Tail and MGF boundsRelationship Between Tail and MGF bounds

Recall definitions for single exponentials

20

Tail and MGF Bound Constructions.Tail and MGF Bound Constructions.D/M ArrivalsD/M Arrivals

21

Tail Bound Construction. Tail Bound Construction. D/Pareto ArrivalsD/Pareto Arrivals

22

ApproximateApproximate the Service Process the Service Process

• Problem: How to represent ’s service in order to achieve ‘‘Scheduling Abstraction” (Feature 1)?− Or … How to abstract away the details of many

scheduling algorithms by a uniform service representation?

• To get the idea consider two cases depending on whether the server is1. always busy or2. sometimes idle

23

Case 1: Always Busy ServerCase 1: Always Busy Server

• Observations− Busy means that there is always backlog to serve− R.V. denotes the maximum possible service at time− Time varying service capacity (e.g., due to cross traffic)

• Then the cumulative offered service in time interval

24

Case 2: Sometimes Idle ServerCase 2: Sometimes Idle Server

• Denote the beginning of the last busy period before

• Observe that• Then

• Note: arrival and departure processes are related!

25

Cases 1 + 2: Service ProcessCases 1 + 2: Service Process

• Service process abstractly characterized by

• Observations− defined as a bivariate random process− convolution provides a probabilistic lower

bound (note the inequality) on ’s service (i.e., service guarantees)

26

Analogy with the Standard ConvolutionAnalogy with the Standard Convolution

27

A First Glimpse into Multi-Node Analysis.A First Glimpse into Multi-Node Analysis.Recall Feature 2: ‘‘Convolution Form Networks”Recall Feature 2: ‘‘Convolution Form Networks”

• Remark: computation of end-to-end queueing measures involves a ‘convolution’

• Problem: What if the two service processes are not statistically independent?

• Basic idea: ‘‘Move the randomness” of service processes

28

Moving the Randomness of a Service Process: Moving the Randomness of a Service Process: Stochastic Service CurveStochastic Service Curve

29

(General) Service Process(General) Service Process

• Formally defined as

• Observations− is a bivariate random process− is a non-random error function− Some randomness (dependencies) may be moved in

• This way it is possible to convolve originally dependent service processes, by considering instead the following

30

Example 0: Reduction to Deterministic Service CurveExample 0: Reduction to Deterministic Service Curve

31

Examples 1+2: Stochastic Service CurvesExamples 1+2: Stochastic Service Curves

Can be constructed from:− Scheduling assumptions + hyperexponentially (or

heavy-tailed self-similar) bounded arrivals − Measurements + fitting

32

Achieving ‘‘Scheduling Abstraction” (Feature 1)Achieving ‘‘Scheduling Abstraction” (Feature 1)

33

Some Assumptions for Achieving Some Assumptions for Achieving ‘‘Scheduling Abstraction” ‘‘Scheduling Abstraction”

• Upper bounds on the (aggregate) cross traffic− Recall that a service (curve) process sets lower bounds

• Server with capacity guarantees• Driving factors:

− the scheduling algorithm− (choice of fluid/packetized service model (t.b.d. later))

34

Scheduling Example 1: Arbitrary SchedulingScheduling Example 1: Arbitrary Scheduling

35

Scheduling Example 1: Arbitrary Scheduling (cont.)Scheduling Example 1: Arbitrary Scheduling (cont.)

36

Scheduling Example 2: FIFOScheduling Example 2: FIFO

37

More on the Versatility of the (min,+) Convolution More on the Versatility of the (min,+) Convolution for Approximating the Service Processfor Approximating the Service Process

• So far … approximations of (per-flow) service process in terms of capacity guarantees, i.e.,

− These imply (per-flow) delay guarantees (t.b.d. later)

• But is it possible to directly approximate the (per-flow) service process in terms of delay guarantees?

38

Approximate the Service Process with Approximate the Service Process with Delay GuaranteesDelay Guarantees

39

Approximate the Service Process with Approximate the Service Process with Delay Guarantees. Example: ExponentialDelay Guarantees. Example: Exponential

40

• Possible when per-flow/per-node service processes expressed in terms of delay guarantees

• …or capacity guarantees, or combinations of the two, e.g.,

A Second Glimpse into Multi-Node Analysis.A Second Glimpse into Multi-Node Analysis.Recall Feature 2: ‘‘Convolution Form Networks”Recall Feature 2: ‘‘Convolution Form Networks”

41

Wrap-up on Probabilistic Approximation of Wrap-up on Probabilistic Approximation of Arrival/ServiceArrival/Service

42

Derivation of Queueing MeasuresDerivation of Queueing Measures

• Recall that SNC provides a ‘‘Uniform Queueing Algebra” for queueing systems− Based on ‘‘Scheduling Abstraction” (Feature 1) and the

idea of ‘‘Convolution Form Networks” (Feature 2) − Main purpose: derivation of per-flow measures (backlog

and delay process, output characterization)

• Main steps for per-flow queueing algebra1. Arrival representation with a tail/MGF bound2. (Per-flow) service representation with a service process3. Estimation of sample-path events, e.g.,

43

The Root of Sample Path Events: History Matters … The Root of Sample Path Events: History Matters …

• Lindley’s equation for w.-c. server with rate

− Captures the history of the queueing process • Leads to Reich’s equation

• Relation of with Reich’s equation

44

Bounding The Backlog Process. Bounding The Backlog Process. Case 1: Independent Arrivals/ServiceCase 1: Independent Arrivals/Service

45

Bounding The Backlog Process. Bounding The Backlog Process. Case 1: Independent Arrivals/Service (contd.)Case 1: Independent Arrivals/Service (contd.)

46

Bounding The Backlog Process. Bounding The Backlog Process. Case 2: Not Necessarily Ind. Arrivals/ServiceCase 2: Not Necessarily Ind. Arrivals/Service

47

Bounding The Backlog Process. Bounding The Backlog Process. Case 3: Tail Bounds AssumptionsCase 3: Tail Bounds Assumptions

48

Bounding The Backlog Process. Bounding The Backlog Process. Case 4: Delay Guarantees.Case 4: Delay Guarantees.

49

Bounding The Delay Process. Bounding The Delay Process. Only the Case of Independent Arrivals/ServiceOnly the Case of Independent Arrivals/Service

50

Bounding The Output Process. Bounding The Output Process. Only the Case of Independent Arrivals/ServiceOnly the Case of Independent Arrivals/Service

51

Bounding The Bounding The MeanMean of Backlog/Delay Processes. of Backlog/Delay Processes. Only the Case of Independent Arrivals/ServiceOnly the Case of Independent Arrivals/Service

52

Multi-Node Analysis.Multi-Node Analysis.Recall Feature 2: ‘‘Convolution Form Networks”Recall Feature 2: ‘‘Convolution Form Networks”

…

53

Construction of a Network Service ProcessConstruction of a Network Service ProcessCase 1: Zero Error FunctionsCase 1: Zero Error Functions

…

54

Construction of a Network Service Process.Construction of a Network Service Process.Case 1: Zero Error Functions - ConclusionCase 1: Zero Error Functions - Conclusion

…

55

Construction of a Network Service ProcessConstruction of a Network Service ProcessCase 2: Positive Error FunctionsCase 2: Positive Error Functions

…

56

Construction of a Network Service ProcessConstruction of a Network Service ProcessCase 2: Positive Error Functions (contd.)Case 2: Positive Error Functions (contd.)

…

57

Construction of a Network Service ProcessConstruction of a Network Service ProcessCase 2: Positive Error Functions - ConclusionCase 2: Positive Error Functions - Conclusion

…

58

Is the Network Service Process Useful?Is the Network Service Process Useful?… or, Are ‘‘Convolution Form Networks” Useful?… or, Are ‘‘Convolution Form Networks” Useful?

• Not without either a tail/Laplace bound, i.e.,

• In conjunction with tail/MGF bounds for , i.e.,

• … one can derive network queueing measures simply by applying single-node results

59

Construction of Laplace Bounds for a Network Construction of Laplace Bounds for a Network Service ProcessService Process

60

Application 1: Reanalyzing Classical Queueing Application 1: Reanalyzing Classical Queueing Systems. Case 1: M/M/1Systems. Case 1: M/M/1

61

Application 1: Reanalyzing Classical Queueing Application 1: Reanalyzing Classical Queueing Systems. Case 2: M/D/1Systems. Case 2: M/D/1

62

Application 2: End-to-End Delay in a Packet Application 2: End-to-End Delay in a Packet Network with (EBB) Cross TrafficNetwork with (EBB) Cross Traffic

• The network scenario

• All arrivals are marked point processes, e.g.,

• Other assumptions

• Because of compound arrivals, a new service model for abstracting the packetization process is needed

…

63

Construction of Service Process for Packetization.Construction of Service Process for Packetization.Case 1: No Cross TrafficCase 1: No Cross Traffic

64

Construction of Stochastic Service Curve for Construction of Stochastic Service Curve for Packetization. Case 1: No Cross trafficPacketization. Case 1: No Cross traffic

65

Construction of Service Process for Packetization.Construction of Service Process for Packetization.Case 2: Cross TrafficCase 2: Cross Traffic

66

Construction of Service Process for Packetization.Construction of Service Process for Packetization.Case 2: Cross Traffic (contd.)Case 2: Cross Traffic (contd.)

67

Construction of Service Process for Packetization. Construction of Service Process for Packetization. Case 2: Cross TrafficCase 2: Cross Traffic

68

Construction of Stochastic Service Curve for Construction of Stochastic Service Curve for Packetization. Case 2: Cross TrafficPacketization. Case 2: Cross Traffic

69

End-to-End Delay in a Packet Network with (EBB) End-to-End Delay in a Packet Network with (EBB) Cross Traffic (contd.)Cross Traffic (contd.)

…

70

End-to-End Delay in a Packet Network with (EBB) End-to-End Delay in a Packet Network with (EBB) Cross Traffic (contd.)Cross Traffic (contd.)

…

71

End-to-End Delay in a Packet Network with (EBB) End-to-End Delay in a Packet Network with (EBB) Cross Traffic - ConclusionCross Traffic - Conclusion

72

Application 3: End-to-End Analysis of a Network with Application 3: End-to-End Analysis of a Network with Heavy-Tailed/Self-Similar (Cross) TrafficHeavy-Tailed/Self-Similar (Cross) Traffic

• The network scenario

• All arrivals are htss, e.g.,

• Applying earlier sample-path arguments would yield end-to-end bounds of the form

• Idea: geometric sample-paths

…

73

Geometric Sample-PathsGeometric Sample-Paths

• Recall sample-paths so far

• Time was discrete and the sampling was arithmetic

• For the htss application we will sample time at the points of the geometric series

• As we now work in continuous time we also have the time parameter

74

Example: Construction of a Stochastic Network Example: Construction of a Stochastic Network Service Curve. Only 2 NodesService Curve. Only 2 Nodes

75

Construction of a Stochastic Network Service Construction of a Stochastic Network Service Curve. Only 2 Nodes (contd.)Curve. Only 2 Nodes (contd.)

76

Construction of a Network Service Curve Process. Construction of a Network Service Curve Process. Only 2 Nodes - ConclusionOnly 2 Nodes - Conclusion

77

Application 3: End-to-End Analysis of a Network with Application 3: End-to-End Analysis of a Network with Heavy-Tailed/Self-Similar (Cross) Traffic - ConclusionHeavy-Tailed/Self-Similar (Cross) Traffic - Conclusion

…

• Construction of per-node stochastic service curves and the derivation of queueing measures follow similar geometric sample-path arguments

• Explicit end-to-end delay bounds can be thus obtained

• No statistical independence of the arrivals was assumed

78

Application 4. Output Behavior at Application 4. Output Behavior at Overloaded Queues Overloaded Queues

• The network scenario

• The arrival processes have long-term rates

• Assume the overloaded queue case, i.e.,

• Problem: What can one say about the tail of , i.e.,

79

Output Behavior at Overloaded Queues; FIFOOutput Behavior at Overloaded Queues; FIFO

80

Output Behavior at Overloaded Queues; FIFO; EBB Output Behavior at Overloaded Queues; FIFO; EBB ArrivalsArrivals

81

Output Behavior at Overloaded Queues; FIFO; Output Behavior at Overloaded Queues; FIFO; EBB Arrivals. The DerivationEBB Arrivals. The Derivation

82

Output Behavior at Overloaded Queues; FIFO; Output Behavior at Overloaded Queues; FIFO; EBB Arrivals. The Derivation - ConclusionEBB Arrivals. The Derivation - Conclusion

83

On the Accuracy of the Derived SNC BoundsOn the Accuracy of the Derived SNC Bounds

84

On the Accuracy of the Derived SNC Bounds.On the Accuracy of the Derived SNC Bounds.Is the Boole/Chernoff Combination Tight?Is the Boole/Chernoff Combination Tight?

• No, this is already known from the Effective Bandwidth literature− Implicitly, the same holds for the SNC literature (SNC uses roughly

the same large deviation techniques as in the EB literature for estimating sample-paths events)

• In particular, the application of the Chernoff bound, though attractive, can be loose in statistical multiplexing regimes, i.e., when

• Tight improvement over the Chernoff bound by using the Bahadur-Rao result for large deviations

− Yet to be incorporated in the SNC literature− The benefits of doing so would be in the derivation of per-flow end-

to-end results (recall Features 1 and 2: ‘‘Scheduling Abstraction’’ and ‘‘Convolution Form Networks”)

85

On the Accuracy of the Derived SNC Bounds.On the Accuracy of the Derived SNC Bounds.What About Boole’s Inequality?What About Boole’s Inequality?

86

On the Accuracy of the Derived SNC Bounds.On the Accuracy of the Derived SNC Bounds.Conclusions.Conclusions.

• The accuracy of SNC bounds strongly depends on available results in large deviations

• A challenge remains the development of tighter bounds, than Boole inequality, for the multi-node analysis− Recall the Laplace estimate for the convolution of two service

processes

• Conjecture: In principle, the key SNC concept of a service process (or stochastic service curve) can be incorporated in existing techniques for queueing analysis and achieve− Per-flow results by ‘‘Scheduling Abstraction’’ and ‘‘Convolution

Form Networks”− Arbitrarily tight bounds

87

Conclusions of the TutorialConclusions of the Tutorial

• The Stochastic Network Calculus is a relatively recent (~20 years) tool, or technique, for queueing analysis

• Main idea: abstracts away technical difficulties related to arrival/scheduling/service/multi-node by using bounds− On arrivals (tail/MGF bounds)− On service (service process, stochastic service curve)

• Key benefits of using SNC− Broad arrival/service classes, statistical (in)dependence− ‘‘Scheduling Abstraction’’ and ‘‘Convolution Form Networks”− Closed-form results− Accuracy of the bounds: in principle as tight as related results

from probability theory

• In a nutshell, SNC is a simplified queueing algebra by applying the principle: ‘‘It’s easier to approximate!”