Utility Optimization for Event-Driven Distributed Infrastructures Cristian Lumezanu University of...

Utility Optimization for Event-Driven Distributed

Infrastructures

Cristian LumezanuUniversity of Maryland, College Park

Sumeer Bhola Mark AstleyIBM T.J. Watson Research Center

Utility Optimization for Event-Driven Distributed Infrastructures ICDCS 2006 2

Event Driven Infrastructure

EVENT DRIVEN INFRASTRUCTU

RE

Producers

Consumers

Consumers


Event Driven Infrastructure

Consumers

Consumers

ProducersFlow

Flow

Publish/subscribeStream processing

overlaysEnterprise Service Bus


Flows and Consumer Classes

Consumers

MAXIMIZE SYSTEM UTILITY j j in U (r)

Flow

Consumers

ProducersFlow

FLOW 1Rate: r1

FLOW 2Rate: r2

CONSUMER CLASS 1Number of consumers: n1

Utility: U1(r1)

CONSUMER CLASS 2Number of consumers: n2

Utility: U2(r2)


Model Summary

• Network of nodes interconnected by links• Flows and classes of consumers• Control variables:

• Flow rates (for rate control)• Number of consumers (for admission control)

• Utility function• Associated with each consumer• Depends on the rate of the flow that serves the consumer• Assumed to be strictly concave


Optimization Problem

Consumers

ProducersFlow

Lii flows(L

L)

L cir

LINK CONSTRAINT

Ai Aj

i flows(A) j classes(Ai i

,i)AB C cAjr n r

NODE CONSTRAINT

Find the rate allocation and the number of consumerssuch that the total utility of the system is maximized

and the constraints are satisfied

jj classes

max U ( )

j in r

OBJECTIVE


Optimization Problems

Lii flows(L

L)

L cir

LINK CONSTRAINT

Ai Aj

i flows(A) j classes(Ai i

,i)AB C cAjr n r

NODE CONSTRAINT

Find the rate allocation and the number of consumerssuch that the total utility of the system is maximized

and the constraints are satisfied

jj classes

U ( )j in r

SYSTEM UTILITY Optimization depends on both rate allocation and consumer allocation

System utility not concaveConstraint set not convex


LRGP

LAGRANGIAN RATES, GREEDY POPULATIONS

Finds the optimal rates

for each flow at a certain

moment given a constant number of consumers

Finds the optimal number of consumers for each class at a certain moment given constant

flow rates

OPTIMIZATION PROBLEMRATE

ALLOCATIONCONSUMER

ALLOCATIONPRICE

COMPUTATION

Makes trade-offs between rate control and admission

control


Consumer Allocation

Increasing the number of consumers has a local effect on the system

1. Sort classes in decreasing order of their benefit/cost ratio

2. Allocate consumers for each class in the order established above until the node constraint is violated

utility nBC

resource n


Rate Allocation

Increasing the flow rates has a global effect on the system


Prices

Prices• associated with each resource (node, link)• reflect how congested the resource is• provide a way to control the rate

Node Price• implements a trade-off between “increasing the number of

consumers” and “increasing the rate”• reflects the maximum benefit/cost ratio of the node

Link Price• adjusted using a gradient projection algorithm (Low et al.)


LRGP

1. Each node resource performs local consumer allocation2. Each resource computes a new price…

3. …and sends it to the sources of the flows that go through the resource

4. Each source computes a new rate for its flow…5. …and sends the rate to all nodes and links in the path of the flow

1. Each node resource performs local consumer allocation




2. Each resource computes a new price…





4. Each source computes a new rate for its flow…


Results• Several workloads• Several utility functions

•

• CONVERGENCE• How fast does it reach the result?• LRGP converges in less than 50 iterations

• OPTIMALITY• How good is the result?• LRGP achieves better utility than a centralized simulated

annealing algorithm

kj jU rank f(r) log( f(r) ,r1: r)


Convergence

price benefit / costprice(t 1) (1 ) price(t) (benefit / cost)


Convergence

Adaptive : Incremental increase, multiplicative decrease


Optimality

ITERATIONS UNTIL CONVERGENCE• the number of iterations until convergence does not vary with

an increase in the number of flows or consumers

ROBUSTNESS• recovers quickly when flows or consumers are removed

UTILITY• comparison with a centralized simulated annealing (SA)

algorithm• 6 different workloads• LRGP finds a utility between 6.47% and 18.75% higher than SA


Conclusions and Future Work

CONCLUSIONS• distributed algorithm for optimizing utility in an event-driven

infrastructure• greedy approach to control the consumers + Lagrangian approach to

control the rates• prices make trade-offs between admission control and rate control• simulation results show good convergence and scalability

FUTURE WORK• other utility functions• asynchronous algorithm• other types of resources• implementation


Questions


Dual Problem

( ) ( ) ( ) ( , )

( , , ) log(1 )e l i ij j li ei ej ji j C i l L

l ii e E i j C e i

l el e

e

l e

L A n L B Cr p p r r n

c c

p r p

p p

( )

( ) ( , )

( )

subject to

max log(1 )

,

,

i

i

j ji

i

j C i

ei ej ej ei F e j C e i

li li

iF l

r

r r

A n

B C

r

n c e

L c l


Dual Problem

• the ‘r’ that maximizes the system utility, also maximizes L• L concave, thus it has only one maximum, given by:

( ) ( ) ( ) ( , )

( , ) max ( , , )

max log(1 )

e

j j li ei ej j

l r e l

i i l i ei j C i l L i e E i j C e i

le

l e el

D Lp p r p p

rA n L B C nr p r p

p pc c

( )

( ) ( , ) ( )

0, 1j j

j C ii

iei ej ej e li l

e E i j C e i l L i

A nL

i rr

B C n p L p

Find the rate allocation and the prices such that D(pe,pl) is

minimized


Recovery / Different utility

Recovery from system changes 0.75jrank rClass utility is

Utility Optimization for Event-Driven Distributed Infrastructures Cristian Lumezanu University of...

Documents

Transcript of Utility Optimization for Event-Driven Distributed Infrastructures Cristian Lumezanu University of...