· Joseph-Louis Lagrange (1736-1813) source: Wikipedia W.W.R. Ball, History of Mathematics (3rd...

Price Signalsin

Energy Optimization

Claudia Sagastizabal(IMECC-UNICAMP, adjunct researcher)

Workshop de Otimizacao

IM-UFRJRio, May 7, 2018

Joseph-Louis Lagrange (1736-1813)

source: Wikipedia

W.W.R. Ball, History of Mathematics (3rd Ed., 1901)

I regard as quite useless the reading of largetreatises of pure analysis: too large a number of

methods pass at once before the eyes. It is in theworks of application that one must study them; one

judges their utility there and appraises the manner ofmaking use of them.

Je considere comme completement inutile la lecture de grostraites d’analyse pure: un trop grand nombre de methodes

passent en meme temps devant les yeux. C’est dans les travauxd’application qu’on doit les etudier; c’est la qu’on juge leurs

capacites et qu’on apprend la maniere de les utiliser.

There are many great quotes by Lagrange . . .


source: Wikipedia

W.W.R. Ball, History of Mathematics (3rd Ed., 1901)

I regard as quite useless the reading of largetreatises of pure analysis: too large a number of

methods pass at once before the eyes. It is in theworks of application that one must study them; one

judges their utility there and appraises the manner ofmaking use of them.

Je considere comme completement inutile la lecture de grostraites d’analyse pure: un trop grand nombre de methodes

passent en meme temps devant les yeux. C’est dans les travauxd’application qu’on doit les etudier; c’est la qu’on juge leurs

capacites et qu’on apprend la maniere de les utiliser.There are many great quotes by Lagrange . . .


source: Wikipedia

Before we take to sea we walk on land.Before we create we must understand.

Following Lagrange’s leadin today’s talkwe shall walk on the landof energy optimizationand make a few interesting observations


source: Wikipedia

Thanks to Lagrangewe know electricity prices

1st obse

rvat

ion

Walking on the application’s landManage at minimum cost the different technologies of aninterconnected electric power system, so that demand is satisfied

source: ONS

Illustration with a simple example

Two power plants

pT ∈PT pH ∈PH

CT (pT ) CH(pH)

pT + pH = d (demand)

Illustration with a simple example min CT (pT ) + CH(pH)s.t. pT ∈PT ,pH ∈PH

pT + pH = dTwo power plants

pT ∈PT pH ∈PH

CT (pT ) CH(pH)

pT + pH = d (demand)

Pricing via marginal costThe perturbation function

p( u ) :=

inf CT (pT ) + CH(pH)s.t. pT ∈PT ,pH ∈PH

pT + pH = d+uis the key to price electricity:what is the cost increaseif demand increases in one unit?

Answer:compute p(·)’s directional derivative at 0

(p(0) is the optimal value of our problem)

Pricing via marginal cost: Danskin’s theoremFor max-functions, differentiating the argument provides the

directional derivative:

ϕ(u) = supx

F(x ,u) =⇒ ϕ′(u, ·) = 〈∇uF(x ,u), ·〉

p(u) = sup?? of the Lagrangian!

p(u) =

{inf

p∈PCT (pT ) + CH(pH)

s.t. pT + pH = d + u ← x + ↑

= infp∈P

supx

CT (pT ) + CH(pH) + 〈x ,d + u−pT −pH〉

s.t. pT + pH = d + u



ϕ(u) = supx

F(x ,u) =⇒ ϕ′(u, ·) = 〈∇uF(x ,u), ·〉

p(u) = sup??

of the Lagrangian!

p(u) =

{inf


s.t. pT + pH = d + u ← x + ↑

= infp∈P

supx





ϕ(u) = supx

F(x ,u) =⇒ ϕ′(u, ·) = 〈∇uF(x ,u), ·〉


p(u) =

{inf


s.t. pT + pH = d + u ← x + ↑

= infp∈P

supx





ϕ(u) = supx

F(x ,u) =⇒ ϕ′(u, ·) = 〈∇uF(x ,u), ·〉


p(u) =

{inf



← x + ↑

= infp∈P

supx





ϕ(u) = supx

F(x ,u) =⇒ ϕ′(u, ·) = 〈∇uF(x ,u), ·〉


p(u) =

{inf


s.t. pT + pH = d + u ← x

+ ↑

= infp∈P

supx





ϕ(u) = supx

F(x ,u) =⇒ ϕ′(u, ·) = 〈∇uF(x ,u), ·〉


p(u) =

{inf


s.t. pT + pH = d + u ← x + ↑

= infp∈P

supx





ϕ(u) = supx

F(x ,u) =⇒ ϕ′(u, ·) = 〈∇uF(x ,u), ·〉


p(u) =

{inf



= infp∈P

supx


supx

infp∈P


supx{F(x ,u) := 〈x ,u〉+ constant terms in u}

Pricing via marginal cost: Danskin’s theoremAt u = 0 Lagrange multiplier x gives directional derivative

p(0) =

min CT (pT ) + CH(pH)s.t. pT ∈PT ,pH ∈PH

pT + pH = d+0

= minp∈P

maxx

CT (pT ) + CH(pH) + 〈x ,d−pT −pH〉

L(p,x) = CT (pT ) + CH(pH) + 〈x ,d−pT −pH〉is the Lagrangian

p(0) = minp∈P

maxx

L(p,x)

maxx

minp∈P

L(p,x)

thanks to Lagrangewe know electricity prices

Pricing via marginal cost: Danskin’s theoremAt u = 0 Lagrange multiplier x gives directional derivative

p(0) =

min CT (pT ) + CH(pH)s.t. pT ∈PT ,pH ∈PH

pT + pH = d+0= min

p∈Pmax

xCT (pT ) + CH(pH) + 〈x ,d−pT −pH〉

L(p,x) = CT (pT ) + CH(pH) + 〈x ,d−pT −pH〉is the Lagrangian

p(0) = minp∈P

maxx

L(p,x)

maxx

minp∈P

L(p,x)

thanks to Lagrangewe know electricity prices

Pricing via marginal cost

At u = 0 Lagrange multiplier x gives directional derivative

p(0) = minp∈P

maxx

L(p,x)

maxx

minp∈P

L(p,x)

without convexity

Pricing via marginal cost: there is a catch!


p(0) = minp∈P

maxx

L(p,x)

p∗∗(0) = maxx

minp∈P

L(p,x)

without convexity p(0)≥ p∗∗(0)= dualPros and cons+ Separability ( 6= technologies )

+ Accurate marginal prices in short CPU timeif using a bundle method with on-demand accuracy

(subgradient/Uzawa-like methods do not get enough precision)– Loss of primal feasibility



p(0) = minp∈P

maxx

L(p,x)

p∗∗(0) = maxx

minp∈P

L(p,x)

without convexity p(0)≥ p∗∗(0)= dual

Pros and cons+ Separability ( 6= technologies )





p(0) = minp∈P

maxx

L(p,x)

p∗∗(0) = maxx

minp∈P

L(p,x)

without convexity p(0)≥ p∗∗(0)= dualPros and cons+ Separability ( 6= technologies )



Illustration with a simple examplePower plants have same capacity (P) but different technology

0-1 variables continuous variablesgenerates either 0 or P any power in [0,P]

PT = {0,P} PH = [0,P]expensive cheap

Suppose demand is d = 1.5P so that both power plants aredispatched: pT = P and pH = 1

2PThe dual approach, that finds x via p∗∗(0), amounts to solvingan LP with PT = {0,P} replaced by conv(PT ) = [0,P]Because p∗∗ = conv(p) < p, the associated dispatch isinfeasible: pdual

T = 12P and pdual

H = P


0-1 variables continuous variables

generates either 0 or P any power in [0,P]PT = {0,P} PH = [0,P]

expensive cheapSuppose demand is d = 1.5P so that both power plants aredispatched: pT = P and pH = 1


T = 12P and pdual

H = P



PT = {0,P} PH = [0,P]

expensive cheapSuppose demand is d = 1.5P so that both power plants aredispatched: pT = P and pH = 1


T = 12P and pdual

H = P




Suppose demand is d = 1.5P so that both power plants aredispatched:

pT = P and pH = 12P

The dual approach, that finds x via p∗∗(0), amounts to solvingan LP with PT = {0,P} replaced by conv(PT ) = [0,P]Because p∗∗ = conv(p) < p, the associated dispatch isinfeasible: pdual

T = 12P and pdual

H = P





2P

The dual approach, that finds x via p∗∗(0), amounts to solvingan LP with PT = {0,P} replaced by conv(PT ) = [0,P]Because p∗∗ = conv(p) < p, the associated dispatch isinfeasible: pdual

T = 12P and pdual

H = P





2PThe dual approach, that finds x via p∗∗(0), amounts to solvingan LP with PT = {0,P} replaced by conv(PT ) = [0,P]

Because p∗∗ = conv(p) < p, the associated dispatch isinfeasible: pdual

T = 12P and pdual

H = P






T = 12P and pdual

H = P


For our simple problem p(u) ={

inf CT (pT )+CH (pH )s.t. pT ∈PT ,pH ∈PH

pT +pH = d +u

is the minimum of two quadratic functions

There is a duality gap

when p is not convex(p(0) > p∗∗(0))

p(0)→← p∗∗(0)




pT +pH = d +u



when p is not convex

(p(0) > p∗∗(0))

p(0)→← p∗∗(0)




pT +pH = d +u



when p is not convex(p(0) > p∗∗(0))

p(0)→← p∗∗(0)

Remarks:

Price signals in Energy Optimization

I are always related to dual variables

I nonconvexity complicates calculations

I direct 0-1 solution not possible:there are no dual variables!

Remarks:




I direct 0-1 solution not possible:

there are no dual variables!

Remarks:




I direct 0-1 solution not possible:there are no dual variables!

Another star? in decomposition methods

source: Twitter

A less simple example

Two power plants

pT ∈PT pH ∈PH

PT = [0,P] PH = [0,P]

x ∈ {0,1} and pT ≤ x P〈c,x〉+ CT (pT ) CH(pH)

pT + pH = d(demand)


Two power plants

pT ∈PT pH ∈PH

PT = [0,P] PH = [0,P]x ∈ {0,1} and pT ≤ x P

〈c,x〉+ CT (pT ) CH(pH)

pT + pH = d(demand)


Two power plants

pT ∈PT pH ∈PH

PT = [0,P] PH = [0,P]x ∈ {0,1} and pT ≤ x P〈c,x〉+ CT (pT ) CH(pH)

pT + pH = d(demand)

Optimization problemmin 〈c,x〉+CT (pT )+CH(pH)

s.t. x ∈ {0,1} ,pT ,pH ∈ [0,P]pT ≤ x PpT +pH = d

Optimization problem

min 〈c,x〉+ CT (pT ) + CH(pH)s.t. x ∈ {0,1} ,pT ,pH ∈ [0,P]

pT ≤ x PpT + pH = d

min 〈c,x〉+ Q(x)s.t. x ∈ {0,1}

Ax = bfor Q(x) :=

min CT (pT ) + CH(pH) ↔ C(p)s.t. pT ,pH ∈ [0,P] ↔ p ∈ [0,P]


}↔ Wp = h−Tx

Bender’s decomposition separates the optimizationproblem in two decision levels.




min 〈c,x〉+ Q(x)s.t. x ∈ {0,1}

Ax = bfor Q(x) :=



}↔ Wp = h−Tx

Bender’s decomposition separates the optimizationproblem in two decision levels.

If d or C uncertain,this is a 2-stage stochastic programming problem




min 〈c,x〉+ Q(x)s.t. x ∈ {0,1}

Ax = bfor Q(x) :=



}↔ Wp = h−Tx

Bender’s decomposition separates the optimizationproblem in two decision levels. If d or C uncertain,this is a 2-stage stochastic programming problem

Price Signals in 2-SLPmin 〈c,x〉+E [Q(x ,ω)]s.t. x ∈ {0,1}

Ax = bQ(x ,ω) =

min C(p)s.t. p ∈ [0,P]

Wp = h(ω)−Tx

← π(x ,ω)

SAA: ω ∈ {ω1, . . . ,ωS}⇒ E [Q(x ,ω)]≈ 1

S ∑s

Q(x ,ωs) and E [π(x ,ω)]≈ 1S ∑

sπ(x ,ωs)

A different sample: ω ∈ {ω1, . . . , ωS}⇒ E [Q(x ,ω)]≈ 1

S ∑s

Q(x , ωs) and E [π(x ,ω)]≈ 1S ∑

sπ(x , ωs)


Ax = bQ(x ,ω) =


Wp = h(ω)−Tx ← π(x ,ω)

SAA: ω ∈ {ω1, . . . ,ωS}⇒ E [Q(x ,ω)]≈ 1

S ∑s


sπ(x ,ωs)


S ∑s

Q(x , ωs) and E [π(x ,ω)]≈ 1S ∑

sπ(x , ωs)


Ax = bQ(x ,ω) =



SAA: ω ∈ {ω1, . . . ,ωS}

⇒ E [Q(x ,ω)]≈ 1S ∑

sQ(x ,ωs) and E [π(x ,ω)]≈ 1

S ∑s

π(x ,ωs)


S ∑s

Q(x , ωs) and E [π(x ,ω)]≈ 1S ∑

sπ(x , ωs)


Ax = bQ(x ,ω) =



SAA: ω ∈ {ω1, . . . ,ωS}⇒ E [Q(x ,ω)]≈ 1

S ∑s

Q(x ,ωs)

and E [π(x ,ω)]≈ 1S ∑

sπ(x ,ωs)


S ∑s

Q(x , ωs) and E [π(x ,ω)]≈ 1S ∑

sπ(x , ωs)


Ax = bQ(x ,ω) =



SAA: ω ∈ {ω1, . . . ,ωS}⇒ E [Q(x ,ω)]≈ 1

S ∑s


sπ(x ,ωs)


S ∑s

Q(x , ωs) and E [π(x ,ω)]≈ 1S ∑

sπ(x , ωs)


Ax = bQ(x ,ω) =



SAA: ω ∈ {ω1, . . . ,ωS}⇒ E [Q(x ,ω)]≈ 1

S ∑s


sπ(x ,ωs)

A different sample: ω ∈ {ω1, . . . , ωS}

⇒ E [Q(x ,ω)]≈ 1S ∑

sQ(x , ωs) and E [π(x ,ω)]≈ 1

S ∑s

π(x , ωs)


Ax = bQ(x ,ω) =



SAA: ω ∈ {ω1, . . . ,ωS}⇒ E [Q(x ,ω)]≈ 1

S ∑s


sπ(x ,ωs)


S ∑s

Q(x , ωs) and E [π(x ,ω)]≈ 1S ∑

sπ(x , ωs)


Ax = bQ(x ,ω) =



SAA: ω ∈ {ω1, . . . ,ωS}⇒ E [Q(x ,ω)]≈ 1

S ∑s


sπ(x ,ωs)


S ∑s

Q(x , ωs) and E [π(x,ω)]≈ 1S ∑

sπ(x, ωs)

source: Clara Lage

How much prices change? 1

[1]R&D contract with , joint with C. Lage and M. Solodov

Multiplier sensitivity to uncertainty representationA simple exercise

I Take a 2-SLP with uncertain right-hand side(h(ω))

I Given the sample ω ∈ {ω1m, . . . ,ω

Sm}:

I The SAAm has first-stage solution xmI The associated optimal mean price is

πm :=1S ∑

sπ(xm,ω

sm)

I Repeat for m = 1, . . . ,M and compute thevariance of π1, . . . , πM

Var [π] can be very large!

Stabilizing dual variablesTo mitigate wild oscillations, for β > 0 consider

Qβ (x ,ω) :=

min C(p) + 1

2β‖Tx + Wp−h(ω)‖2

s.t. p ∈ [0,P]

Wp = h(ω)−Tx ↗

I corresponds to a quadratic term β

2‖π‖2 in the

dualI if pβ = pβ (x ,ω) solves the regularized problem,

then

πβ (x ,ω) :=

1β

(Tx + Wpβ −h(ω)

)I xβ now solves

{min 〈c,x〉+E

[Qβ (x ,ω)

]s.t. Ax = b ,x ≥ 0


Qβ (x ,ω) :=

min C(p) + 1


s.t. p ∈ [0,P]

Wp = h(ω)−Tx ↗


2‖π‖2 in the

dual

I if pβ = pβ (x ,ω) solves the regularized problem,then

πβ (x ,ω) :=

1β

(Tx + Wpβ −h(ω)

)I xβ now solves

{min 〈c,x〉+E

[Qβ (x ,ω)

]s.t. Ax = b ,x ≥ 0


Qβ (x ,ω) :=

min C(p) + 1


s.t. p ∈ [0,P]

Wp = h(ω)−Tx ↗


2‖π‖2 in the


then

πβ (x ,ω) :=

1β

(Tx + Wpβ −h(ω)

)

I xβ now solves

{min 〈c,x〉+E

[Qβ (x ,ω)

]s.t. Ax = b ,x ≥ 0


Qβ (x ,ω) :=

min C(p) + 1


s.t. p ∈ [0,P]

Wp = h(ω)−Tx ↗


2‖π‖2 in the


then

πβ (x ,ω) :=

1β

(Tx + Wpβ −h(ω)

)I xβ now solves

{min 〈c,x〉+E

[Qβ (x ,ω)

]s.t. Ax = b ,x ≥ 0

Stabilizing dual variablesTo mitigate wild oscillations, for βk → 0 consider

Qβk (x ,ω) :=

{min C(p) + 1

2βk‖Tx + Wp−h(ω)‖2

s.t. p ∈ [0,P]

I corresponds to a quadratic term βk2 ‖π‖

2 in thedual

I if pk = pβk (x ,ω) solves the regularized problem,then

πk(x ,ω) :=

1βk

(Tx + Wpk −h(ω)

)I xk now solves

{min 〈c,x〉+E

[Qβk (x ,ω)

]s.t. Ax = b ,x ≥ 0

Stabilizing dual variables: theoremUnder suitable CQ and 2nd order conditions,as βk → 0 ,

I the sequence {xk} is bounded,I each accumulation point x∞ solves the original

problemI the sequence of mean regularized prices{

πk :=

1S ∑

sπ

k(xk ,ωs)

}converges to the

multiplier π with minimum norm .

Stabilizing dual variablesAs βk decreases the variance Var

[πk(xk ,ω)

]increases

=⇒ a suitable value for βk must be found

Remarks (suite):



I uncertainty complicates calculations

I quadratic regularization in the dual stabilizes theoutcome


I regarding nonconvexity . . .

Solucoes Matematicas para Problemas Industriais

I Modelling schoolI WorkshopI 2018 topic:

Formacao de precosno Despacho Hidrotermico

de Curto PrazoI July 9 to 20th, 2018I Sao Carlos, SP

http://www.cemeai.icmc.usp.br/3WSMPI/

http://www.cemeai.icmc.usp.br/3WSMPI/

· Joseph-Louis Lagrange (1736-1813) source: Wikipedia W.W.R. Ball, History of Mathematics (3rd...

Documents

Transcript of · Joseph-Louis Lagrange (1736-1813) source: Wikipedia W.W.R. Ball, History of Mathematics (3rd...