Electric Vehicles in smart grids: a Hybrid Benders/EPSO Solver for Stochastic Reservoir Optimization

Vladimiro Miranda Hrvoje Keko Fellow IEEE Administration Board INESC TEC President INESC P&D Brasil vladimiro.miranda@inesctec.pt hrvoje.keko@inesctec.pt

1

EV-based Stochastic Storage Concept

- a group of vehicles connected to the grid comprises a stochastic storage

- all variables are time dependent and stochastic (incl. max and min energy storable in the reservoir)

- partial contributions from each electric vehicles as seen from the generation system form a cluster of vehicles = stochastic reservoir

2

EV-based Stochastic Storage Concept

The desired energy Ed: the difference between the final state of charge of an aggregated reservoir and the initial state The energy deficit is the difference between desired and actual SoC at the time of disconnection; analogous to ENS

3

ener

gy

Emax

Emin

time

Ea(t=tend)amount provided

start

Def{Ed : energy desired by this cluster

end

Ea(t=t0)

Time Decomposition of Charging: Network Flow Model

The constraints are formulated as flow constraints, decomposed per time period. Energy that the power system can supply to / draw from the reservoir is limited: flow limits (aggregated interface limitations, grid limits etc). After a group of vehicles is disconnected, the power cannot flow from the system to it (and vice versa) so there’s no connection (e.g. cluster 1 in t=3)

4

𝑬𝑬𝒓𝒓 𝒕𝒕, 𝒄𝒄 ≤ 𝑬𝑬𝒓𝒓 𝒕𝒕, 𝒄𝒄 ≤ 𝑬𝑬𝒓𝒓 𝒕𝒕, 𝒄𝒄 , ∀𝒄𝒄,∀𝒕𝒕 c = vehicle cluster

Er(c=2, t=3)

P(t=3)

cluster 1

P(t=0) P(t=1) P(t=2)

cluster 2

Er(c=1, t=0)

Er(c=1, t=2)

Transition to V2G: three EV Charging Modes

- “dumb” charging: - time of disconnection is equal to the end of charging, no hourly

arbitrage possible, inflexible demand (not flexible regarding power drawn from the grid)

𝑡𝑡𝒆𝒆𝒆𝒆𝒆𝒆 𝒄𝒄𝒄𝒄𝒄𝒄𝒄𝒄𝒕𝒕𝒆𝒆𝒓𝒓 = 𝑡𝑡𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 𝐸𝐸𝑐𝑐 𝑡𝑡, 𝑐𝑐 = 𝐸𝐸𝑐𝑐 𝑡𝑡, 𝑐𝑐 , ∀𝑡𝑡

- unlike additional load: the cost of failing to provide the desired amount of energy may be different from ENS!

- smart charging:

- permits hourly arbitrage – delayed charging - charging amount controllable - does not permit the energy flow from the vehicle to the grid

𝐸𝐸𝑐𝑐 𝑡𝑡, 𝑐𝑐 ≤ 𝐸𝐸𝑑𝑑 𝑐𝑐 − � 𝐸𝐸𝑐𝑐(𝑡𝑡, 𝑐𝑐)𝑡𝑡=𝑡𝑡−1

𝑡𝑡=𝑡𝑡0 𝑐𝑐

, ∀𝑡𝑡𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 , 𝑡𝑡0 𝑐𝑐 ≤ 𝑡𝑡𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 ≤ 𝑡𝑡𝑓𝑓(𝑐𝑐)

𝐸𝐸𝑐𝑐 𝑡𝑡, 𝑐𝑐 ≥ 0, ∀𝑡𝑡 𝐸𝐸𝑐𝑐 𝑡𝑡, 𝑐𝑐 ≤ 𝐸𝐸𝑐𝑐 𝑡𝑡, 𝑐𝑐

5

Transition to V2G: Three EV Charging Modes

- vehicle to grid interface charging: - the real stochastic reservoir: EV pool is a stochastic storage - energy can flow in both directions - the equations link the time periods

𝐸𝐸𝑐𝑐 𝑡𝑡, 𝑐𝑐 ≤ 𝐸𝐸𝑐𝑐 𝑡𝑡, 𝑐𝑐 ≤ 𝐸𝐸𝑐𝑐 𝑡𝑡, 𝑐𝑐

𝐸𝐸𝑐𝑐 𝑡𝑡, 𝑐𝑐 < 0

𝐸𝐸𝑐𝑐 𝑡𝑡, 𝑐𝑐 ≤ 𝐸𝐸𝑑𝑑 𝑐𝑐 − � 𝐸𝐸𝑐𝑐(𝑡𝑡, 𝑐𝑐)𝑡𝑡=𝑡𝑡−1

𝑡𝑡=𝑡𝑡0 𝑐𝑐

, ∀𝑡𝑡𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐, 𝑡𝑡0 𝑐𝑐 ≤ 𝑡𝑡𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 ≤ 𝑡𝑡𝑓𝑓(𝑐𝑐)

𝐸𝐸𝑑𝑑 𝑐𝑐 − � 𝐸𝐸𝑐𝑐 𝑡𝑡, 𝑐𝑐 ≥ 𝐸𝐸𝑚𝑚𝑐𝑐𝑐𝑐 𝑡𝑡, 𝑐𝑐𝑡𝑡=𝑡𝑡−1

𝑡𝑡=𝑡𝑡0 𝑐𝑐

, ∀𝑡𝑡

- the model permits modeling gradual implementation of V2G connections

6

The Energy Deficit (in EV reservoirs)

- The unit commitment cost function becomes: Minimize ∑ 𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂 𝐏𝐏 𝐂𝐂 +𝐂𝐂 ∑ 𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂 𝐃𝐃𝐃𝐃𝐃𝐃 𝐜𝐜𝐜𝐜 subject to system restrictions

- i.e. minimize the system operational cost plus the energy deficit “cost”: the

price of not completely providing adequate charging to the EV owners

- the inclusion of the deficit makes the problem always mathematically feasible, favorable for cut generation and algorithm speed (only optimality cuts are generated)

- different from the cost of energy not being supplied (ENS) - aggregators may have a diversity of contractual obligations with the

EV owners - this is the cost of customer dissatisfaction

7

Benders and Dual Dynamic Programming: basics … a set of cascading “independent” problems:

STRUCTURE OF THE PROBLEM

8

Min

Subj

t tnt

n

n n n n

C X D Y D YAX bJ X K Y g

J X K Y g

+ + +≥

+ ≥

+ ≥

1 1

1 1 1 1

...

: ... ... ...

t t t1 2 n n

1

n n n

Min ...

Subj : ... ...

...

+ + +≥

+ ≥≥

+ ≥

1 2

1 1 1

1 2 2 2

2 2

C X C X C XA X bE X A X b

E XA X b

t t t1 2 n n

1

n n n n 1 n 1

Min ...

Subj : ... ...

− −

+ + +≥

+ ≥ −≥

+ ≥ −

1 2

1 1 1

2 2 2 1

C X C X C XA X b

A X b E X

A X b E X

BENDERS master and slaves

9

MASTER (integer)

Slave 1 Slave n (LP)

Slave 1 (LP)

Slave 2 (LP) …

Stochastic extension

Admit a problem with two time stages and with two scenarios for the second stage. The structure of the problem may be where p1, p2 are the probabilities of each scenario

1 2

21

22

Time stages

(scenarios)

t t t1 1 2 1 2 2 22

21 1

1 2 22 22

Min p p

Subj :

+ +≥

+ ≥+ ≥

1 2

1 1 1

2 1 2 2

2

C X C X C XA X bE X A X bE X A X b

The Benders trick

Primal and dual forms of the sub-problem:

The domain of the dual does not depend on Xn-1 !!! The optimum of the dual may be found among the vertices of a fixed domain (if Xn-1 changes, the optimum may change vertex, but the feasible region is constant)

11

( ) tn 1 n n

n n n 1 n 1

n

min

Subj :

−

− −

α =

≥ −≥

X C XA X b E XX 0

( )tn 1 n 1tn n

Max

Subj.:

− −α = − π

π ≤π ≥

b E X

A C0

Solving the dual

( )

0DK

JXg

≥π≤π

π−=α

:Subj.

Max t

t

Searching for vertices

Benders

Two different slopes for the objective function, from different values of X

( )( )

( ) nt

iit

it

...

min

π−≥α

π−≥α

π−≥α

α

JXg

JXg

JXg

For a given X*, each vertex defined by the dual variables π will have an α value - and we wish to select the vertex π* with maximum α

Adding a constraint to the master

Solving the sub-problem for a given X*, we find a vertex of the dual, which corresponds to a valid constraint that can be added to the master: If we solve now the master, we get a new value for X* – which will allow finding another vertex π in the sub-problem dual – which will allow a new constraint to be added to the master – which... BUT Adding a constraint to the problem may be replaced by adding a penalty to the Master objective function…

THE MASTER PROBLEM MAY BE SOLVED BY A META-HEURISTIC WITH EVOLVING LANDSCAPE!

Benders

( )

Min Subj

t

t

C XAX b

g JX

+≥

≥ −

α

α

:

*

π

Stochastic modeling: scenarios

- In systems with storage, the decisions in a time step are reflected on other time steps - “snapshot” analysis - considering each time period as independent from others – is

not adequate - sequence of marginal distributions also inappropriate! - missing temporal evolution of variables

- The chosen model of uncertainty: scenarios

- Sampled from an estimator model

- for wind power: covariance matrix estimation

- for EV behavior: Gaussian copula-based Monte Carlo model or extracting data from agent-based model simulations of traffic behavior

- Result: a large set of sampled scenarios -> requires clustering to reduce the number of scenarios!

14

Sequential Monte Carlo generating patterns

Representing distinct behavior models:

15

0

20

40

60

80

100

120

Monday Tuesday Wednesday Thursday Friday Saturday Sunday

y

(

)

— Methodic citizens (charging the EV at the end of the day only) — Obsessed citizens (charging the EV whenever possible) — Relaxed citizens (charging the EV only when the battery is empty)

SOC (%) of the EV battery

Stochastic modeling: scenario reduction

16

- From a large set of Monte Carlo sampled scenarios, a clustering process delivers a set of weighted scenarios according to a similarity metric

- Clustering problem: maximize entropy among clusters, minimize entropy within each cluster, assign relative weights

- In the cases tested, the distance metric used was the absolute per-hour deviation

Unit Commitment with Renewables and Stochastic Storages

17

The day-ahead stochastic UC problem decomposed into three stages:

binary decisions min up and down time constraints startup and shutdown

Stage 0 Master problem Stage 1 Stage 2

continuous variables ramping constraints decomposed hydro constraints (max energy per day) max inclusion of renewable power min customer dissatisfaction

(Classic) UC Problem Formulation - Quadratic fuel cost functions of thermal units, piecewise linearized per interval

𝑪𝑪𝒈𝒈(𝒕𝒕) = 𝒂𝒂 ∙ 𝒄𝒄𝒄𝒄𝒈𝒈 𝒕𝒕 + 𝒃𝒃𝑷𝑷𝒈𝒈 𝒕𝒕 + 𝒄𝒄𝑷𝑷𝒈𝒈 𝒕𝒕 2 - Start-up and shut down costs

𝑺𝑺𝑼𝑼𝒈𝒈 𝒕𝒕 = 𝒄𝒄𝒄𝒄𝒈𝒈 𝒕𝒕 − 𝒄𝒄𝒄𝒄𝒈𝒈 𝒕𝒕 − 𝟏𝟏 𝑺𝑺𝑼𝑼𝒈𝒈

- Hydro generators with large storage: total energy produced during the day is constrained (from long term optimization governing classic storage operation)

𝑬𝑬𝒈𝒈,𝒎𝒎𝒎𝒎𝒆𝒆 ≤�𝑷𝑷𝒈𝒈 𝒕𝒕 ≤ 𝑬𝑬𝒈𝒈,𝒎𝒎𝒂𝒂𝒎𝒎𝒕𝒕

- Min up time and min down time constraints + unit initial conditions: s=1 if unit changes state, 0 otherwise

� 𝒄𝒄𝒈𝒈,𝒐𝒐𝒆𝒆 𝒎𝒎 ≤ 𝒄𝒄𝒄𝒄𝒈𝒈(𝒕𝒕)𝒕𝒕−𝒕𝒕𝒈𝒈,𝒐𝒐𝒆𝒆≤𝒎𝒎≤𝒕𝒕

, � 𝒄𝒄𝒈𝒈,𝒐𝒐𝒐𝒐𝒐𝒐 𝒎𝒎 ≤ 𝟏𝟏 − 𝒄𝒄𝒄𝒄𝒈𝒈(𝒕𝒕)𝒕𝒕−𝒕𝒕𝒈𝒈,𝒐𝒐𝒐𝒐𝒐𝒐≤𝒎𝒎≤𝒕𝒕

- Generation limits

𝑷𝑷𝒈𝒈,𝒎𝒎𝒎𝒎𝒆𝒆 ≤ 𝑷𝑷𝒈𝒈 𝒕𝒕 ≤ 𝑷𝑷𝒈𝒈,𝒎𝒎𝒂𝒂𝒎𝒎

- Ramping limits −𝑹𝑹𝒈𝒈,𝒆𝒆𝒐𝒐𝒅𝒅𝒆𝒆 ≤ 𝑷𝑷𝒈𝒈 𝒕𝒕 − 𝑷𝑷𝒈𝒈 𝒕𝒕 − 𝟏𝟏 ≤ 𝑹𝑹𝒈𝒈,𝒄𝒄𝒖𝒖

18

EPSO and the generalized version DEEPSO

The Master problem on integer variables) is solved by a customized version of EPSO algorithm

DEEPSO concept: o inertia: moving in the same direction o perception: sensing a local gradient (by the swarm) o cooperation: attraction to the proximity of the global best * subject to mutation P – communication probability

new * * * *I M r1 C Gw w ( ) w ( )= + − + −V V X X P b X

= +new newX X V

inertia

(memory)

cooperation

Xold

Xnew

X

bi

bG*

Xr1

bG

perception

EVOLVING SWARMS – EPSO AND DEEPSO

EPSO – the gradient perception is based on the particle self-memory term DEEPSO – a flavor of Differential Evolution added to EPSO Variants : sampled among the current generation : Sg sampled among the matrix bi of individual past bests : Pb as a uniform recombination of the current generation : Sg-rnd as a uniform recombination within the matrix bi : Pb-rnd for the latter 2: not taking in account the direction of : .- minus taking in acc. the direction of : .+ plus

taking in acc. the direction of in each coordinate: .0 zero

DEEPSO: winner of the 2014 IEEE competition of m-h for the OPF problem

20

new * * * *I M i C Gw w ( ) w ( )= + − + −V V b X P b X

new * * * *I M r1 C Gw w ( ) w ( )= + − + −V V X X P b X

r1X

r1( )−X Xr1( )−X Xr1( )−X X

Custom EPSO for UC (Stage 0)

- Initialization of unit commitment status: - Custom tailored heuristic, not general but with more insight into the

problem - Heuristic order-based rule to commit units and construct initial population

of solutions - Commit enough units until sum(p) is enough to cover the max load

- Checks and repairs for violating of on/off restrictions (commiting and

decommiting when violated)

- Each particle maps the {0,1} space of unit commitment decisions to fitness value EACH OF THE SCENARIOS IN STAGES 1 and 2 ESSENTIALY REPRESENTS AN

ADDITIONAL PENALTY TO THE FITNESS LANDSCAPE!

21

Linearized subproblems in stages 1 and 2

- The subproblems involving wind power and storage are formulated as LP problems

- Solved using a industry standard LP solver to optimality – the dual is easily obtained

- The linear formulations in stage 1 include PNS and in stage 2 include Energy Deficit:

- There always exists a mathematically feasible solution - Faster to solve to (mathematical) feasibility even if technical

feasibility is not obtained - The cuts generated by the subproblems: they are optimality cuts

and not feasibility, so no distortion in the EPSO search space

22

Scenarios as penalties to the fitness function: risk handling

In the expected value formulation, the fitness function is

𝑪𝑪𝒐𝒐𝒄𝒄𝒕𝒕 = 𝑬𝑬 𝑪𝑪𝒐𝒐𝒄𝒄𝒕𝒕 =𝟏𝟏𝑵𝑵𝒄𝒄

�𝑪𝑪𝒐𝒐𝒄𝒄𝒕𝒕𝒄𝒄𝒄𝒄∈𝑺𝑺

For a robust optimization (worst case), the worst case defines the cost

𝑪𝑪𝒐𝒐𝒄𝒄𝒕𝒕 = 𝒎𝒎𝒂𝒂𝒎𝒎𝒄𝒄∈𝑺𝑺𝑪𝑪𝒐𝒐𝒄𝒄𝒕𝒕𝒄𝒄 For a minimax regret formulation, the cost is the maximum cost deviation from what would be the cost with perfect foresight (if the particular scenario occurred exactly)

𝑪𝑪𝒐𝒐𝒄𝒄𝒕𝒕 = 𝒎𝒎𝒂𝒂𝒎𝒎𝒄𝒄∈𝑺𝑺 𝑪𝑪𝒐𝒐𝒄𝒄𝒕𝒕𝒄𝒄 − 𝑪𝑪𝒖𝒖𝒐𝒐(𝒄𝒄)

23

Scenarios as penalties to the fitness function

An adaptive scheme handles the constraint contributions to the fitness function

𝒐𝒐 = � 𝝀𝝀𝒄𝒄𝒖𝒖𝒄𝒄𝑪𝑪𝒐𝒐𝒄𝒄𝒕𝒕𝒄𝒄𝒄𝒄

• Weight factors for each scenario cost are adapted according to the risk

model and the probability of a certain scenario • Remember: the scenarios have a probability as a result of clustering

• Exponential decay is used to „forget” scenarios that do not discriminate

between the current population of solutions 𝝀𝝀𝒄𝒄,𝒎𝒎+𝟏𝟏 = 𝝁𝝁 𝝀𝝀𝒄𝒄,𝒎𝒎;𝝁𝝁 < 𝟏𝟏

• In the initial phase of the algorithm a limited subset of scenarios is used to

speed up convergence and then resampling is used

24

Finishing an iteration of the algorithm

• In the classic EPSO, the location information is shared between individuals; here, the solutions also share scenario values

• Optimal cost values for stage 2 and reduced cost coming from the equality constraint are shared!

• This way the stage 2 solutions can be approximately calculated using the Benders decomposition cost-to-go principle!

• Cut calculation includes historical values: no recalculation if not

necessary

• After each iteration, binary solutions of stage 0 are repaired if there are violations in the startup and shutdown time constraints

25

Results: comparison with a MILP solver (GUROBI)

- classic formulation with 1000 wind power scenarios and 5 units

- can be solved both with MILP solver directly - the same solver used within EPSO for LP problems used here for

the whole problem - stable and consistent results of EPSO with comparable performance

- approx 20% loss in performance provides more flexibility

26

Average run times: 5 unit 1000 wind scenarios (no reduction), 100 repeated runs, expected value formulation: MILP UC: 121 seconds EPSO UC: 163 seconds (on a i7 3820qm laptop computer)

Results: 10 Unit illustration problem (1)

- 1000 wind power scenarios reduced to a set of 20 scenarios - 500 scenarios of EV integration reduced to a set of 20 scenarios - 10 unit thermal system, enough power but relatively constrained with

regard to flexibility (ramps) - no electric vehicles, wind power integration only:

27

Results: 10 Unit illustration problem (2)

- 33.000 vehicles with 16 kWh average desired energy, on avg 550 MWh of additional energy required throughout the day

- 80% of vehicles dumb charging, 20% smart charge, no V2G

28

smart chargers help avoiding wind spill but the resulting additional load not favorable… most covered by expensive generator 7

no PNS, wind not spilled, energy deficit at bay …

1.8% additional energy -> 3.1% increase in system cost!

Results: 10 Unit illustration problem + strong V2G (3)

- 20% of vehicles have smart charging and 80% V2G, no dumb charging

- system cost reduced due to more charging in cheaper valley hours - some extra demand still occurs in peak hours; 2.0% increase in overall cost

compared to NO EV case - 6.81% lower overall cost compared to dumb charging case - 35% less

increase in system costs

29

the most expensive generator works far less

additional load distributed more favorably: 1.8% additional load -> 2.0% increase in overall cost!

CONCLUSIONS

A Benders decomposition in the form of {Dual LP Slave problems + EPSO solver for the Master integer Problem}

is successful! • A stochastic model with high number of scenarios (wind + EV use) • Allows the modeling of different strategies of EV integration • Allows for several risk managing strategies • Master problem optimized under an evolving landscape (Benders

cuts transformed into penalties)

Competitive performance on problems solvable with MILP – allows the reasonable expectation of superior performance in high dimension

problems, because of the Benders decomposition principle

The algorithm is also promising for planning purposes (simulated pricing behavior) and for the TSOs and aggregators.

30