Handling Uncertainty in Sector Coupled Systems using ......2019/09/27 · System Identification...

Handling Uncertainty in Sector Coupled Systems using

Dynamic Programming and Model Predictive Control

Smart Energy System International Conference, Copenhagen 2019

Frederik Banis

Technical University of Denmark

Outline

• Model Predictive Control example: Microgrid frequency stabilization• Overview• Temporal Control Hierarchy

• MPC with Active Learning under Model Uncertainty• Model Uncertainties• Bayesian recursion• Dynamic Programming

• Indirect Control with Active Learning• Augmented Objective• Temporal Control Hierarchy

• EMPC with Active Learning• Objective• Temporal Control Hierarchy

• Considered modeling tools• System Identification and uncertainty estimation

2 DTU Compute Handling Uncertainty in Sector Coupled Systems using Dynamic Programmingand Model Predictive Control

11.9.2019

Model Predictive Control example: Microgrid frequency stabilizationSystem model

Classical State-Space system:

xt+1 = Axt +But +Gdt + w (1)yt = Cxt + v (2)

Main state: Swing equation

d

dt∆f(t) = − D

2H∆f(t) +

1

2H∆Pmech(t) (3)


11.9.2019

Model Predictive Control example: Microgrid frequency stabilizationSimulation: High available ramping flexibility

0.02

0.00

f /H

z

YhatYm

100 200 300 400 500 600 700Timesteps

0.0

0.1

0.2

0.3

0.4

P / p

.u.

U0U1U2Ubar0Ubar1Ubar2


11.9.2019

Model Predictive Control example: Microgrid frequency stabilizationSimulation: Lower available ramping flexibility

0.05

0.00

0.05f /

Hz

YhatYm

100 200 300 400 500 600 700Timesteps

0.0

0.1

0.2

0.3

0.4

P / p

.u.

U0U1U2Ubar0Ubar1Ubar2


11.9.2019

Model Predictive Control example: Microgrid frequency stabilizationTemporal Control Hierarchy with Indirect Control

EMSStochastic Program

Aggregator

1h

RDReal–time re–dispatch

1-15 min

Direct ControlFast dynamics

1-5s

Indirect ControlUncertain price–responsiveness

60-600s

Basic control Prosumer

Microgrid Controller


11.9.2019

Model Predictive Control example: Microgrid frequency stabilizationExemplary system responses

0 20 40 60 80 100 120Timesteps / s

0.6

0.5

0.4

0.3

0.2

0.1

0.0

P / p

u

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

q

Figure: Small uncertainty responsecluster

0 20 40 60 80 100 120Timesteps / s

0.6

0.5

0.4

0.3

0.2

0.1

0.0

P / p

u

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

q

Figure: Large uncertainty responsecluster


11.9.2019

Model Predictive Control example: Microgrid frequency stabilizationSimulation: Freq. stab. with uncertain consumption I

0.1

0.0

0.1

f

DCDC+IC

0.0

0.2

0.4

P u

200 225 250 275 300 325 350 375 400Timesteps / s

0.25

0.00

P d


11.9.2019

Model Predictive Control example: Microgrid frequency stabilizationDC+IC

Actor (Deterministic) + Prosumer response (Uncertain)

Uncertainty should be compensated for post–realization and pro–realization


11.9.2019

Model Predictive Control example: Microgrid frequency stabilizationDC+IC

Actor (Deterministic) + Prosumer response (Uncertain)Uncertainty should be compensated for post–realization and pro–realization


11.9.2019

Model Predictive Control example: Microgrid frequency stabilizationIC control objective

min{pk+j}N−1j=0

ΦIC =1

2

N−1∑j=0

||Ψ||2Q + ||∆pk+j ||2R (4)

s.t. x̂k+1|k = Ax̂k|k +Bpk (5)

x̂k+1+j|k = Ax̂k+j|k +Bpk+j (6)

j = 1, 2, . . . ,N− 1ŷk+j|k = Cx̂k+j|k j = 1, 2 . . . , N (7)

pmin ≤ pk+j ≤ pmax (8)


11.9.2019

MPC with Active Learning under Model UncertaintyLinear/Non–Linear system

Exemplary formulation

Mj :

{xk+1 = f(xk, uk, θ, wk)

yk = h(xk, θk, vk)(9)

Source of formulation: Heirung et al. 2018

Sources of uncertainties

• Structural uncertainty

• Parametric uncertainty


11.9.2019

MPC with Active Learning under Model UncertaintyDP: General cost function

Jk(ζk, πk) = E[

N−1∑j=k

lj(xj , uj) + lN (xN )] (11)


11.9.2019

MPC with Active Learning under Model UncertaintyBellman equation: Recursive problem solution

J?k (ζk) = minukEk[lk(xk, uk) + J

?k+1(ζk+1)], k = 0, 1, . . . , N − 1 (12)


11.9.2019

Indirect Control with Active LearningIC control objective + Active Learning

min{pk+j}N−1j=0

ΦIC =1

2

N−1∑j=0

||Ψ||2Q + ||∆pk+j ||2R (13)

s.t. x̂k+1|k = f(xk, uk, θk) (14)

x̂k+1+j|k = f(xk+j , uk+j , θk+j) (15)

j = 1, 2, . . . ,N− 1pmin ≤ pk+j ≤ pmax (16)


11.9.2019

Indirect Control with Active LearningTemporal Control Hierarchy


Aggregator

RDReal–time re–dispatch


Indirect Control + ActiveLearning

Uncertain price–responsiveness



But: Scope on fast systems


11.9.2019

EMPC with Active LearningEconomic objective function

min{pk+j}N−1j=0

ΦEMPC =∑

k inNpkuk + αvvk (17)

s.t. x̂k+1|k = f(xk, uk, θk) (18)

x̂k+1+j|k = f(xk+j , uk+j , θk+j) (19)

j = 1, 2, . . . ,N− 1pmin ≤ pk+j ≤ pmax (20)


11.9.2019

EMPC with Active LearningTemporal Control Hierarchy: Focus on slow prosumers


Aggregator

1h

IC EMPC + Active Learning

1-15 min


1-5s

Basic control Slow Prosumers



11.9.2019

EMPC with Active LearningTemporal Control Hierarchy: Generalized prosumers


Aggregator

IC EMPC + Active Learning


Indirect Control + ActiveLearning

Uncertain price–responsiveness




11.9.2019

Considered modeling toolsSystem Identification and uncertainty estimation

• Markov Chain Monte Carlo sampling

• see e.g. Stan

• Classical subspace identification techniques

• see e.g. Van Overschee and de Moor 1993

• Dynamic Mode Decomposition

• see e.g. Schmid 2010; Kutz, Fu, and Brunton 2015


11.9.2019

Closing notesClosing notes I

Content:

• Starting point: MPC for aggregated Microgrid operation (Virtual Power Plant)

• Background: MPC with dual effect (Active Learning)

• Goal: Economic MPC considering the dual effect for slow sector coupling


11.9.2019

Closing notesClosing notes II

Thank you for your attention.

Figure: This work has been supported by ENERGINET.DK under the projectmicroGRId positioning (uGrip) and the CITIES project.


11.9.2019

F. Banis et al. “Load Frequency Control in Microgrids Using TargetAdjusted Model Predictive Control”. English. In: Not yet published(2019). ISSN: 1751-8644.

Frederik Banis et al. “Utilizing Flexibility in Microgrids Using ModelPredictive Control”. In: MedPower 2018. Croatia, Nov. 2018.

Bob Carpenter et al. Stan: A Probabilistic Programming Language.

Tor Aksel N. Heirung et al. “Model Predictive Control with ActiveLearning under Model Uncertainty: Why, When, and How”. en. In:AIChE Journal 64.8 (2018), pp. 3071–3081. ISSN: 1547-5905. DOI:10.1002/aic.16180.

J. Nathan Kutz, Xing Fu, and Steven L. Brunton. “Multi-ResolutionDynamic Mode Decomposition”. en. In: arXiv:1506.00564 [math](June 2015). arXiv: 1506.00564 [math].


11.9.2019

http://dx.doi.org/10.1002/aic.16180http://arxiv.org/abs/1506.00564

Gabriele Pannocchia and James B. Rawlings. “Disturbance Models forOffset-Free Model-Predictive Control”. In: AIChE journal 49.2 (2003),pp. 426–437.

Gabriele Pannocchia and James B. Rawlings. “Robustness of MPC andDisturbance Models for Multivariable Ill-Conditioned Processes”. In:TWMCC, Texas-Wisconsin Modeling and Control Consortium (2001).

Peter J. Schmid. “Dynamic Mode Decomposition of Numerical andExperimental Data”. en. In: Journal of Fluid Mechanics 656 (Aug.2010), pp. 5–28. ISSN: 1469-7645, 0022-1120. DOI:10.1017/S0022112010001217.


11.9.2019

http://dx.doi.org/10.1017/S0022112010001217

P. Van Overschee and B. de Moor. “N4SID: Numerical Algorithms forState Space Subspace System Identification”. In: IFAC ProceedingsVolumes. 12th Triennal Wold Congress of the International Federationof Automatic Control. Volume 5 Associated Technologies and RecentDevelopments, Sydney, Australia, 18-23 July 26.2, Part 5 (July 1993),pp. 55–58. ISSN: 1474-6670. DOI:10.1016/S1474-6670(17)48221-8.


11.9.2019

http://dx.doi.org/10.1016/S1474-6670(17)48221-8

AppendixExample

Figure: Solar heat injection station (small scale): Excess heat from solarthermalcollectors can be injected in the district heating network.


11.9.2019

AppendixReference paper on this regulator formulation

See as reference paper for all aspects on this matter shown below: Baniset al. 2019; Banis et al. 2018. This publication is based on approachesoutlined in Pannocchia and Rawlings 2003.


11.9.2019

AppendixSystem model

Classical State-Space system:

xt+1 = Axt +But +Gdt + w (21)yt = Cxt + v (22)

Main state: Swing equation

d

dt∆f(t) = − D

2H∆f(t) +

1

2H∆Pmech(t) (23)


11.9.2019

AppendixObjective function

Stabilization problem T1

J∞,k = ||Φx(x̂k − x∞,k) + Γu(uk − u∞,k)||2 (24)

Dynamic Programming problem T2

JDO,k = ||uk − u?k−1 + γW∆u∆uk||2 (25)

Portfolio constitution T3

JC,k = (1− γ)Πk (26)


11.9.2019

AppendixObjective function: Overview

minu,k

J∞,k + JDO,k + JC,k (27)

s.t. Gkuk ≤ hk (28)


11.9.2019

Appendix(T1) Residual estimation

Inferring input disturbance1

[x̂k+1|kd̂k+1|k

]=

[A Bd0 I

] [x̂k|k−1d̂k|k−1

]+

[B0

]uk+[

L1L2

](ym,k − Cx̂k|k−1 − Cdd̂k|k−1) (29)

1We optimize over deviations encompassing the positive and negative domain → Only first optimal input requiredsatisfactory, imposing these constraints for the whole sequence uk+N−1|k results in numerical issues.31 DTU Compute Handling Uncertainty in Sector Coupled Systems using Dynamic Programming

and Model Predictive Control11.9.2019

Appendix(T1) Stabilizing gain

Solving for g∞2 using least-squares approximation:

M︷︸︸︷[A− I BC 0

] g∞︷︸︸︷[gx,∞gu,∞

]=

[Bd0

](30)

g∞ ≈[Bd0

]M−1 (31)

2See Pannocchia and Rawlings 2003; Pannocchia and Rawlings 200132 DTU Compute Handling Uncertainty in Sector Coupled Systems using Dynamic Programming

and Model Predictive Control11.9.2019

Appendix(T1) Equilibrium point

[x∞u∞

]= g∞d̂ (32)


11.9.2019

Appendix(T2) Dynamic programming terms

Ensure offset-free control→ Even when constraints are active on parts of the portfolio


11.9.2019

Appendix(T3) Portfolio constitution

Πk = α||uk − uEMS,k||2W∆u+β( ||c̃kuk||2 + ||c̃∆,k(uk − uEMS,k)||2W∆u ) (33)

where: α+ β = 1


11.9.2019

AppendixConstraints

GeneralDynamic reformulation via supervisory system: considering additional systemknowledge

Gkuk ≤ hk (34)

Particularity: Ramp rate

Only the first optimal input in the sequence required binding1

∆umin ≤ u?k+1|k − u?k|k ≤ ∆umax (35)


11.9.2019

Model Predictive Control example: Microgrid frequency stabilizationOverviewTemporal Control Hierarchy

MPC with Active Learning under Model UncertaintyModel UncertaintiesBayesian recursionDynamic Programming

Indirect Control with Active LearningAugmented ObjectiveTemporal Control Hierarchy

EMPC with Active LearningObjectiveTemporal Control Hierarchy

Considered modeling toolsSystem Identification and uncertainty estimation

Handling Uncertainty in Sector Coupled Systems using ......2019/09/27 · System Identification...

Documents

Transcript of Handling Uncertainty in Sector Coupled Systems using ......2019/09/27 · System Identification...