A Tutorial on Blockchain for Microgrids: Securing Distributed Optimization

Prof. Scott Moura

Energy, Controls, & Applications Laboratory (eCAL)University of California, Berkeley

EricMunsing

2

Challenges: Intermittency, Variability, Quantity of DERsActors: Consumers, Aggregators, Utilities, System Operators

Motivation

3

Example: Renewable Energy Credits Issues:

SinceRECsarenotphysical,howdoweknowSaraactuallyhas5RECstosell?

HowdoweverifytheRECsarelegitimate?

WhotracksandmanagesRECtransactions?

4

Background Fields and Relevant Tools

Optimization & Control

Power Systems

Economics

• Mathematical Programming

• Optimization Theory

• Statistical Learning

• Optimal Power Flow

• Electricity Mkt Design• Game Theory

• Sequence and Planning

• Blockchains

• Strategic Bidding

• Distributed Optimization

• Regulation

• Economic Dispatch

Distributed v. Decentralized

6

Network Topologies

System Operator

Aggregator

Centralized

Distributed

Fully Decentralized

Centralized: Pros: Simple formulationCons: Limited privacy, slow computations

Distributed: Pros: Fast computation, protect privacyCons: High communication overhead

Pros: Low communication overhead, islanding Cons: Increased vulnerability to cyberattack

Fully-Decentral:

7

Distributed v. Decentralized

Weaknesses of aggregator-based consensus:• Monopoly distortions and trust issues • Single point of failure/attack• Limited scalability

Alternative: Decentralized optimization

• Consensus with neighbors who share variables• Effective for sparsely connected problems

Mota et al: “D-ADMM: A Communication-efficient distributed algorithm for separable optimization” IEEE Transactions on Signal Processing (2012)

8

Do you trust your neighbors?

Challenges:• Identify compromised nodes• Reach stable operation despite

compromised nodes

Assumptions: Compromised nodes…• Can affect problem data/constraints• Can send spurious messages

? ? ?

?

?

? ?

?

?

✓ ✓ ✓

☠

☠

✓ ✓

✓

☠

9

Options for Securing Fully-Decentralized Optimization

✓ ☠ ✓

Create database of state variables• Securely timestamp variables • Requires trusted database manager

10

Blockchain-Secured Decentralized Optimization

Blockchain as decentralized record-of-trust:• Record variable updates• Run feasibility checks• Compute update step• Anything a centralized aggregator can do!

…And more:• Transparent & auditable• Fully decentralized: robust to dropouts

✓ ☠ ✓

✓

✓ ☠ ✓

✓

Abstraction:

11

New Technology: Blockchains

M1t

MNt

Block Bt Contents

M2t

Header

� Block No.

� Timestamp

� Validator ID

� Hash of Contents:

ϒ(B1 )

B1B0M1

0

Bt Bt+1ϒ(B0)

ϒ(B0)

ϒ(Bt−1)

ϒ(Bt+1)ϒ(Bt )

ϒ(Bt )

ϒ(Bt−1)

ϒ(Bt )

Decentralized, transparent database• Consensus mechanism for state updates• Trustless• Guarantees of security + execution• Robust, scalable, stable

Open Research Areas• Coordinating decentralized devices• Decentralized optimization• Trusted sensors and data networks

Appendix

12

Why Blockchains?

Aggregators are great if:• One entity owns everything• No incentives to cheat• Communication is reliable• Aggregator isn’t attacked

Why Blockchains?• Benefits of decentralization• Strengths of central

auditors

Failure Mode Aggregator Fully-Decentralized

BlockchainADMM

Infeasible Constraints ✓ X ✓

Noise-injection attack ✓ ✓ ✓

Communication dropouts X ✓

✓

Monopoly distortions X ✓ ✓

13

Outline

Background: What problem does blockchain solve?

Security: Cybersecurity in decentralized and fully-decentralized optimization

Coordination: Model trustless DER coordination respecting distribution constraints

0

1

2

14

1 Decentralized DER Coordination

Q: How do we address trust issues in decentralized coordination networks?

Novelty• Use blockchains to secure distributed optimization

15

Blockchains and Decentralized Optimization

Appendix

Local Computation

Verification Step

Updated Blockchain

State

Local Computation, … ,

Auditable Consensus

Blockchains:

Examples: Bitcoin, Ethereum, …

Decentralized Optimization:

Local Optimization

Update Step

Solution

Local Optimization,…,

Consensus

Examples: ADMM, Dual Splitting, …

Local Problem

Verifiable Update Step

Solution Stored on Blockchain

Local Problem, … ,

Auditable Consensus

Combined Paradigm:

Novel Paradigm

Munsing, Mather, Moura: “Blockchains for Decentralized Optimization of Energy Resources in Microgrid Networks” CCTA 2017

16

Application: Coordinating Distributed Energy Resources

Local Problem

Verifiable Update Step

Solution Stored on Blockchain

Local Problem, … ,

Auditable Consensus…

…

ADMM Update (on Blockchain)

Consensus: Day-ahead Schedule

Local ProblemLocal Problem

…

Munsing, Mather, Moura: “Blockchains for Decentralized Optimization of Energy Resources in Microgrid Networks” CCTA 2017

• Private problem: Minimize private costs

• Global variables: Voltage & power estimates

• Update step (blockchain): ADMM average

• Converge when agree on network state

17

Distribution Networks and Convex Relaxations of OPF

Single-phase AC distribution flow equations:

li �P 2i +Q2

i

vi()

������

2Pi

2Qi

li � vi

������2

li + vi

SOCP Relaxation:

pi = Pi �X

k2�(i)

Pk + rili, i = 0, . . . , n

qi = Qi �X

k2�(i)

Qk + xili, i = 0, . . . , n

vi = v⇡(i) + 2(riPi + xiQi)� (r

2i + x

2i )li

li =P 2i +Q2

i

vi, i = 1, . . . , n

Evs & Shapeable Loads1:Psi,min

ss(t) Psi,max

TX

t=1

ss(t) = Esi,demand

ssi (t) = 0 8t = 1, . . . , ti,startby

ssi (t) = 0 8t = ti,endby, . . . ,T

Batteries1:sbi (t) = dbi (t)� cbi (t)

Pbi,dis dbi (t) 0 cbi (t) Pb

i,charge

Eb,min

Eb(t) Eb,max

Ebi (t) = Eb

i (t� 1) + cbi (t)�thi,in

� dbi (k)�t/hi,out

(1 + ")Ebi (1) Eb

i (T) (1� ")Ebi (1)

Deferrable Loads2:sd = �d

8t = 1 . . .T :

0 di(t� 1) di(t)

di(t) ai(t)

ai(t� ⇣) d(t)

di(t) 2 (0, 1)

2Le Floch et al: “Plug-and-Play Model Predictive Control for Load Shaping and Voltage Control in Smart Grids,” IEEE Transactions on Smart Grid, to appear3Chang, Alizadeh, Scaglione: “Coordinated Home Energy Management for Real-Time Power Balancing,” IEEE PES General Meeting, 2012

1M.E. Baran and F.F. Wul: “Optimal Capacitor Placements on Radial Distribution Systems”, IEEE Transactions on Power Delivery, 1989

18

Formulation

Decompose using ADMM:• Local variables: Local line flows and injections• Global variables: Network line flows and injections

Appendix

min

TX

t=1

nX

i=1

Ci,t(sgi (t))

s.t. power flow equations t = 1, . . . ,T

and DER constraints, i = 1, . . . , n

over sgi (t) 2 [s

gi , s

gi ], i = 0, . . . , n, t = 1, . . . ,T

(Pi, Qi, li, vi)(t), i = 1, . . . , n, t = 1, . . . ,T

Objective: Minimize Energy Cost

Y. Wang, L. Wu, and S. Wang, “A Fully-Decentralized Consensus-Based ADMM Approach for DC-OPF With Demand Response,” IEEE TSGI, 2016.Q. Peng and S. H. Low, “Distributed Algorithm for Optimal Power Flow on an Unbalanced Radial Network,” 53rd CDC, 2015.P. Sulc, S. Backhaus, and M. Chertkov, “Optimal Distributed Control of Reactive Power Via the Alternating Direction Method of Multipliers,” IEEE TEC, 2014.

19

Decentralize with ADMM and deploy on blockchain

repeatPi: Private Optimization, compute locally

Gather private constraints

Compute x̃i and send to smart contract S1

S1: ADMM Aggregator, on blockchainUpdate z

if krkk2 ✏pri, kskk2 ✏dual thenCompute final schedule and clearing prices

Send schedule to S2

enduntil krkk2 ✏pri, kskk2 ✏dual

Mi: Each Smart MeterRecord energy consumption

Send time-stamped & signed consumption to S2

...time progresses

S2: Billing contract, on blockchainCompare schedule from S1 with meter readings

Compute penalties, payments, and charges

Transfer payments between accounts

…

P1M1

P1M1

S1S2

20

Initial Formulation and Results

Approach:• Goal: Minimize day-ahead energy costs

• Constraint: Voltage at all buses on grid

• Common knowledge: Topology & line resistance

Bus Voltage Across Microgrid

0 4 8 12 16 20 24Time (h)

10

20

30

40

50

Bus

Num

ber

1.0465

1.047

1.0475

1.048

1.0485

1.049

1.0495

Vol

tage

0 5 10 15 20 25Time (h)

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

Rea

l Pow

er C

onsu

mpt

ion

(Gen

erat

ion)

, p.u

.

Average Nodal Power Consumption/Generation

BatteryUncontrolableSolarShapeableDeferable

Tools: • Shapeable loads (e.g. electric vehicles)• Deferrable Loads (e.g. smart appliances)• Batteries

21

Implementation

Implementation:• Smart contracts (S1, S2):

• Ethereum private blockchain• Access with ethjsonrpc

• Convex Optimization (P1): Cvxpy• Hardware: WeMo wifi lightbulbs

• Control with pywemo

22

Revisiting Open Questions

• ✓How can we respect distribution constraints when coordinating DERs?

• ✓How can we manage trust problems among DERs (and utilities)?

• X How can we guarantee security with large populations of DERs?

Appendix

23

2 Security in Distributed Optimization

Q: How do we scale to large networks without exposing our system to cybersecurity threats?

Novelty• Derive security checks that can be built into the

optimization algorithm

24

Do you trust your neighbors?

Challenges:• Identify compromised nodes• Reach stable operation despite

compromised nodes

Assumptions: Compromised nodes…• Can affect problem data/constraints• Can send spurious messages

? ? ?

?

?

? ?

?

?

✓ ✓ ✓

☠

☠

✓ ✓

✓

☠

25

Security in Distributed & Decentralized Optimization

Nefarious nodes may:• Distort private problem• Send unreachable update signal• Stop convergence by injecting noise

Prior work on security in optimization:• Failure-resistant scheduling• Impact of compromised sensors• Communication security• Greenfield: Design security checks into algorithm!

26

Background: Alternating Direction Method of Multipliersminx,z

f(x) + g(z)

s.t. Ax+Bz = c

x 2 Xz 2 Z

Convex Centralized problem:• Separable objective and constraints

x

k+1 := argminx2X f(x) +

⇢

2kAx+Bz

k � c+ u

kk22

z

k+1 := argminz2Z g(z) +

⇢

2kAx

k+1 +Bz � c+ u

kk22

u

k+1 := u

k +Ax

k+1 +Bz

k+1 � c

ADMM: Augmented local problem• Guaranteed convergence• Big hammer, popular tool

Boyd et al: “Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers,” Foundations and Trends in M.L. (2010)

x

k+1 := argminx2X f(x) +

⇢

2kAx+Bz

k � c+ u

k

x

k22

z

k+1 := argminz2Z g(z) +

⇢

2kAxk+1 +Bz � c+ u

k

z

k22

u

k+1x

:= u

k

x

+Ax

k+1 +Bz

k+1 � c

u

k+1z

:= u

k

z

+Ax

k+1 +Bz

k+1 � c

Fully Decentralized ADMM: • Local copies of penalty term• Same guarantees as ADMM!

Mota et al: “D-ADMM: A Communication-efficient distributed algorithm for separable optimization” IEEE Transactions on Signal Processing (2012)

27

Attack vector: Noise Injection

Stop convergence by injecting noise:•

• Defense neededx̃

k+1 = x

⇤k+1 + �(k)

28

Algorithm for Noise Detection

Given public information,

x1

�1.0�0.50.0

0.51.0x2

�1.0�0.8�0.6�0.4�0.2 0.0

f(x

1,x

2)

02468

Collect Iterates

1. Collect Iterates

x1

�1.0�0.50.0

0.51.0x2

�1.0�0.8�0.6�0.4�0.2 0.0

f(x

1,x

2)

02468

Assess Gradients

2. Assess Gradient

x1

�1.0�0.50.0

0.51.0x2

�1.0�0.8�0.6�0.4�0.2 0.0

f(x

1,x

2)

�2.50.02.55.07.5

Construct Hessian and Assess Convexity

3. Construct Hessian & Evaluate Convexity

Hypothesis: The local objective function is convex.If not, we conclude there was an attack

Eric Munsing and Scott Moura “Cybersecurity in Distributed and Fully-Decentralized Optimization: Distortions, Noise Injection, and ADMM” arXiv:1805.11194 [math.OC], 2018.

29

Assess Gradient

x

i = argminx2X f(x) +

⇢

2kAx+Bz

i�1 � c+ u

i�1k22

0 = @f(x)|i +A

T + ⇢A

T (Ax

i +Bz

i�1 � c)

@f(x)|i = �A

T (⇢ui � ⇢B(zi � z

i�1))

From ADMM convergence proof, assess gradient from public information:

x1

0.0 0.5 1.0 1.5 2.0x2

0.00.5

1.01.5

2.0012345

f(xi)

f(xj)

xi xj

�x1�x2

Changes in the gradient between two points is associated with the Hessian…… We can use this to assess convexity!

Eric Munsing and Scott Moura “Cybersecurity in Distributed and Fully-Decentralized Optimization: Distortions, Noise Injection, and ADMM” arXiv:1805.11194 [math.OC], 2018.

30

x1

0.0 0.5 1.0 1.5 2.0x2

0.00.5

1.01.5

2.0012345

f(xi)

f(xj)

xi xj

�x1�x2

From Gradient to Hessian

Changes in the gradient between two points is associated with the Hessian…… We can use this to assess convexity!

@

@x1f(x)|x

j

=@

@x1f(x)|x

i

+@

@

2x1

f(x)|xi

(xj

1 � x

i

1) +@

@x1@x2f(x)|x

i

(xj

2 � x

i

2)

@

@x2f(x)|x

j

=@

@x2f(x)|x

i

+@

@

2x2

f(x)|xi

(xj

2 � x

i

2) +@

@x2@x1f(x)|x

i

(xj

1 � x

i

1)

1st-order Taylor approximation between points xi and xj , 2-d case:

ITERATES ITERATESGRADIENT GRADIENT HESSIAN HESSIAN

31

2

666666664

@

@x1f(x)|kj

@

@x2f(x)|kj

@

@x1f(x)|ki

@

@x2f(x)|ki

3

777777775

=

2

664

(x1)|kj (x2)|kj 0 00 0 (x1)|kj (x2)|kj

(x1)|ki (x2)|ki 0 00 0 (x1)|ki (x2)|ki

3

775

2

666666664

@

@

2x1

f(x)

@

@x1@x2f(x)

@

@x2@x1f(x)

@

@

2x2

f(x)

3

777777775

For 2-d case:

D =

2

6664

In ⌦ (xk � x

a)T

In ⌦ (xk � x

b)T

...In ⌦ (xk � x

k�1)T

3

7775

For n-D:

Rearrange to�!G = D

�!H

From Gradient to Hessian

Eric Munsing and Scott Moura “Cybersecurity in Distributed and Fully-Decentralized Optimization: Distortions, Noise Injection, and ADMM” arXiv:1805.11194 [math.OC], 2018.

32

Numerical Tests: 10,000 Randomly Generated Quadratic Programs

RandomlyGenerate• Dimensionsofxandz• Positivesemi-definiteP,Q• A,B,c,d

ConsiderConvexQP

ADMMalgorithmis…

Randomlyinjectnoiseintox-update• 𝑥"#$ = Δ ⋅ argmin ⋯• Δ ∼ Ber 0.5 , Δ ∈ {0.9, 1.1}

Eric Munsing and Scott Moura “Cybersecurity in Distributed and Fully-Decentralized Optimization: Distortions, Noise Injection, and ADMM” arXiv:1805.11194 [math.OC], 2018.

33

Classification Results

Atta

ck

x1

�1.0�0.50.0

0.51.0x2

�1.0�0.8�0.6�0.4�0.2 0.0

f(x

1,x

2)

0

2

4

Convexity of Attacked Hessian

x1

�1.0�0.50.0

0.51.0x2

�1.0�0.8�0.6�0.4�0.2 0.0

f(x

1,x

2)

�2.50.02.55.07.5

Construct Hessian and Assess ConvexityNo Attack AttackNo

Atta

ck

x1

�1.0�0.50.0

0.51.0x2

�1.0�0.8�0.6�0.4�0.2 0.0

f(x

1,x

2)

�2.50.02.55.07.5

Convexity of Attacked Hessian

Repo

rted

Actual

x1

�1.0�0.50.0

0.51.0x2

�1.0�0.8�0.6�0.4�0.2 0.0

f(x

1,x

2)

0

5

10

15

Convexity of Attacked Hessian

4.2%

96%

0%

100%

Recap

35

Summary of Contributions

Optimization & Control

Power Systems

Economics

• Optimization Theory

• Statistical Learning

• Optimal Power Flow

• Electricity Mkt Design• Game Theory

• Sequence and Planning

• Blockchains

• Strategic Bidding

• Distributed Optimization

• Regulation

• Economic Dispatch

1

Coordination

Security

1

2

2

36

Thank You!

e.munsing@berkeley.edueric.munsing@gmail.com

(781)492-0614