Distributed Randomized Controlfor Ancillary Service to the Power Grid

With application to rational pools

LIDS Seminar May 6, 2014

Ana Busic and Sean Meyn

Prabir Barooah, Yue Chen, and Jordan Ehren

TREC & Departement d’Informatique

INRIA & ENSElectrical and Computer Engineering

University of Florida

Thanks to NSF, AFOSR, ANR, and DOE / TCIPG

Outline

1 Distributed Control of Loads

2 Design for Intelligent Loads

3 Linearized Dynamics and Passivity

4 One Million Pools

5 Conclusions and Extensions

6 References

0 / 28

Power GridControl Water PumpBatteries

CoalGas Turbine

BP

BP

BP C

BP

BP

Voltage Frequency Phase

HCΣ−

Actuator feedback loop

A

LOAD

Architecture for Distributed Control

Distributed Control of Loads

Power grid as a feedback control systemMassive disturbance rejection problem

One day in Gloucester:

1 / 28

Distributed Control of Loads

Power grid as a feedback control systemMassive disturbance rejection problem

Two typical weeks in the Pacific Northwest (BPA):

0

2

4

6

8

2

4

6

8

0

0.8

-0.8

1

-1

0

0.8

-0.8

1

-1

0

Sun Mon Tue

October 20-25 October 27 - November 1

Hydro

Wed Thur FriSun Mon Tue Wed Thur Fri

Gen

erat

ion

and

Laod

GW

GW

GW

Reg

ulat

ion

GW

Thermal

Wind

Load

Generation

Regulation

1 / 28

Distributed Control of Loads

Control of Deferrable LoadsControl Goals and Architecture

Power GridControl Water PumpBatteries

CoalGas Turbine

BP

BP

BP C

BP

BP

Voltage Frequency Phase

HCΣ−

Actuator feedback loop

A

LOAD

Context: Consider population of similar loads that are deferrable.Two examples of flexible energy hogs:

Chillers in HVAC systems and residential pool pumps

Control Constraints:

Grid operator demands reliable ancillary service;Consumers demand reliable service (and more)

2 / 28

Distributed Control of Loads

Control of Deferrable LoadsControl Goals and Architecture

Power GridControl Water PumpBatteries

CoalGas Turbine

BP

BP

BP C

BP

BP

Voltage Frequency Phase

HCΣ−

Actuator feedback loop

A

LOAD

Context: Consider population of similar loads that are deferrable.Two examples of flexible energy hogs:

Chillers in HVAC systems and residential pool pumps

Control Constraints:

Grid operator demands reliable ancillary service;Consumers demand reliable service (and more)

2 / 28

Distributed Control of Loads

One Million Pools in FloridaPools Service the Grid Today

On Call1: Utility controls residential pool pumps and other loads

Contract for services: no price signals involved

Used only in times of emergency — Activated only 3-4 times a year

Opportunity:FP&L already has their hand on the switch of nearly one million pools!

Surely pools can provide much more service to the grid

Summary (beyond pools):For many household loads, contracts with consumers are already in place

For other loads, such as plug-in electric vehicles (PEVs),consumer risk may be too great.

1Florida Power and Light, Florida’s largest utility.www.fpl.com/residential/energysaving/programs/oncall.shtml

3 / 28

Distributed Control of Loads

One Million Pools in FloridaPools Service the Grid Today

On Call1: Utility controls residential pool pumps and other loads

Contract for services: no price signals involved

Used only in times of emergency — Activated only 3-4 times a year

Opportunity:FP&L already has their hand on the switch of nearly one million pools!

Surely pools can provide much more service to the grid

Summary (beyond pools):For many household loads, contracts with consumers are already in place

For other loads, such as plug-in electric vehicles (PEVs),consumer risk may be too great.

1Florida Power and Light, Florida’s largest utility.www.fpl.com/residential/energysaving/programs/oncall.shtml

3 / 28

Distributed Control of Loads

One Million Pools in FloridaPools Service the Grid Today

On Call1: Utility controls residential pool pumps and other loads

Contract for services: no price signals involved

Used only in times of emergency — Activated only 3-4 times a year

Opportunity:FP&L already has their hand on the switch of nearly one million pools!

Surely pools can provide much more service to the grid

Summary (beyond pools):For many household loads, contracts with consumers are already in place

For other loads, such as plug-in electric vehicles (PEVs),consumer risk may be too great.

1Florida Power and Light, Florida’s largest utility.www.fpl.com/residential/energysaving/programs/oncall.shtml

3 / 28

Distributed Control of Loads

One Million Pools in FloridaPools Service the Grid Today

On Call1: Utility controls residential pool pumps and other loads

Contract for services: no price signals involved

Used only in times of emergency — Activated only 3-4 times a year

Opportunity:FP&L already has their hand on the switch of nearly one million pools!

Surely pools can provide much more service to the grid

Summary (beyond pools):For many household loads, contracts with consumers are already in place

For other loads, such as plug-in electric vehicles (PEVs),consumer risk may be too great.

1Florida Power and Light, Florida’s largest utility.www.fpl.com/residential/energysaving/programs/oncall.shtml

3 / 28

Distributed Control of Loads

One Million Pools in FloridaPools Service the Grid Today

On Call1: Utility controls residential pool pumps and other loads

Contract for services: no price signals involved

Used only in times of emergency — Activated only 3-4 times a year

Opportunity:FP&L already has their hand on the switch of nearly one million pools!

Surely pools can provide much more service to the grid

Summary (beyond pools):For many household loads, contracts with consumers are already in place

For other loads, such as plug-in electric vehicles (PEVs),consumer risk may be too great.

1Florida Power and Light, Florida’s largest utility.www.fpl.com/residential/energysaving/programs/oncall.shtml

3 / 28

Distributed Control of Loads

One Million Pools in FloridaPools Service the Grid Today

On Call1: Utility controls residential pool pumps and other loads

Contract for services: no price signals involved

Used only in times of emergency — Activated only 3-4 times a year

Opportunity:FP&L already has their hand on the switch of nearly one million pools!

Surely pools can provide much more service to the grid

Summary (beyond pools):For many household loads, contracts with consumers are already in place

For other loads, such as plug-in electric vehicles (PEVs),consumer risk may be too great.

1Florida Power and Light, Florida’s largest utility.www.fpl.com/residential/energysaving/programs/oncall.shtml

3 / 28

Distributed Control of Loads

Control of Deferrable LoadsRandomized Control Architecture

Context: Consider population of similar loads that are deferrable.

Constraints: Grid operator demands reliable ancillary service; Consumer demands

reliable service

Control strategy

Requirements:

1.

2.

4 / 28

Distributed Control of Loads

Control of Deferrable LoadsRandomized Control Architecture

Context: Consider population of similar loads that are deferrable.

Constraints: Grid operator demands reliable ancillary service; Consumer demands

reliable service

Control strategy

Requirements:

1. Minimal communication: Each load should know the needs of thegrid, and the status of the service it is intended to provide.

2.

4 / 28

Distributed Control of Loads

Control of Deferrable LoadsRandomized Control Architecture

Context: Consider population of similar loads that are deferrable.

Constraints: Grid operator demands reliable ancillary service; Consumer demands

reliable service

Control strategy

Requirements:

1.

2. Smooth behavior of the aggregate=⇒ Randomization

4 / 28

Distributed Control of Loads

Control of Deferrable LoadsRandomized Control Architecture

Context: Consider population of similar loads that are deferrable.

Constraints: Grid operator demands reliable ancillary service; Consumer demands

reliable service

Control strategy

Requirements:

1. Minimal communication: Each load should know the needs of thegrid, and the status of the service it is intended to provide.

2. Smooth behavior of the aggregate=⇒ Randomization

Need: A practical theory for distributed control based on this architecture

For recent references see thesis of J. Mathieu

4 / 28

Distributed Control of Loads

Example: One Million Pools in FloridaHow Pools Can Help Regulate The Grid

1,5KW 400V

Needs of a single pool

. Filtration system circulates and cleans: Average pool pump uses1.3kW and runs 6-12 hours per day, 7 days per week

. Pool owners are oblivious, until they see frogs and algae

. Pool owners do not trust anyone: Privacy is a big concern

Randomized control strategy is needed.

5 / 28

Distributed Control of Loads

Example: One Million Pools in FloridaHow Pools Can Help Regulate The Grid

1,5KW 400V

Needs of a single pool

. Filtration system circulates and cleans: Average pool pump uses1.3kW and runs 6-12 hours per day, 7 days per week

. Pool owners are oblivious, until they see frogs and algae

. Pool owners do not trust anyone: Privacy is a big concern

Randomized control strategy is needed.

5 / 28

Distributed Control of Loads

Example: One Million Pools in FloridaHow Pools Can Help Regulate The Grid

1,5KW 400V

Needs of a single pool

. Filtration system circulates and cleans: Average pool pump uses1.3kW and runs 6-12 hours per day, 7 days per week

. Pool owners are oblivious, until they see frogs and algae

. Pool owners do not trust anyone: Privacy is a big concern

Randomized control strategy is needed.

5 / 28

Distributed Control of Loads

Example: One Million Pools in FloridaHow Pools Can Help Regulate The Grid

1,5KW 400V

Needs of a single pool

. Filtration system circulates and cleans: Average pool pump uses1.3kW and runs 6-12 hours per day, 7 days per week

. Pool owners are oblivious, until they see frogs and algae

. Pool owners do not trust anyone: Privacy is a big concern

Randomized control strategy is needed.

5 / 28

ytControl @ Utility

Gain

One Million Pools

Disturbance to be rejected

Proportion of pools on

desiredζt

d

dtµt = µtDζt yt = 〈µt,U〉

Intelligent Appliances

Distributed Control of Loads

General ModelControlled Markovian Dynamics

Assumptions

1 Continuous-time model on finite state space X

2 Controlled Markovian generator Dζ(x, y) : x, y ∈ X, ζ ∈ R.

D = D0 models nominal behavior of load.

Each load is subject to common controlled Markovian dynamics.For the ith load,

P{Xi(t+ s) = x+ | Xi(t) = x−} ≈ sDζt(x−, x+) +O(s2)

6 / 28

Distributed Control of Loads

General ModelControlled Markovian Dynamics

Assumptions

1 Continuous-time model on finite state space X

2 Controlled Markovian generator Dζ(x, y) : x, y ∈ X, ζ ∈ R.D = D0 models nominal behavior of load.

Each load is subject to common controlled Markovian dynamics.For the ith load,

P{Xi(t+ s) = x+ | Xi(t) = x−} ≈ sDζt(x−, x+) +O(s2)

6 / 28

Distributed Control of Loads

General ModelControlled Markovian Dynamics

Assumptions

1 Continuous-time model on finite state space X

2 Controlled Markovian generator Dζ(x, y) : x, y ∈ X, ζ ∈ R.D = D0 models nominal behavior of load.

3 Smooth and Lipschitz: For a constant b,∣∣∣ ddζDζ(x, y)

∣∣∣ ≤ b, for x, y ∈ X, ζ ∈ R.

Each load is subject to common controlled Markovian dynamics.For the ith load,

P{Xi(t+ s) = x+ | Xi(t) = x−} ≈ sDζt(x−, x+) +O(s2)

6 / 28

Distributed Control of Loads

General ModelControlled Markovian Dynamics

Assumptions

1 Continuous-time model on finite state space X2 Controlled Markovian generator Dζ(x, y) : x, y ∈ X, ζ ∈ R.

D = D0 models nominal behavior of load.

3 Smooth and Lipschitz: For a constant b,∣∣∣ ddζDζ(x, y)

∣∣∣ ≤ b, for x, y ∈ X, ζ ∈ R.

4 Utility function U : X→ Rrepresents state-dependent power consumption

Each load is subject to common controlled Markovian dynamics.For the ith load,

P{Xi(t+ s) = x+ | Xi(t) = x−} ≈ sDζt(x−, x+) +O(s2)

6 / 28

Distributed Control of Loads

General ModelControlled Markovian Dynamics

Assumptions

1 Continuous-time model on finite state space X

2 Controlled Markovian generator Dζ(x, y) : x, y ∈ X, ζ ∈ R.D = D0 models nominal behavior of load.

Each load is subject to common controlled Markovian dynamics.For the ith load,

P{Xi(t+ s) = x+ | Xi(t) = x−} ≈ sDζt(x−, x+) +O(s2)

6 / 28

Distributed Control of Loads

General ModelMean Field Model

Aggregate model: N loads running independently,each under the command ζ.

Empirical Distributions:

µNt (x) =1N

N∑i=1

I{Xi(t) = x}, x ∈ X

7 / 28

Distributed Control of Loads

General ModelMean Field Model

Aggregate model: N loads running independently,each under the command ζ.

Empirical Distributions:

µNt (x) =1N

N∑i=1

I{Xi(t) = x}, x ∈ X

Limiting model:

d

dtµt(x′) =

∑x∈X

µt(x)Dζt(x, x′), yt :=∑x

µt(x)U(x)

via Law of Large Numbers for martingales

7 / 28

Distributed Control of Loads

General ModelMean Field Model

Aggregate model: N loads running independently,each under the command ζ.

Empirical Distributions:

µNt (x) =1N

N∑i=1

I{Xi(t) = x}, x ∈ X

Mean-field model:

d

dtµt = µtDζt , yt = µt(U)

ζt = ft(µ0, . . . , µt) by design

7 / 28

Distributed Control of Loads

General ModelMean Field Model

Mean-field model:

d

dtµt = µtDζt , yt = µt(U)

ζt = ft(µ0, . . . , µt) by design

ytControl @ Utility

Gain

N Loads

Disturbance to be rejecteddesired

ζt

µt+1 = µtPζtyt = 〈µt,U〉

Questions: 1. How to control this complex nonlinear system? CDC 2013

2. How to design {Dζ} so that control is easy? focus here

8 / 28

Distributed Control of Loads

General ModelMean Field Model

Mean-field model:

d

dtµt = µtDζt , yt = µt(U)

ζt = ft(µ0, . . . , µt) by design

ytControl @ Utility

Gain

N Loads

Disturbance to be rejecteddesired

ζt

µt+1 = µtPζtyt = 〈µt,U〉

Questions: 1. How to control this complex nonlinear system? CDC 2013

2. How to design {Dζ} so that control is easy?

focus here

8 / 28

Distributed Control of Loads

General ModelMean Field Model

Mean-field model:

d

dtµt = µtDζt , yt = µt(U)

ζt = ft(µ0, . . . , µt) by design

ytControl @ Utility

Gain

N Loads

Disturbance to be rejecteddesired

ζt

µt+1 = µtPζtyt = 〈µt,U〉

Questions: 1. How to control this complex nonlinear system? CDC 2013

2. How to design {Dζ} so that control is easy? focus here

8 / 28

Design

Design for Intelligent Loads

Mean Field ModelDesign via “Bellman-Shannon” Optimal Control

Goal: Construct a family of generators {Dζ : ζ ∈ R}.

Explicit solution for finite T :

p∗ζ(xT0 ) ∝ exp

(ζ

∫ T

t=0U(xt) dt

)p0(xT0 )

Markovian, but not time-homogeneous.As T →∞, we obtain generator DζSimple construction via eigenvector problem:

Dζ(x, y) =v(y)v(x)

[ζU(x)− Λ +D(x, y)

]where Dv = Λv, D(x, y) = ζU(x) +D(x, y)

Extension/reinterpretation of [Todorov 2007] ⊕ Kontoyiannis & M 200X

9 / 28

Design for Intelligent Loads

Mean Field ModelDesign via “Bellman-Shannon” Optimal Control

Goal: Construct a family of generators {Dζ : ζ ∈ R}.Design: Consider first the finite-horizon control problem:Choose distribution pζ to maximize

1T

(ζEpζ

[∫ T

t=0U(Xt)

]−D(pζ‖p0)

)D denotes relative entropy.

p0 denotes nominal Markovian model.

Explicit solution for finite T :

p∗ζ(xT0 ) ∝ exp

(ζ

∫ T

t=0U(xt) dt

)p0(xT0 )

Markovian, but not time-homogeneous.As T →∞, we obtain generator DζSimple construction via eigenvector problem:

Dζ(x, y) =v(y)v(x)

[ζU(x)− Λ +D(x, y)

]where Dv = Λv, D(x, y) = ζU(x) +D(x, y)

Extension/reinterpretation of [Todorov 2007] ⊕ Kontoyiannis & M 200X

9 / 28

Design for Intelligent Loads

Mean Field ModelDesign via “Bellman-Shannon” Optimal Control

Goal: Construct a family of generators {Dζ : ζ ∈ R}.Design: Consider first the finite-horizon control problem:Choose distribution pζ to maximize

1T

(ζEpζ

[∫ T

t=0U(Xt)

]−D(pζ‖p0)

)D denotes relative entropy.

p0 denotes nominal Markovian model.

Explicit solution for finite T :

p∗ζ(xT0 ) ∝ exp

(ζ

∫ T

t=0U(xt) dt

)p0(xT0 )

Markovian, but not time-homogeneous.As T →∞, we obtain generator DζSimple construction via eigenvector problem:

Dζ(x, y) =v(y)v(x)

[ζU(x)− Λ +D(x, y)

]where Dv = Λv, D(x, y) = ζU(x) +D(x, y)

Extension/reinterpretation of [Todorov 2007] ⊕ Kontoyiannis & M 200X

9 / 28

Design for Intelligent Loads

Mean Field ModelDesign via “Bellman-Shannon” Optimal Control

Explicit solution for finite T :

p∗ζ(xT0 ) ∝ exp

(ζ

∫ T

t=0U(xt) dt

)p0(xT0 )

Markovian, but not time-homogeneous.As T →∞, we obtain generator DζSimple construction via eigenvector problem:

Dζ(x, y) =v(y)v(x)

[ζU(x)− Λ +D(x, y)

]where Dv = Λv, D(x, y) = ζU(x) +D(x, y)

Extension/reinterpretation of [Todorov 2007] ⊕ Kontoyiannis & M 200X

9 / 28

Design for Intelligent Loads

Mean Field ModelDesign via “Bellman-Shannon” Optimal Control

Explicit solution for finite T :

p∗ζ(xT0 ) ∝ exp

(ζ

∫ T

t=0U(xt) dt

)p0(xT0 )

Markovian, but not time-homogeneous.As T →∞, we obtain generator Dζ

Simple construction via eigenvector problem:

Dζ(x, y) =v(y)v(x)

[ζU(x)− Λ +D(x, y)

]where Dv = Λv, D(x, y) = ζU(x) +D(x, y)

Extension/reinterpretation of [Todorov 2007] ⊕ Kontoyiannis & M 200X

9 / 28

Design for Intelligent Loads

Mean Field ModelDesign via “Bellman-Shannon” Optimal Control

Explicit solution for finite T :

p∗ζ(xT0 ) ∝ exp

(ζ

∫ T

t=0U(xt) dt

)p0(xT0 )

Markovian, but not time-homogeneous.

As T →∞, we obtain generator DζSimple construction via eigenvector problem:

Dζ(x, y) =v(y)v(x)

[ζU(x)− Λ +D(x, y)

]where Dv = Λv, D(x, y) = ζU(x) +D(x, y)

Extension/reinterpretation of [Todorov 2007] ⊕ Kontoyiannis & M 200X

9 / 28

Design for Intelligent Loads

Mean Field ModelDesign via “Bellman-Shannon” Optimal Control

Explicit solution for finite T :

p∗ζ(xT0 ) ∝ exp

(ζ

∫ T

t=0U(xt) dt

)p0(xT0 )

Markovian, but not time-homogeneous.

As T →∞, we obtain generator DζSimple construction via eigenvector problem:

Dζ(x, y) =v(y)v(x)

[ζU(x)− Λ +D(x, y)

]

where Dv = Λv, D(x, y) = ζU(x) +D(x, y)

Extension/reinterpretation of [Todorov 2007] ⊕ Kontoyiannis & M 200X

9 / 28

Design for Intelligent Loads

Mean Field ModelDesign via “Bellman-Shannon” Optimal Control

Explicit solution for finite T :

p∗ζ(xT0 ) ∝ exp

(ζ

∫ T

t=0U(xt) dt

)p0(xT0 )

Markovian, but not time-homogeneous.

As T →∞, we obtain generator DζSimple construction via eigenvector problem:

Dζ(x, y) =v(y)v(x)

[ζU(x)− Λ +D(x, y)

]where Dv = Λv, D(x, y) = ζU(x) +D(x, y)

Extension/reinterpretation of [Todorov 2007] ⊕ Kontoyiannis & M 200X

9 / 28

d

dtµt = µtDζt

yt = 〈µt,U〉

d

dtΦt = AΦt + Bζt

γt = CΦt

Linearized Dynamics

Linearized Dynamics and Passivity

Mean Field ModelLinearized Dynamics

Mean-field model: ddtµt = µtDζt , yt = µt(U)

Linear state space model:Φt+1 = AΦt +Bζt

γt = CΦt

Interpretations: |ζt| is small, and π denotes invariant measure for D.

• Φt ∈ R|X|, a column vector withΦt(x) ≈ µt(x)− π(x), x ∈ X

• γt ≈ yt − y0; deviation from nominal steady-state

• A = DT, Ci = U(xi), and input dynamics linearized:

BT =d

dζπDζ

∣∣∣ζ=0

10 / 28

Linearized Dynamics and Passivity

Mean Field ModelLinearized Dynamics

Mean-field model: ddtµt = µtDζt , yt = µt(U)

Linear state space model:Φt+1 = AΦt +Bζt

γt = CΦt

Interpretations: |ζt| is small, and π denotes invariant measure for D.

• Φt ∈ R|X|, a column vector withΦt(x) ≈ µt(x)− π(x), x ∈ X

• γt ≈ yt − y0; deviation from nominal steady-state

• A = DT, Ci = U(xi), and input dynamics linearized:

BT =d

dζπDζ

∣∣∣ζ=0

10 / 28

Linearized Dynamics and Passivity

Mean Field ModelLinearized Dynamics

Mean-field model: ddtµt = µtDζt , yt = µt(U)

Linear state space model:Φt+1 = AΦt +Bζt

γt = CΦt

Interpretations: |ζt| is small, and π denotes invariant measure for D.

• Φt ∈ R|X|, a column vector withΦt(x) ≈ µt(x)− π(x), x ∈ X

• γt ≈ yt − y0; deviation from nominal steady-state

• A = DT, Ci = U(xi), and input dynamics linearized:

BT =d

dζπDζ

∣∣∣ζ=0

10 / 28

Linearized Dynamics and Passivity

Mean Field ModelLinearized Dynamics

Mean-field model: ddtµt = µtDζt , yt = µt(U)

Linear state space model:Φt+1 = AΦt +Bζt

γt = CΦt

Interpretations: |ζt| is small, and π denotes invariant measure for D.

• Φt ∈ R|X|, a column vector withΦt(x) ≈ µt(x)− π(x), x ∈ X

• γt ≈ yt − y0; deviation from nominal steady-state

• A = DT, Ci = U(xi), and input dynamics linearized:

BT =d

dζπDζ

∣∣∣ζ=0

10 / 28

Linearized Dynamics and Passivity

Mean Field ModelLinearized Dynamics

Mean-field model: ddtµt = µtDζt , yt = µt(U)

Linear state space model:Φt+1 = AΦt +Bζt

γt = CΦt

Interpretations: |ζt| is small, and π denotes invariant measure for D.

• Φt ∈ R|X|, a column vector withΦt(x) ≈ µt(x)− π(x), x ∈ X

• γt ≈ yt − y0; deviation from nominal steady-state

• A = DT, Ci = U(xi), and input dynamics linearized:

BT =d

dζπDζ

∣∣∣ζ=0

10 / 28

Linearized Dynamics and Passivity

Mean Field ModelLinearized Dynamics

Mean-field model: ddtµt = µtDζt , yt = µt(U)

Linear state space model:Φt+1 = AΦt +Bζt

γt = CΦt

Interpretations: |ζt| is small, and π denotes invariant measure for D.

• Φt ∈ R|X|, a column vector withΦt(x) ≈ µt(x)− π(x), x ∈ X

• γt ≈ yt − y0; deviation from nominal steady-state

• A = DT, Ci = U(xi), and input dynamics linearized:

BT =d

dζπDζ

∣∣∣ζ=0

10 / 28

Linearized Dynamics and Passivity

Linearized DynamicsTransfer Function

Linear state space model:

Φt+1 = AΦt +Bζt

γt = CΦt A = DT, Ci = U(xi), BT =d

dζπDζ

∣∣∣ζ=0

Transfer Function:

G(s) = C[Is−A]−1B = C[Is−DT]−1B

hmmmm...

= CRTsB TF for L-MFM � Resolvent for one load

Resolvent Matrix:

Rs =∫ ∞

0e−stetD dt = [Is−D]−1

11 / 28

Linearized Dynamics and Passivity

Linearized DynamicsTransfer Function

Linear state space model:

Φt+1 = AΦt +Bζt

γt = CΦt A = DT, Ci = U(xi), BT =d

dζπDζ

∣∣∣ζ=0

Transfer Function:

G(s) = C[Is−A]−1B = C[Is−DT]−1B hmmmm...

= CRTsB TF for L-MFM � Resolvent for one load

Resolvent Matrix:

Rs =∫ ∞

0e−stetD dt = [Is−D]−1

11 / 28

Linearized Dynamics and Passivity

Linearized DynamicsTransfer Function

Linear state space model:

Φt+1 = AΦt +Bζt

γt = CΦt A = DT, Ci = U(xi), BT =d

dζπDζ

∣∣∣ζ=0

Transfer Function:

G(s) = C[Is−A]−1B = C[Is−DT]−1B hmmmm...

= CRTsB TF for L-MFM � Resolvent for one load

Resolvent Matrix:

Rs =∫ ∞

0e−stetD dt = [Is−D]−1

11 / 28

Linearized Dynamics and Passivity

Linearized DynamicsTransfer Function

Linear state space model:

Φt+1 = AΦt +Bζt

γt = CΦt A = DT, Ci = U(xi), BT =d

dζπDζ

∣∣∣ζ=0

Transfer Function:

G(s) = C[Is−A]−1B = C[Is−DT]−1B hmmmm...

= CRTsB TF for L-MFM � Resolvent for one load

Resolvent Matrix:

Rs =∫ ∞

0e−stetD dt = [Is−D]−1

11 / 28

Linearized Dynamics and Passivity

Linearized DynamicsPassive Pools

Theorem: Reversibility =⇒ Passivity

Suppose that the nominal model is reversible.Then its linearization satisfies,

ReG(jω) = PSDY (ω), ω ∈ R ,

whereG(s) = C[Is−A]−1B for s ∈ C.

PSDY (ω) =∫ ∞−∞

e−jωEπ[U(X0)U(Xt)] dt

Implication for control: G(s) is positive real

12 / 28

Linearized Dynamics and Passivity

Linearized DynamicsPassive Pools

Theorem: Reversibility =⇒ Passivity

Suppose that the nominal model is reversible.Then its linearization satisfies,

ReG(jω) = PSDY (ω), ω ∈ R ,

whereG(s) = C[Is−A]−1B for s ∈ C.

PSDY (ω) =∫ ∞−∞

e−jωEπ[U(X0)U(Xt)] dt

Implication for control: G(s) is positive real

12 / 28

Linearized Dynamics and Passivity

Linearized DynamicsPassive Pools

Theorem: Reversibility =⇒ Passivity

Suppose that the nominal model is reversible.Then its linearization satisfies,

ReG(jω) = PSDY (ω), ω ∈ R ,

whereG(s) = C[Is−A]−1B for s ∈ C.

PSDY (ω) =∫ ∞−∞

e−jωEπ[U(X0)U(Xt)] dt

Implication for control: G(s) is positive real

12 / 28

Linearized Dynamics and Passivity

Linearized DynamicsExample Without Passivity

Example: Eight state model Utility function U(xi) = i.

1 2

3 4

5 6

7 8

a c

c

a

a a

a a

a a

b b b b

Not reversible

13 / 28

Linearized Dynamics and Passivity

Linearized DynamicsExample Without Passivity

Example: Eight state model Utility function U(xi) = i.Generator is the 8× 8 matrix,

D =

−a a 0 0 0 0 0 0a −(a+ b+ c) 0 b c 0 0 0b 0 −(a+ b) a 0 0 0 00 0 a −a 0 0 0 00 0 0 0 −a a 0 00 0 0 0 a −(a+ b) 0 b0 0 0 c b 0 −(a+ b+ c) a0 0 0 0 0 0 a −a

13 / 28

Linearized Dynamics and Passivity

Linearized DynamicsExample Without Passivity

1 2

3 4

5 6

7 8

a c

c

a

a a

a a

a a

b b b b

Example: Eight state model a = c = 10, b = 1

−2

−1

0

1

2

Real Axis−1 0 1 2

Imag

inar

y A

xis

−25 −20 −15 −10 −5 0 5 10−15

−10

−5

0

5

10

15

Real Part

Imag

inar

y Pa

rt

Pole-zero Plot

G(jω), ω >0

Non-minimum phase zero

Nyquist Plot

G(s) = C[Is−A]−1B not positive real13 / 28

ytControl @ Utility

Gain

One Million Pools

Disturbance to be rejected

Proportion of pools on

desiredζt

µt+1 = µtPζtyt = 〈µt,U〉

One Million Pools

One Million Pools

A Single PoolControl Architecture

Transition diagram for a single pool:

1 2 T−1 T. . .

T

On

O�

12T−1

...

Numerics using discrete-time model

1 2 T−1 T. . .

T

On

O�

12T−1

...

Utility:

U(x) = I{

Pool is on}∣∣∣x∝ (Power consumption)

∣∣∣x

Controlled dynamics: As ζ increases, probability of turning on increases:

0 24120

0.5

1

zζ=

- 4

ζ =

-2

ζ = 4

ζ = 2

ζ = 0

(ζ)

14 / 28

One Million Pools

A Single PoolControl Architecture

1 2 T−1 T. . .

T

On

O�

12T−1

...

Utility:

U(x) = I{

Pool is on}∣∣∣x∝ (Power consumption)

∣∣∣x

Controlled dynamics: As ζ increases, probability of turning on increases:

0 24120

0.5

1

zζ=

- 4

ζ =

-2

ζ = 4

ζ = 2

ζ = 0

(ζ)

14 / 28

One Million Pools

A Single PoolControl Architecture

1 2 T−1 T. . .

T

On

O�

12T−1

...

Utility:

U(x) = I{

Pool is on}∣∣∣x∝ (Power consumption)

∣∣∣x

Controlled dynamics: As ζ increases, probability of turning on increases:

0 24120

0.5

1

zζ=

- 4

ζ =

-2

ζ = 4

ζ = 2

ζ = 0

(ζ)

14 / 28

One Million Pools

Mean Field Pool ModelLinearization: Minimum Phase Pole-Zero Plot

−1 0 1

−1

0

1

Real PartIm

agin

ary

Part

Transfer function

Normalized Frequency ( ×π rad/sample)1/24 hrs0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

-180

0

20

Phas

e (d

egre

es)

Mag

nitu

de (d

B)

40

-90

0

15 / 28

One Million Pools

Mean Field Pool ModelStochastic simulation — Filtered regulation signal

−400

−200

0

200

400

MW

July 18 July 20July 19 July 21 July 23July 22 July 24

BPA Balancing Reserves Deployed (July 2013)

−400

−200

0

200

400

MW

July 18 July 20July 19 July 21 July 23July 22 July 24

Filtered BPA reserve signal that can be tracked by pools BPA Balancing Reserves Deployed (July 2013)

∗transmission.bpa.gov/Business/Operations/Wind/reserves.aspx

16 / 28

One Million Pools

Mean Field Pool ModelStochastic simulation — Filtered regulation signal

−400

−200

0

200

400

MW

July 18 July 20July 19 July 21 July 23July 22 July 24

BPA Balancing Reserves Deployed (July 2013)

−400

−200

0

200

400

MW

July 18 July 20July 19 July 21 July 23July 22 July 24

Filtered BPA reserve signal that can be tracked by pools BPA Balancing Reserves Deployed (July 2013)

∗transmission.bpa.gov/Business/Operations/Wind/reserves.aspx

16 / 28

One Million Pools

Mean Field Pool ModelStochastic simulation using N = 105 pools PI control

Reference (from Bonneville Power Authority)

−300

−200

−100

0

100

200

300

Trac

king

BPA

Reg

ulat

ion

Sign

al

(MW

)

Reference (from Bonneville Power Authority)Output deviation y

−300

−200

−100

0

100

200

300

Trac

king

BPA

Reg

ulat

ion

Sign

al

(MW

)

Reference (from Bonneville Power Authority)Output deviation y

−300

−200

−100

0

100

200

300

-2

-1

-3

2

3

0

1

Input ζ

ζ

Trac

king

BPA

Reg

ulat

ion

Sign

al

(MW

)

17 / 28

One Million Pools

Mean Field Pool ModelStochastic simulation using N = 105 pools PI control

Reference (from Bonneville Power Authority)

−300

−200

−100

0

100

200

300

Trac

king

BPA

Reg

ulat

ion

Sign

al

(MW

)

Reference (from Bonneville Power Authority)Output deviation y

−300

−200

−100

0

100

200

300

Trac

king

BPA

Reg

ulat

ion

Sign

al

(MW

)

Reference (from Bonneville Power Authority)Output deviation y

−300

−200

−100

0

100

200

300

-2

-1

-3

2

3

0

1

Input ζ

ζ

Trac

king

BPA

Reg

ulat

ion

Sign

al

(MW

)

17 / 28

One Million Pools

Mean Field Pool ModelStochastic simulation using N = 105 pools PI control

Reference (from Bonneville Power Authority)

−300

−200

−100

0

100

200

300

Trac

king

BPA

Reg

ulat

ion

Sign

al

(MW

)

Reference (from Bonneville Power Authority)Output deviation y

−300

−200

−100

0

100

200

300

Trac

king

BPA

Reg

ulat

ion

Sign

al

(MW

)

Reference (from Bonneville Power Authority)Output deviation y

−300

−200

−100

0

100

200

300

-2

-1

-3

2

3

0

1

Input ζ

ζ

Trac

king

BPA

Reg

ulat

ion

Sign

al

(MW

)

17 / 28

One Million Pools

Mean Field Pool ModelStochastic simulation using N = 105 pools PI control

Two scenarios, using two different reference signals:

0 20 40 60 80 100 120 140 160t/hour

0 20 40 60 80 100 120 140 160 0 20 40 60 80 100 120 140 160t/hour

0 20 40 60 80 100 120 140 160

Refe

renc

eO

utpu

t dev

iatio

n (M

W)

Refe

renc

eO

utpu

t dev

iatio

n (M

W)

Refe

renc

eO

utpu

t dev

iatio

n (M

W)

Refe

renc

eO

utpu

t dev

iatio

n (M

W)

−300

−200

−100

0

100

200

300

−300

−200

−100

0

100

200

300

−600

−400

−200

0

200

400

600

−400

−200

0

200

400

600

800

-2

-1

-3

2

3

0

1

−3

−2

−1

0

1

2

3

−5

0

5

−4

−2

0

2

4

6

Inpu

t

Inpu

t

Inpu

t

Inpu

t

12 Hour Cleaning Cycle 8 Hour Cleaning Cycle

Trac

king

BPA

Reg

ulat

ion

Sign

alBP

A R

egul

atio

n Si

gnal

- Dou

bled

(a) (b)

(c) (d)

18 / 28

One Million Pools

Mean Field Pool ModelStochastic simulation using N = 105 pools PI control

The impact of exceeding capacity

0 20 40 60 80 100 120 140 160

0 20 40 60 80 100 120 140 160

Refe

renc

eO

utpu

t dev

iatio

n (M

W)

Refe

renc

eO

utpu

t dev

iatio

n (M

W)

Refe

renc

eO

utpu

t dev

iatio

n (M

W)

Refe

renc

eO

utpu

t dev

iatio

n (M

W)

0 20 40 60 80 100 120 140 160t/hour

0 20 40 60 80 100 120 140 160

−1000

−500

0

500

1000

−1000

−500

0

500

1000

−1000

−500

0

500

1000

−1000

−500

0

500

1000

−20

−10

0

10

20

−20

−10

0

10

20

−10

0

10

−10

0

10

Inpu

t

Inpu

t

Inpu

t

Inpu

t

12 Hour Cleaning Cycle 8 Hour Cleaning Cycle

Regu

latio

n ex

ceed

s ca

paci

tyIn

tegr

ator

win

d-up

Rec

over

y

(a) (b)

(c) (d)

19 / 28

Power GridControl Water PumpBatteries

CoalGas Turbine

BP

BP

BP C

BP

BP

Voltage Frequency Phase

HCΣ−

Actuator feedback loop

A

LOAD

Conclusions and Extensions

Conclusions and Extensions

ConclusionsRecap

QUIZ: Why intelligence at the loads?

To simplify control at the grid level

A particular approach to distributed control is proposed

The grid level control problem is simple because:

? Mean field model is simple, and a good approximation of finite system

? LTI approximation is positive real, which implies minimum phase

20 / 28

Conclusions and Extensions

ConclusionsRecap

QUIZ: Why intelligence at the loads?To simplify control at the grid level

A particular approach to distributed control is proposed

The grid level control problem is simple because:

? Mean field model is simple, and a good approximation of finite system

? LTI approximation is positive real, which implies minimum phase

20 / 28

Conclusions and Extensions

ConclusionsRecap

QUIZ: Why intelligence at the loads?To simplify control at the grid level

A particular approach to distributed control is proposed

The grid level control problem is simple because:

? Mean field model is simple, and a good approximation of finite system

? LTI approximation is positive real, which implies minimum phase

20 / 28

Conclusions and Extensions

ConclusionsRecap

QUIZ: Why intelligence at the loads?To simplify control at the grid level

A particular approach to distributed control is proposed

The grid level control problem is simple because:

? Mean field model is simple, and a good approximation of finite system

? LTI approximation is positive real, which implies minimum phase

20 / 28

Conclusions and Extensions

ConclusionsRecap

QUIZ: Why intelligence at the loads?To simplify control at the grid level

A particular approach to distributed control is proposed

The grid level control problem is simple because:

? Mean field model is simple, and a good approximation of finite system

? LTI approximation is positive real, which implies minimum phase

20 / 28

Conclusions and Extensions

ConclusionsAre the customers happy?

No time for details, but no ...

There will be “rare events” in which the pool is under- or over-cleaned.

Proposed approach: Additional layer of control at each load, so that hardconstraints on performance can be assured.

This will create some modeling error at grid level.

Preliminary experiments on the pool model:No loss of performance at grid level.

21 / 28

Conclusions and Extensions

ConclusionsAre the customers happy?

No time for details, but no ...

QoS without local control−100 −80 −60 −40 −20 0 20 40 60 80 1000

0.01

0.02

0.03

0.04

0.05

fract

ion

of p

ools

Histogram of pool healthGaussian distribution

There will be “rare events” in which the pool is under- or over-cleaned.

Proposed approach: Additional layer of control at each load, so that hardconstraints on performance can be assured.

This will create some modeling error at grid level.

Preliminary experiments on the pool model:No loss of performance at grid level.

21 / 28

Conclusions and Extensions

ConclusionsAre the customers happy?

No time for details, but no ...

QoS without local control−100 −80 −60 −40 −20 0 20 40 60 80 1000

0.01

0.02

0.03

0.04

0.05

fra

ctio

n of

poo

ls

Histogram of pool healthGaussian distribution

There will be “rare events” in which the pool is under- or over-cleaned.

Proposed approach: Additional layer of control at each load, so that hardconstraints on performance can be assured.

This will create some modeling error at grid level.

Preliminary experiments on the pool model:No loss of performance at grid level.

21 / 28

Conclusions and Extensions

ConclusionsAre the customers happy?

No time for details, but no ...

QoS without local control−100 −80 −60 −40 −20 0 20 40 60 80 1000

0.01

0.02

0.03

0.04

0.05

fra

ctio

n of

poo

ls

Histogram of pool healthGaussian distribution

There will be “rare events” in which the pool is under- or over-cleaned.

Proposed approach: Additional layer of control at each load, so that hardconstraints on performance can be assured.

This will create some modeling error at grid level.

Preliminary experiments on the pool model:No loss of performance at grid level.

21 / 28

Conclusions and Extensions

ConclusionsAre the customers happy?

No time for details, but no ...

QoS without local control−100 −80 −60 −40 −20 0 20 40 60 80 1000

0.01

0.02

0.03

0.04

0.05

fra

ctio

n of

poo

ls

Histogram of pool healthGaussian distribution

There will be “rare events” in which the pool is under- or over-cleaned.

Proposed approach: Additional layer of control at each load, so that hardconstraints on performance can be assured.

This will create some modeling error at grid level.

Preliminary experiments on the pool model:No loss of performance at grid level.

21 / 28

Conclusions and Extensions

ConclusionsThe customers are happy

Proposed approach: Additional layer of control at each load, so that hardconstraints on performance can be assured.

QoS without local control−100 −80 −60 −40 −20 0 20 40 60 80 1000

0.01

0.02

0.03

0.04

0.05

fract

ion

of p

ools

Histogram of pool healthGaussian distribution

QoS without local control QoS with local control−100 −80 −60 −40 −20 0 20 40 60 80 1000

0.01

0.02

0.03

0.04

0.05

fract

ion

of p

ools

Histogram of pool healthConditional Gaussian distribution

Histogram of pool healthGaussian distribution

−100 −80 −60 −40 −20 0 20 40 60 80 100

Load opts-out when its QoS is outside of prescribed bounds

Analysis: LQG approximation for individual load – CDC 2014

22 / 28

Conclusions and Extensions

ConclusionsThe customers are happy

Proposed approach: Additional layer of control at each load, so that hardconstraints on performance can be assured.

QoS without local control−100 −80 −60 −40 −20 0 20 40 60 80 1000

0.01

0.02

0.03

0.04

0.05

fract

ion

of p

ools

Histogram of pool healthGaussian distribution

QoS without local control QoS with local control−100 −80 −60 −40 −20 0 20 40 60 80 1000

0.01

0.02

0.03

0.04

0.05

fract

ion

of p

ools

Histogram of pool healthConditional Gaussian distribution

Histogram of pool healthGaussian distribution

−100 −80 −60 −40 −20 0 20 40 60 80 100

Load opts-out when its QoS is outside of prescribed bounds

Analysis: LQG approximation for individual load – CDC 2014

22 / 28

Conclusions and Extensions

ConclusionsMany issues skipped because they are topics of current research

1 Information at the load: Should these loads simply act as frequencyregulators? Measurement of local frequency deviation will suffice.

Or, more information may be valuable: Two measurements at eachload, the BA command, and local frequency measurements.

2 Information at the BA: Does this macro control view suffice?

Power Grid

Actuation

Control

Water Pump

Batteries

HVAC

Coal

Gas Turbine

BP

BP

BP

BP

BP

Baseline Generation

Disturbancesfrom nature

Measurements: Voltage Frequency Phase

HCΣ−

Does the grid operator need to know the real-time powerconsumption of each population of loads?

23 / 28

Conclusions and Extensions

ConclusionsMany issues skipped because they are topics of current research

1 Information at the load: Should these loads simply act as frequencyregulators? Measurement of local frequency deviation will suffice.

Or, more information may be valuable: Two measurements at eachload, the BA command, and local frequency measurements.

2 Information at the BA: Does this macro control view suffice?

Power Grid

Actuation

Control

Water Pump

Batteries

HVAC

Coal

Gas Turbine

BP

BP

BP

BP

BP

Baseline Generation

Disturbancesfrom nature

Measurements: Voltage Frequency Phase

HCΣ−

Does the grid operator need to know the real-time powerconsumption of each population of loads?

23 / 28

Conclusions and Extensions

ConclusionsMany issues skipped because they are topics of current research

1 Information at the load: Should these loads simply act as frequencyregulators? Measurement of local frequency deviation will suffice.

Or, more information may be valuable: Two measurements at eachload, the BA command, and local frequency measurements.

2 Information at the BA: Does this macro control view suffice?

Power Grid

Actuation

Control

Water Pump

Batteries

HVAC

Coal

Gas Turbine

BP

BP

BP

BP

BP

Baseline Generation

Disturbancesfrom nature

Measurements: Voltage Frequency Phase

HCΣ−

Does the grid operator need to know the real-time powerconsumption of each population of loads?

23 / 28

Conclusions and Extensions

ConclusionsMany issues skipped because they are topics of current research

1 Information at the load: Should these loads simply act as frequencyregulators? Measurement of local frequency deviation will suffice.

Or, more information may be valuable: Two measurements at eachload, the BA command, and local frequency measurements.

2 Information at the BA: Does this macro control view suffice?

Power Grid

Actuation

Control

Water Pump

Batteries

HVAC

Coal

Gas Turbine

BP

BP

BP

BP

BP

Baseline Generation

Disturbancesfrom nature

Measurements: Voltage Frequency Phase

HCΣ−

Does the grid operator need to know the real-time powerconsumption of each population of loads?

23 / 28

Conclusions and Extensions

ConclusionsMany issues skipped because they are topics of current research

3 How can we engage consumers?

The formulation of contracts with customers requires a betterunderstanding of the value of ancillary service, as well as consumerpreferences.

We will not use real timeprices, even via aggregator,if we want responsive, reliableancillary service

Refe

renc

eO

utpu

t dev

iatio

n (M

W)

−300

−200

−100

0

100

200

300

-2

-1

-3

2

3

0

1

Inpu

t

Trac

king

BPA

Reg

ulat

ion

Sign

al

(a)

24 / 28

Conclusions and Extensions

ConclusionsMany issues skipped because they are topics of current research

3 How can we engage consumers?

The formulation of contracts with customers requires a betterunderstanding of the value of ancillary service, as well as consumerpreferences.

We will not use real timeprices, even via aggregator,if we want responsive, reliableancillary service

Refe

renc

eO

utpu

t dev

iatio

n (M

W)

−300

−200

−100

0

100

200

300

-2

-1

-3

2

3

0

1

Inpu

t

Trac

king

BPA

Reg

ulat

ion

Sign

al

(a)

24 / 28

Conclusions and Extensions

ConclusionsMany issues skipped because they are topics of current research

3 How can we engage consumers?

The formulation of contracts with customers requires a betterunderstanding of the value of ancillary service, as well as consumerpreferences.

We will not use real timeprices, even via aggregator,if we want responsive, reliableancillary service

Refe

renc

eO

utpu

t dev

iatio

n (M

W)

−300

−200

−100

0

100

200

300

-2

-1

-3

2

3

0

1

Inpu

t

Trac

king

BPA

Reg

ulat

ion

Sign

al

(a)

24 / 28

Conclusions and Extensions

Conclusions

Thank You!

25 / 28

Control TechniquesFOR

Complex Networks

Sean Meyn

Pre-publication version for on-line viewing. Monograph available for purchase at your favorite retailer More information available at http://www.cambridge.org/us/catalogue/catalogue.asp?isbn=9780521884419

Markov Chainsand

Stochastic Stability

S. P. Meyn and R. L. Tweedie

August 2008 Pre-publication version for on-line viewing. Monograph to appear Februrary 2009

π(f

)<∞

∆V (x) ≤ −f(x) + bIC(x)

‖P n(x, · )− π‖f → 0

supCE

x [Sτ

C(f)]

<∞

References

References

References: Demand Response

S. Meyn, P. Barooah, A. Busic, and J. Ehren. Ancillary service to the grid from deferrableloads: the case for intelligent pool pumps in Florida (Invited). In Proceedings of the 52ndIEEE Conf. on Decision and Control, 2013.

A. Busic and S. Meyn. Passive dynamics in mean field control. ArXiv e-prints:arXiv:1402.4618. Submitted to the 53rd IEEE Conf. on Decision and Control (Invited),2014.

S. Meyn, Y. Chen, and A. Busic. Individual risk in mean-field control models fordecentralized control, with application to automated demand response. Submitted to the53rd IEEE Conf. on Decision and Control (Invited), 2014.

J. L. Mathieu. Modeling, Analysis, and Control of Demand Response Resources. PhDthesis, Berkeley, 2012.

D. Callaway and I. Hiskens, Achieving controllability of electric loads. Proceedings of theIEEE, 99(1):184–199, 2011.

S. Koch, J. Mathieu, and D. Callaway, Modeling and control of aggregated heterogeneousthermostatically controlled loads for ancillary services, in Proc. PSCC, 2011, 1–7.

H. Hao, A. Kowli, Y. Lin, P. Barooah, and S. Meyn Ancillary Service for the Grid ViaControl of Commercial Building HVAC Systems. ACC 2013

(much more on our websites)

26 / 28

References

References: Markov Models

I. Kontoyiannis and S. P. Meyn. Spectral theory and limit theorems for geometricallyergodic Markov processes. Ann. Appl. Probab., 13:304–362, 2003.

I. Kontoyiannis and S. P. Meyn. Large deviations asymptotics and the spectral theory ofmultiplicatively regular Markov processes. Electron. J. Probab., 10(3):61–123 (electronic),2005.

E. Todorov. Linearly-solvable Markov decision problems. In B. Scholkopf, J. Platt, andT. Hoffman, editors, Advances in Neural Information Processing Systems, (19) 1369–1376.MIT Press, Cambridge, MA, 2007.

M. Huang, P. E. Caines, and R. P. Malhame. Large-population cost-coupled LQG problemswith nonuniform agents: Individual-mass behavior and decentralized ε-Nash equilibria.IEEE Trans. Automat. Control, 52(9):1560–1571, 2007.

H. Yin, P. Mehta, S. Meyn, and U. Shanbhag. Synchronization of coupled oscillators is agame. IEEE Transactions on Automatic Control, 57(4):920–935, 2012.

P. Guan, M. Raginsky, and R. Willett. Online Markov decision processes withKullback-Leibler control cost. In American Control Conference (ACC), 2012, 1388–1393,2012.

V.S.Borkar and R.Sundaresan Asympotics of the invariant measure in mean field modelswith jumps. Stochastic Systems, 2(2):322-380, 2012.

27 / 28

References

References: Economics

G. Wang, M. Negrete-Pincetic, A. Kowli, E. Shafieepoorfard, S. Meyn, and U. Shanbhag.Dynamic competitive equilibria in electricity markets. In A. Chakrabortty and M. Illic,editors, Control and Optimization Theory for Electric Smart Grids. Springer, 2011.

G. Wang, A. Kowli, M. Negrete-Pincetic, E. Shafieepoorfard, and S. Meyn. A controltheorist’s perspective on dynamic competitive equilibria in electricity markets. In Proc.18th World Congress of the International Federation of Automatic Control (IFAC), Milano,Italy, 2011.

S. Meyn, M. Negrete-Pincetic, G. Wang, A. Kowli, and E. Shafieepoorfard. The value ofvolatile resources in electricity markets. In Proc. of the 10th IEEE Conf. on Dec. andControl, Atlanta, GA, 2010.

I.-K. Cho and S. P. Meyn. Efficiency and marginal cost pricing in dynamic competitivemarkets with friction. Theoretical Economics, 5(2):215–239, 2010.

U.S. Energy Information Administration. Smart grid legislative and regulatory policies andcase studies. December 12 2011.http://www.eia.gov/analysis/studies/electricity/pdf/smartggrid.pdf

28 / 28