8/2/2019 cdna_cdc99

1/36

Context-Dependent

Network AgentsEPRI/ARO CINS Initiative

CDNA ConsortiumCMU, RPI, TAMU, Wisconsin, UIUC

8/2/2019 cdna_cdc99

2/36

The CDNA Consortium

Carnegie Mellon University

Prof. Pradeep Khosla

Prof. Bruce Krogh

Dr. Eswaran Subrahmanian

Prof. Sarosh Talukdar

Rensselaer Polytechnic Institute

Prof. Joe Chow

Texas A&M University

Prof. Garng Huang

Prof. Mladen Kezunovic

University of Illinois at Urbana-

Champaign

Prof. Lui Sha

University of Minnesota

Prof. Bruce Wollenberg

8/2/2019 cdna_cdc99

3/36

CDNA Objective

s Improve

x agilityand robustness (survivability) of large-scale dynamic

networks

x that face newand unanticipatedoperating conditions.

s Target Networks:

x U.S. Power Grid

x Local networks

8/2/2019 cdna_cdc99

4/36

CDNA Approach

s Improve

x decision-making competenceof components distributed

throughout the network,

x particularlyexisting and future control devices, such as

relays, voltage regulators and FACTS.

8/2/2019 cdna_cdc99

5/36

Why CDNA?

s centralized real-time control is

x infeasible in many situations because of the distribution of

information and growing number of independent decision

makers on the grid

x intractable - robust control algorithms simply dont scale, theproblems are NP hard

x undesirable - we contend that centralized solutions are less

robustagainst major network upsets and less adaptive to

new situations

8/2/2019 cdna_cdc99

6/36

Why CDNA? (contd.)

s control devices are already pre-programmedfor anticipated

situations

BUTone-size fits all strategies are conservative in most

cases, and wrongin some (the most critical!) situations

s necessary communication and computation technology for

CDNA exists today

8/2/2019 cdna_cdc99

7/36

Key Research Issues

s modeling

x operating modes

x contingencies

x impact of restructured power systems

x device capabilities/influence

8/2/2019 cdna_cdc99

8/36

Key Research Issues - 2

s state estimation

x using local information

x network state estimation

x real-time constraints

s hybrid control

x adaptive mode switching

x coverage

8/2/2019 cdna_cdc99

9/36

Key Research Issues - 3

s learning

x distributed learning

x state-space decomposition

s coordination

x collaboration strategies

x moving off-line techniques for asynchronous algorithms on-

line

8/2/2019 cdna_cdc99

10/36

Decentralized Large AreaPower System Control

Bruce WollenbergUniversity of Minnesota

8/2/2019 cdna_cdc99

11/36

Objectivess Research goal is to show how all standard functions built

on a power flow calculation can be accomplished without a

large area (centralized) model and computer system

s Each region of the power system retains its own control

system, models it own power network and communicates

with immediate neighborss Functions that now require central computing

x Security Analysis

x Optimal Power Flow

x

Available Transfer Capability

8/2/2019 cdna_cdc99

12/36

A

C

B

D

E

L A R G E A R E A

C O N T R O L S Y S T E M

R E G I O N A

C O N T R O L

S Y S T E M

R E G I O N C

C O N T R O L

S Y S T E M

R E G I O N B

C O N T R O L

S Y S T E M R E G I O N D

C O N T R O L

S Y S T E M

R E G I O N E

C O N T R O L

S Y S T E M

Typical PowerPool or ISO

Trends:

- Getting larger

- Standard data formats- Less functionality in

regional systemsExamples:

- California ISO- Midwest ISO

8/2/2019 cdna_cdc99

13/36

A

R E G I O N A

C O N T R O L

S Y S T E M

CR E G I O N C

C O N T R O L

S Y S T E M

B

R E G I O N B

C O N T R O L

S Y S T E M

DR E G I O N D

C O N T R O L

S Y S T E M

E

R E G I O N E

C O N T R O L

S Y S T E M

Networked ControlSystems

- Region can be any size

- Can extend to any

number of regions- Aggregate has same

functionality as large

area control system

- Can new functionality

be added that would notbe available in a central

system?

8/2/2019 cdna_cdc99

14/36

Collaborative Nets

Eduardo Camponogara and Sarosh Talukdar

Institute for Complex Engineered Systems

Carnegie Mellon University

8/2/2019 cdna_cdc99

15/36

Controlling Large Networks

Operating goals fall

into categories:

Limitations:

Control Solution:

s Costs & profits

s Safety

s Regulations

s Equipment Limits

s No organization can

cope with alloperating goals

s Need of diverse skills

s Multitudes of agents

s Delegate goals to

separate

organizations

Organization:

Agent:

A network of agents and communication links.

Any entity that makes and implements decisions

such as relays, control devices, and humans.

M l i l O i i i

8/2/2019 cdna_cdc99

16/36

Multiple Organizations inthe Power Grid

Governors, exciters

optimization soft.

Agents

Generator Control Security Systems

Relays

Protection Systems

Simulation & learn.

tools, humans

Goals Keep equipmentunder limits

Reduce costs.t. constraints

Prevent cascadingfailures

Reaction

Time

0.01 to 0.1secs Seconds Hours, days

Low Agent Skills High

Large Number of Agents Small

Fast Agent Speed Slow

O i i D N

8/2/2019 cdna_cdc99

17/36

Organizations Do NotCollaborate

Generator Control Security SystemsProtection Systems

Current Scenario: s Agents in separate organizations do not talk

s

Agents might work at cross-purposes Organizations might interfere with one another

How do we make individual agents more effective?

How do we prevent interference between organizations?

I i O ll

8/2/2019 cdna_cdc99

18/36

Improving OverallPerformance of NetsThe suggested answer is based on:

Generator Control Security SystemsProtection Systems

1.) The use of a common framework to specify agent tasks.

2.) The implementation of a sparse, collaborative net that can cut

across the hierarchic organizations.

3.) The design of collaboration protocols to promote effective

exchange of information.

C-NetC-Net

8/2/2019 cdna_cdc99

19/36

What Is A Collaborative Net?

A flat organization of dissimilar agents that

can integrate hierarchic organization.

Properties:

s Agents are autonomous within the C-Net. They

have initiative, make and implement decisions.

s Agents collaborate with their neighbors.

The collaboration protocol determines:

x what information is exchanged,

x in which way, and

x how agents make use of it.

Advantages: Disadvantages:

s Quick

s Fault Tolerant

s Open

s No structural coordination. if necessary, it can

emerge from the collaboration protocol.

s Unfamiliar.

Th R lli H i

8/2/2019 cdna_cdc99

20/36

The Rolling HorizonFormulation

A framework to solve dynamic control problems

as a series of static optimization problems.

The dynamic control problem

The steps of the rolling horizon formulation:

1.) Choose a horizon [t0,..,tN], I.E. a set of timepoints where t0 is the current time.

2.) Letx(tn) be the state predicted at time tn.

x(t0) is the current state.

3.) Let u(tn) be the planned actions at time tn.4.) Let X=[x(t0),,x(tN)] and U=[u(t0),,u(tN)]

5.) Choose a model to predictx(tn+1) fromx(tn)

and u(tn). Possibly, a discrete approximation

of the dynamic equations (e.g., Eulers

step).

Minimize f(x,dx/dt,u,t)

Subject to h(x,dx/dt,u,t)=0

g(x,dx/dt,u,t)

8/2/2019 cdna_cdc99

21/36

The Rolling HorizonAlgorithm

1.) The current time it t0.

2.) Sense the current statex(t0)

3.) Instantiate the static optimization problem (P).

4.) Solve (P) to obtain the control actions

U=[u(t0),,u(tN)].

5.) Implement the control action u(t0).

6.) Pause and let the physical network progress

in time. The horizon rolls forward.

7.) Repeat from step 1.

s A model is used to predict the future state of the physical network

over a set of discrete points in time (horizon).

s An optimization procedure computes the control actions, over the horizon,

that minimize error.

Steps of the Algorithm:

s The horizon has to be

long enough to avoid

present actions with

poor long-term effects.

s

Accuracy of theprediction model.

Design Issues:

8/2/2019 cdna_cdc99

22/36

t0 t1 t4Time

Control

t2 t3

now

The Rolling Horizon

Plan ahead

model predicted controlimplemented control

8/2/2019 cdna_cdc99

23/36

t0 t1 t4Time

Control

t2 t3

now

plans at t0

plans at t1

The Rolling Horizon

Update plans frequently

A F k f S if i

8/2/2019 cdna_cdc99

24/36

A Framework for SpecifyingAgent Tasks

Break up the static optimization problem, (P),

into

a set of M small, localized subproblems, {(Pm)}.

Assemble M agents into a C-Net, so that each

agent matches one subproblem.

Agent m and its subproblem (Pm)

It has partial perception of,

and limited authority over,

the physical network.

Neighborhood variables (ym)

Variables sensed or set by neighbors.

Proximate variables (xm,um):

It senses the values of a subsetxm ofx.

It sets the values of a subset um ofu.

Remote variables (zm):

All the other variables.

(P)

(P1) (P3)(P2) (P4)

Ag1

C-Net

M t hi A t t

8/2/2019 cdna_cdc99

25/36

Matching Agents toSubproblems

The rolling horizon

formulation of (Pm)

Minimize fm(Xm,Um,Ym,Zm)

Subject to Hm(Xm,Um,Ym,Zm) = 0

Gm(Xm,Um,Ym,Zm)

8/2/2019 cdna_cdc99

26/36

Collaboration Protocols

A protocol prescribes: a) the data exchanged by agents,

b) in which way, and

c) how agents use the data to solve their problems.

Vers

ions

Voting

Proximate Exchange

Each agent broadcasts its plans to nearby agents

which, in turn, take these plans into account.

Semi-synchronous, semi-parallel (mutual

help).

Synchronization between neighbors.

Parallel work if agents are non-neighbors.

In setting the values of its controls, each agent takes

the votes of its neighbors into account.

Asynchronous, parallel.

Two protocols

Equivalence and

8/2/2019 cdna_cdc99

27/36

Equivalence andConvergence

Two Questions:

Equivalence:

When are the solutions to the network of subproblems,{(Pm)}, solutions to (P)?

Sufficient conditions for equivalence and convergence:

The C-Net must provide complete coverage of the network.1.) Coverage:

The matching of agents to subproblems must be exact.2.) Density:

(P) must be convex.3.) Convexity:

(P) must be strictly feasible.4.) Feasibility:

The agents must use an interior-point-method.5.) Int-Pt-Mtd:

The agents run the semi-synchronous, semi-parallel protocol.6.) Serial Work:

Convergence:

When does the effort of the collaborative agents

converge to a solution of {(Pm)}?

Relaxing Sufficient

8/2/2019 cdna_cdc99

28/36

Relaxing SufficientConditions in PracticeWe believe that the following conditions can be relaxed in practice:

Near matching of agents to problems are likely to be adequate.1.) Density:

It is impractical in real-world networks.2.) Convexity:

Serial work within a neighborhood is too slow.3.) Serial work:

A prototypical network: A forest of pendulums.

- One agent at each pend.

- Agents control two forces:

Horizontal & Orthogonal.

- Agents collaborate with

nearest neighbors.

The Dynamic Control

8/2/2019 cdna_cdc99

29/36

The Dynamic ControlProblemProblem: Drive pendulums to the pre-disturbance mode, that is,

minimize cumulative error (from desired trajectory) and

total control-input cost.

dtubdtxxuxf

t

t

t

t

2

00

2~),( =

=

=

=

+=

0),,( =uxxh

Minimize

Subject to

Three ControlSolutions:

C2

C-Net

C1 A centralized, nonlinear optimization

package that solve the stat. opt. prob.

(P).

A centralized, feedback linearization

controller.

A collaborative net, with one agent at each

pendulum, that solves {(Pm)}.

C Net and C1: Experimental

8/2/2019 cdna_cdc99

30/36

C-Net and C1: ExperimentalSet-upGoal:

Scenarios:

2-Pendulum Forest

Evaluate the loss in quality of the Collaborative Net solution.

Set-up: C-Nets and C1s restore synchronous mode of pendulums.

At each sample time t,

1.) solve the network of subproblems, {(Pm)}, with the C-Net,

2.) record the obj-function evaluation of the C-Net, F(C-Net),

3.) solve the static optimization problem, (P), with C1, and

4.) record the obj-function evaluation of C1, F(C1).

Output Data: A list of obj-function-evaluation pairs [F(C-Net),F(C1)].

Place pendulums in a line to form forests of 2 to 9 pendulums.

3-Pendulum Forest

Add 1

Pend.

8/2/2019 cdna_cdc99

31/36

C-Net and C1: Results

C-Net Excess:

F(C-Net) is the obj-function evaluation attained by the C-Net.

F(C1) is the obj-function evaluation attained by controller C1.

The difference in quality between the C-Net and C1 solutions.

C-Net excess = [F(C-Net) F(C1)] / F(C1)

C-Net Penalty: The mean value of the C-Net excess.

C-NetPe

nalty(%)

Number of Pendulums

C-Net penalty is low

C Net and C2: Experimental

8/2/2019 cdna_cdc99

32/36

C-Net and C2: ExperimentalSet-upGoal:

Scenario:

Evaluate the performance of the C-Net and the feedback

linearization controller, C2, a traditional control technique.

Set-up: C-Net and C2 restore synchronous mode of pendulums.

Output Data: The cumulative error and input-cost, f(x,u), for the C-Net & C2.

A forest with 9 pendulums placed in grid.

8/2/2019 cdna_cdc99

33/36

C-Net and C2: Results

dtubdtxxuxf

t

t

t

t

2

00

2~),(

=

=

=

=+=Objective:

Control-Input

Cost (b)

Objective Function Evaluation: f(x,u)

C2 (feedback lin) C-Net

10e-4 9.56 11.89

10e-3 10.49 12.32

10e-2 17.05 16.00

10e-1 82.64 32.07

The lower the f(x,u),

the better the solution

Minimize

C-Net performance

improves

C Net and C2: Trajectory of

8/2/2019 cdna_cdc99

34/36

C-Net and C2: Trajectory ofPendulumsPendulums under control of

C2 (feedback linearization)

Pendulums under control of

the C-Net

C2 immediately drives

pendulums to the

desired trajectory.

The C-Net waits until itbecomes cheaper to

drive pendulums.

8/2/2019 cdna_cdc99

35/36

Conclusion

The experiments show that C-Nets are promising.

Current research effort:

s Development of collaboration protocols that allow agents

to work asynchronously and in parallel, at their own speed.

- Use of safety margins to guarantee feasibility, and

foster effective work between slow and fast agents.

s A taxonomy of collaboration protocols.

What else have we done?

s Employed C-Nets to recover synchronous operation of generators

in power networks IEEE-14, -30, -57.

s Preliminary work on the decomposition of (P) into {(Pm)}:

- Models and algorithms to specify neighborhood perception.

8/2/2019 cdna_cdc99

36/36

Hybrid Control Strategies

PLANT

C1

C1

Cn

M1

M1

Mn

Decision

Module

controllers

performance

monitors

u yu1

u1

un