Www.iesd.dmu.ac.uk/~cascade Institute of Energy and Sustainable Development 1 CASCADE Author Date...

www.iesd.dmu.ac.uk/~cascadeInstitute of Energy and Sustainable Development

1

CASCADE

Author

Date

Complex Adaptive Systems, Cognitive Agents and Distributed Energy

BEHAVIOUR AND LEARNING IN AN AGENT BASED MODEL OF THE SMART GRID

Richard Snape

22nd March 2011


2

CONTENT

• Context

• CASCADE model overview

• Agent description

• Behaviour and learning description

• Prototype implementation and results

• Conclusion and questions


3

THE TEAM

Prof. M. Rylatt – PIDr. P. Boait – CIDr. M. Lemon – CIDr. V. PakkaB. Mahdavi ArdestaniJ. R. Snape – PhD student

Prof. P. Allen – CIProf. M. Savill – CIDr. L. VargaDr. M. Strathern


4

ELECTRICITY NETWORK

Heterogeneous actors

Multiple posited future scenarios

Multiple (nested?) physical scales

Multiple economic scales

Multiple temporal scales

Multiple networks

Highly complex problem domain !!


5

SMART GRID

€, £, $

Bidirectional power flow

Bidirectional information flow

Image: Copyright 2006 by Hawaiian Electric Company, Inc., all rights reserved


6

WHY MODEL THE SMART GRID?“The UK Government’s low carbon strategy and Ofgem’s Low Carbon Network Fund and RPI-X@20 project are the

key context for the development of smart grid”ENSG,

A Smart Grid Routemap, 2010, p. 6

[The Routemap requires us to] “Demonstrate new business and revenue models and associated regulatory and

commercial frameworks that support demand reduction and better energy management”

ibid, p. 14

[including] “Development of new customer products and services and ESCO / VPP etc. business models – building

on existing models and relationships”ibid, p.17


7

NOT JUST SMART METERS!

Smart meters

Smart grid


8

MODEL OVERVIEW


9

COMPONENTS AND INFORMATION FLOWS

Local physical model

Local aggregator agent (can be ESCO, retail supplier, virtual island)

Prosumer agent (can be generator, load, or both)

Agent based model

Forward pricebroadcast

Prosumer response

Prosumer energy flows

Distribution costs and constraints

Social interaction between prosumers

Grid & wholesale market agent

Agent environ-ment (policy, weather, etc)


10

MODEL DIAGRAM

Context Model

Environmental (contextual) variables

E.g.

• Weather• Policy• Regulation• CO2 price

Market

Social network (household prosumers 4 -> N)

Aggregator 1

Aggregator 2

Aggregator M

Prosumer 1

Prosumer 2

Prosumer 3

Prosumer N+1

Prosumer N+2

...

Output – overall consumption

...


11

AGENT DESCRIPTION


12

PROSUMER

Prosumers

Pure consumer Pure generator

CHP

Net generatorNet consumer

Pmax -Pmax0


13

AGGREGATOR

•Behaviour

•Objectives

•Learning•Decision

•£/kWh; or•CO2/kWh; or•Exergy loss;

or•?

•kWh/30 min•kWh/5 min•?


14

AGENT ANALOGUES

Prosumers may represent:

•Automata e.g. A household smart controller

•Individuals e.g. A householder

•Collectives e.g. Generating companies

Aggregators generally represent:

•Collectives e.g. firms, communities, housing associations...


15

POSSIBLE MODEL CONFIGURATIONS

Business as usual– Aggregators = Utility companies– Prosumers = Power stations, wind farms, households

Energy Service Company (ESCo) model– Aggregators = ESCos– Prosumers = as above plus meso level generation e.g.

Community wind farms or district level CHP

Microgrids– Aggregators = local scale agents with objective net

demand of zero– Prosumers = as above


16

BEHAVIOUR AND LEARNING


17

BEHAVIOUR AND LEARNING ALGORITHMS

Psychology based

Optimisation based

Organisation theory based


18

PSYCHOLOGY BASED LEARNING 1

Does the name Pavlov ring a bell?References

•Sutton & Barto (1998)•Roth & Erev (1995)


19

PSYCHOLOGY BASED LEARNING 2

• More elaborate models

References – theory•Triandis, H.C. (1977)•Bandura, A. (1986)•Ajzen, I. (1991)•Stern, P.C. (2000)

References – application in electricity

•Thøgersen & Grønhøj (2010)

•Zhang & Nuttall (2011)

Society

Attitude

Norms...


20

OPTIMISATION BASED LEARNING


21

ORGANISATIONAL LEARNING

References•Senge (1990) •March (1991)•Argyris & Schon (1996)

Explore vs exploit

Cyclical causality

Reinforcing vs balancing

Organisational rules

Drawn from Nancy Dixon’s The Organisational Learning Cycle - How we can learn collectively, McGraw-Hill, 1994.


22

PROTOTYPE IMPLEMENTATION AND RESULTS


23

PROTOTYPE IMPLEMENTATION

Selected RePast Simphony toolbox– Java, fully open source, well used, powerful

Extensibility

Scalability

Interface between scales – temporal and spatial


24

PROTOTYPE MODEL

Stylised test case– 1000 household prosumers – 1 wind turbine– 1 aggregator

Only household prosumers learnAggregator transmits price signal to its

prosumers to encourage smoothing of demand


25

HOUSEHOLD PROSUMER BEHAVIOURS TESTED IN

PROTOTYPE1. No adaptation2. Householders manually respond to signal

by reducing up to 15% of demand3. Householders learn to adopt automated

smart device which then time shifts up to 15% of demand within day

4. Householders learn to adopt automated smart device which then totally smoothes demand within day (stylised edge case)


26

PROTOTYPE RESULTS (1 YEAR RUN)

Scenario 1 Scenario 2

Scenario 3 Scenario 4


27

SUMMARY

ABM suited to the highly complex electricity system under transition to smart grid

Modelled using prosumer and aggregator abstraction

Behaviour and learning algorithms crucial to model behaviour

Challenges remain...


28

Any questions or comments?

For further information seewww.iesd.dmu.ac.uk/~cascade

http://www.iesd.dmu.ac.uk/~cascade


29

REFERENCESENSG (2010) Electricity Networks Strategy Group

- A Smart Grid Routemap. United KingdomRoth, A.E. & Erev, I. (1995) Learning in extensive-

form games: Experimental data and simple dynamic models in the intermediate term*. Games and Economic Behavior, 8 (1), pp.164-212.

Sutton, R. & Barto, A. (1998) Reinforcement Learning: An Introduction. Cambridge, MA, MIT Press.

Thøgersen, J. & Grønhøj, A. (2010) Electricity saving in households--A social cognitive approach. Energy Policy, 38 (12), pp.7732-7743.

Stern, P.C. (2000) New environmental theories: toward a coherent theory of environmentally significant behavior. Journal of social issues, 56 (3), pp.407-424.

Triandis, H.C. (1977) Interpersonal behavior. Monterey, Brooks/Cole Pub. Co.

Argyris, C. & Schon, D.A. (1996) Organizational learning II: Theory, method and practice. Reading MA, Addison-Wellesley

March, J.G. (1991) Exploration and exploitation in organizational learning. Organization science, 2 (1), pp.71-87.

Senge, P.M. (1990) The fifth discipline: Mastering the five practices of the learning organization. New York.

Ajzen, I. (1991) The theory of planned behavior. Organizational behavior and human decision processes, 50 (2), pp.179-211.

Bandura, A. (1986) Social foundations of thought and action. Stanford University, Prentice-Hall, Inc., Eaglewood Cliffs, New Jersey.

Zhang, T. & Nuttall, W.J. (2011) Evaluating Government's Policies on Promoting Smart Metering Diffusion in Retail Electricity Markets via Agent-Based Simulation. Journal of Product Innovation Management, 28 (2), pp.169-186.

Vriend, N.J. (2000) An illustration of the essential difference between individual and social learning, and its consequences for computational analyses. Journal of Economic Dynamics and Control, 24 (1), pp.1-19.

Www.iesd.dmu.ac.uk/~cascade Institute of Energy and Sustainable Development 1 CASCADE Author Date...

Documents

Transcript of Www.iesd.dmu.ac.uk/~cascade Institute of Energy and Sustainable Development 1 CASCADE Author Date...