19th November 2008

Agent-Based Decentralised Control of Complex Distributed Systems

 Alex Rogers

School of Electronics and Computer ScienceUniversity of Southampton

acr@ecs.soton.ac.ukhttp://users.ecs.soton.ac.uk/acr/

19th November 2008

Contents• Agent-Based Decentralised Control• Cooperative Systems

– Local Message Passing Algorithms– Max-sum algorithm– Graph Colouring– Wide Area Surveillance Scenario

• Competitive Systems– Game Theory– Mechanism Design– Eliciting Effort in Open Information Systems

• Decentralised Energy Systems

19th November 2008

Electronics and Computer Science• 5* for Electrical and Electronic

Engineering• 5* for Computer Science

• 100 academic staff• 36 professors

• 150 research fellows• 250 PhD students

• Research grant income: • £15 million per annum• £10 million from UK Research Councils

19th November 2008

Intelligence, Multimedia and Agents Research Group

Design and application of computing systems for complex information and knowledge processing tasks

• Agent-Based Computing• Digital Libraries• Decentralised Information Systems• E-Business Technologies• Grid and Distributed Computing• Human Computer Interaction• Web Science • Knowledge Technologies • Trust and Provenance

19th November 2008

Contents• Agent-Based Decentralised Control• Cooperative Systems

– Local Message Passing Algorithms– Max-sum algorithm– Graph Colouring– Wide Area Surveillance Scenario

• Competitive Systems– Game Theory– Mechanism Design– Eliciting Effort in Open Information Systems

• Decentralised Energy Systems

19th November 2008

Agent-Based Decentralised Control

Agents

• Multiple conflicting goals and objectives• Discrete set of possible actions

19th November 2008

Agent-Based Decentralised Control

Sensors

• Multiple conflicting goals and objectives• Discrete set of possible actions

19th November 2008

Agent-Based Decentralised Control

Agents

• Multiple conflicting goals and objectives• Discrete set of possible actions• Some locality of interaction

19th November 2008

Agent-Based Decentralised Control

Agents

Maximise Social Welfare:• Multiple conflicting goals and objectives• Discrete set of possible actions• Some locality of interaction

19th November 2008

Agent-Based Decentralised Control• Cooperative Systems

– All agents represent a single stakeholder– We have access to these agents (closed system)– We can design the strategies that the agents adopt and the

mechanisms by which they interact• Competitive Systems

– Agents represent multiple stakeholders– We can not directly influence the strategies of the agents

(open system)– We can only design the protocols and mechanisms by which

they interact

19th November 2008

Cooperative Systems

Agents

Central point of controlDecentralised control and coordination through local computation and message passing.• Speed of convergence, guarantees of optimality,

communication overhead, computability

No direct communication Solution scales poorly Central point of failure

19th November 2008

Landscape of Algorithms

Complete Algorithms

DPOPOptAPOADOPT

Communication Cost

Optimality

Iterative Algorithms

Best Response (BR)Distributed Stochastic

Algorithm (DSA) Fictitious Play (FP)

Greedy Heuristic

Algorithms

Message Passing

Algorithms

Sum-ProductAlgorithm

19th November 2008

Sum-Product Algorithm

Variable nodes

Function nodes

Factor Graph

A simple transformation:

allows us to use the same algorithms to maximise social welfare:

Find approximate solutions to global optimisation through local computation and message passing:

19th November 2008

Graph Colouring

Agentfunction / utility

variable / state

Graph Colouring Problem Equivalent Factor Graph

19th November 2008

Graph Colouring

Equivalent Factor GraphUtility Function

19th November 2008

Max-Sum Calculations

Variable to Function: Information aggregation

Function to Variable: Marginal Maximisation

Decision:Choose state that maximises

sum of all messages

19th November 2008

Graph Colouring

19th November 2008

Optimality

19th November 2008

Communication Cost

19th November 2008

Robustness to Message Loss

19th November 2008

Hardware Implementation

19th November 2008

Wide Area Surveillance Scenario

Dense deployment of sensors to detect pedestrian and vehicle activity within an urban environment.

Unattended Ground Sensor

19th November 2008

Energy Constrained Sensors

Maximise event detection whilst using energy constrained sensors:– Use sense/sleep duty cycles

to maximise network lifetime of maintain energy neutral operation.

– Coordinate sensors with overlapping sensing fields.

time

duty cycle

time

duty cycle

19th November 2008

Energy-Aware Sensor Networks

19th November 2008

Future Work• Continuous action spaces

– Max-sum calculations are not limited to discrete action space

– Can we perform the standard max-sum operators on continuous functions in a computationally efficient manner?

• Bounded Solutions– Max-sum is optimal on tree and limited

proofs of convergence exist for cyclic graphs– Can we construct a tree from the original

cyclic graph and calculate an lower bound on the solution quality?

19th November 2008

Contents• Agent-Based Decentralised Control• Cooperative Systems

– Local Message Passing Algorithms– Max-sum algorithm– Graph Colouring– Wide Area Surveillance Scenario

• Competitive Systems– Game Theory– Mechanism Design– Eliciting Effort in Open Information Systems

• Decentralised Energy Systems

19th November 2008

Competitive Systems• Controlling open competitive systems is much

more difficult– Global credit crisis

• Key challenges– Understanding the emerging macroscopic properties

of a system of selfish competitive agents• GAME THEORY

– Designing protocols and ‘rules of the game’ such that these macroscopic properties are desirable

• COMPUTATIONAL MECHANISM DESIGN

19th November 2008

Game Theory• For a given ‘game’

– What action should a rational player take?– What is the equilibrium action of all players?

• Nash equilibrium

A Beautiful Mind: Genius and Schizophrenia in the Life of John NashSylvia Nasar Faber and Faber

19th November 2008

Nash Equilibrium

• Two strategies s1 and s2 are in Nash equilibrium if:

1. under the assumption that agent i plays s1, agent j can do no better than play s2; and

2. under the assumption that agent j plays s2, agent i can do no better than play s1.

1. Neither agent has any incentive to deviate from a Nash equilibrium

19th November 2008

Nash Equilibrium

Column Player

LEFT MIDDLE RIGHT

Row

Pla

yer

UP 4 , 3 5 , 1 6 , 2

MIDDLE 2 , 1 8 , 4 3 , 6

DOWN 3 , 0 9 , 6 2 , 8

1

2

3

4

NE

19th November 2008

Computational Mechanism Design• Mechanism design concern the analysis and

design of systems in which the interactions between strategic, autonomous and rational agents leads to predictable global outcomes.– Design interactions to ensure the system has desirable and

predictable Nash equilibrium

• Computational mechanism design– Limited communication– Incomplete information– Bounded computation

19th November 2008

Nash Equilibrium

Column Player

LEFT MIDDLE RIGHT

Row

Pla

yer

UP 4 , 3 5 , 1 6 , 2

MIDDLE 2 , 1 8 , 4 3 , 6

DOWN 3 , 0 9 , 6 2 , 8

1

2

3

4

NE

16th September, 2004

16th September, 2004

19th November 2008

First Price AuctionDesirable properties• Efficiency

Allocation• Item assigned to

the highest bidder

Payment• Pay bid ( )

Bidding strategy• Shade bid• Bayes Nash

19th November 2008

Second Price (Vickrey) AuctionDesirable properties• Efficiency

Allocation• Item assigned to

the highest bidder

Payment• Pay second bid

Bidding strategy• Bid true valuation• Dominant strategy

19th November 2008

Open Information System• Information buyer requires a

prediction of an uncertain

• Tomorrow’s temperature

• Requires certain minimum precision or “certainty”

θ0

θ

c(θ)

θ

c(θ)

θ

c(θ)

• Identify cheapest provider• Make prediction of precision of at least θ0

• Truthfully report this prediction to buyer• Ensure provider’s utility is positive in

expectation

19th November 2008

Two Stage Mechanism• Two stage Mechanism:

1. Ask information producers to declare their costs2. Ask cheapest producer to make measurement and reward

him with a payment using a ‘strictly proper scoring rule’ calculated from the second lowest cost• Payment is made once the event is verified

• Desirable system wide properties– Dominant strategy to truthfully declare costs

• Information buyer can always identify cheapest supplier– Dominant strategy to commit effort and truthfully reveal

prediction

19th November 2008

Challenges• Solution concepts

– Mechanisms with dominant strategy solutions are rare– How do we automate the design process?

• Decentralised Mechanisms– Remove need for a central auctioneer

• Payment Free Mechanism– Non-transferable utility– Induce cooperative behaviour through reciprocity

• Iterated Prisoner’s Dilemma• Trust and reputation models• Match making mechanisms to pair producers and buyers

19th November 2008

Contents• Agent-Based Decentralised Control• Cooperative Systems

– Local Message Passing Algorithms– Max-sum algorithm– Graph Colouring– Wide Area Surveillance Scenario

• Competitive Systems– Game Theory– Mechanism Design– Eliciting Effort in Open Information Systems

• Decentralised Energy Systems

19th November 2008

Micro-CHP

Flywheel Storage

Wireless Sensors

Plug-in Hybrid

Appliances

2016 Zero Carbon Home

19th November 2008

Energy Exchange

19th November 2008

• How to coordinate energy use and make optimal energy trading decisions within the home to minimise energy consumption / costs?– Load management through smart appliances– Predicting load (occupancy, activity, weather conditions)– Understanding and learning thermal characteristics of home– Price prediction in external and local markets– Optimal use of storage devices– Optimal decisions to buy electricity / use CHP

Research Questions

19th November 2008

• What protocols and trading mechanisms generate desirable system wide properties?– Stable, predictable and low prices– Minimise CO2 emissions through flattening demand

One day

Research Questions

19th November 2008

Publications

Farinelli, A., Rogers, A., Petcu, A. and Jennings, N. R. (2008) Decentralised Coordination of Low-Power Embedded Devices using the Max-Sum Algorithm. In: Proceedings of the Seventh International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS 2008), pp. 639-646, Estoril, Portugal.

Papakonstantinou, A., Rogers, A., Gerding, E. and Jennings, N. (2008) A Truthful Two-Stage Mechanism for Eliciting Probabilistic Estimates with Unknown Costs. In: Proceedings of the Eighteenth European Conference on Artificial Intelligence (ECAI 2008), pp. 448-452, Patras, Greece.

R. K. Dash, N. R. Jennings, and D. C. Parkes. (2003) Computational Mechanism Design: A Call to Arms. IEEE Intelligent Systems, pages 40–47.

19th November 2008

Questions

Thank you for your attention.