A Thermal grid coordinated by a Multi Agent Energy Management System

Olaf van Pruissen, Vincent Kamphuis, Armin van der Togt and Ewoud Werkman, Member, IEEE Technical Sciences

TNO The Netherlands

olaf.vanpruissen@tno.nl

Abstract—In the near future an increase of both thermal grids and sustainable suppliers of heat with intermittency behavior, connected to these heat grids, is expected. For smart operation this challenges the current centralized management systems. To deal with this and to optimize cost and energy efficiency a multi-agent approach is proposed, which to a certain extent is similar to agent systems developed for and tested in electricity grids. This study describes the physical and technical properties of a P2P solution which is based on experiences in both electricity grids and climate systems for buildings. Some vital issues of this approach are discussed and technical directions chosen are substantiated. The P2P system called HeatMatcher is tested in simulations with the heat network designed and run in a Matlab/Simulink environment. Results of a field test in a small thermal grid in an apartment complex in the Netherlands are presented.

Index Terms-- Multi-agent systems, Autonomous agents, Smart thermal grids, Power demand, Power supplies, Heating, Heat pumps, Field test, Computer Simulation.

I. INTRODUCTION The interest in thermal grids to distribute heat in cities is

growing [1]. It’s authors envisage that the share of district heating in Europe which is now 12% will increase to 20% in 2020 and 50% in 2050. This may lead to an improvement of energy efficiency for heating buildings of about 40%. Also the introduction of renewable energy generators with intermittent behavior as part of the infrastructure in residential areas and buildings will increase.

The heat will be provided by CHP’s, heat pumps, solar collectors and geothermal heat, each having certain specific constraints. The aquifer to which the heat pumps are connected should be kept in balance [2,3]. Cost efficient running of the geothermal unit may require the availability of heat storage units above ground, however storage in the aquifer is another interesting opportunity.

Since centralized management systems have problems with optimizing all goals in the control of larger and more complex building structures, multi-agent systems are proposed of being more capable of handling this [2-5]. In [3] the PowerMatcher [6] was studied as an example of such a

decentralized control system. During several field tests, coordination of renewable and fossil fuel based devices using the PowerMatcher has shown the ability to coordinate and control the electricity grid [2], for the benefit of different stakeholders involved in several business cases.

Because of both several physical differences between heat and electricity grids as well as certain ICT requirements [7] a new P2P solution was developed. This study describes the considerations which led to this system and the first results obtained in simulations and field tests.

II. THERMAL GRID DESCRIPTION

A. The thermal grid The thermal grid where the P2P Multi-agent system is

deployed, is a small district heating network supplying heat to 79 apartments at two different floor levels. The network consists of a double pipe system where both water at a temperature of about 45 °C for room heating and domestic hot water at around 60 °C flows. Both circuits are separated. The supply of domestic hot water consists of a solar collector with a total surface of 81 m2 and two gas fired boilers. The supply of the room heating (Fig. 1) consists of a buffer of 1000 L connected to a heat pump with a power of 106 kW and two gas fired boilers with a power of 115 kW each. The heat pump is connected to a borehole and can be operated at two stages of equal power by two parallel condensers. A plenum is present to distribute water in case the flow to the houses for floor heating is less than the flow from the supply section to prevent undesired pressure build up. The purpose of the test performed is 1) to investigate whether a multi-agent system can manage the district heating properly and 2) to improve energy efficiency. The latter may result from a) decreasing the share of delivery by the gas boilers by optimally loading the heat pump buffer and b) follow a strategy to achieve a better COP for the heat pump.

In general several other optimisations are possible. In a follow-up field test at another location both circuits for room heating and domestic hot water will be interconnected: a heat buffer connected to a solar collector may deliver heat to the heat pump buffer and vice versa. So the heat pump may deliver water for domestic hot water in the winter when it’s

This work was in part supported by EFRO under contract 21N.011

1

978-1-4799-2984-9/13/$31.00 ©2013 IEEE

2013 4th IEEE PES Innovative Smart Grid Technologies Europe (ISGT Europe), October 6-9, Copenhagen

COP is high and the solar collector may deliver heat in spring and autumn for room heating. With such optimisation abilities a more thorough test of energy efficiency gain with the multi-agent climate system is possible.

B. Maintaining the Integrity of the Specifications The system is managed by a Priva system [8]. This is a

centralized building management system, very commonly used in the Netherlands, especially in the horticulture. The Multi-agent system interfaces Priva through Insiteview, a product developed by Kropman BV, because Priva’s protocols are proprietary.

The performance with respect to the demand side of the system and user satisfaction could be evaluated based on two different kind of feedbacks. First telephone calls were recorded of dissatisfied inhabitants. Second the return temperature of the thermal grid was monitored and it was secured that this temperature didn’t exceed beyond a certain bandwidth.

Two practical points should be mentioned. First the flow of water to the apartments may be less than the flow through the buffer. The installation is designed that operation is still safe and secure on such an occasion, by avoiding build up pressure in the installation. Then part of the water mass just circulates; water from the top of the buffer is pumped backwards through the plenum and re-enters the buffer at the bottom. When also a gas burner is turned on with correspondingly extra water flow, then the hot water flow may even hardly reach the apartments directly: effectively and undesirably the heat pump buffer is filled with water from the gas boiler. Second, temperatures measured by either temperature sensors or meat meters at the same point may sometimes differ by 2 degrees. This is unfortunately common for custom installation systems [5]. In some cases this may hamper the energy efficient operation of the system.

III. MULTIAGENT SYSTEMS AND ELECTRONIC MARKETS To run a smart thermal grid the PowerMatcher [9] was

considered. The PowerMatcher is a general purpose coordination mechanism for balancing supply and demand in electricity network [6]. This technique implements supply and demand matching (SDM) using a multi-agent systems and market-based control approach. The interactions of individual agents in multi-agent systems are made more efficient by using electronic markets, which provide a framework for distributed decision making based on microeconomics.

It was decided that because of several reasons a different market based system should be developed. One such reason is that for heat, power is not the single parameter but it is defined by both temperature and flow. It should however not yet be concluded that PowerMatcher is not capable of managing heat flows in some general or simple structures.

The new solution is based on P2P communication where negotiation involves energy rather than power, and duration of contract. Before this is shortly described some of the fundamental physical properties and parameters involved in the HeatMatcher bidding strategy are given.

Fig. 1. Room heating supply system of small district heating Durghorst.

IV. THE PHYSICAL DESIGN PARAMETERS To create sound bids and offers in the HeatMatcher each

agent in the system should be able to calculate either the amount of energy or power, dependent on flow and temperature. Taking as an example the actual heat demand of the apartments qHDactual is calculated, using a heating curve, which is very commonly used in installation technology. In a heating curve the preferred temperature THC of the water supplying heat to the apartments is calculated from the outdoor temperature by a heuristically created graph. It can be derived that the heat demand qHDactual with ρ, the density of water, Cp the heat capacity of water and FrHD the flow rate of water and THd,out the return temperature of water flowing from the apartments, is:

qHDactual = ρ * Cp * FrHD * (THC – THD,out ) (1)

This basic equation (from physical point of view) for the calculation of heat is used by several agents of the HeatMatcher. So in the case of a buffer with the hottest water at the top of the buffer Tbuffer, top the amount of heat it would supply in a certain period of time is equally defined and proportional to Tbuffer, top – Tentering , where Tentering is the temperature of the water entering the buffer (at the bottom). For simplicity stratification of the buffer is neglected. In the field test temperatures were measured at the bottom, middle and top of the buffer and depending on the strategy also the average temperature can be used.

The HeatMatcher is designed for flexible demand. So the example given above with a heating curve is primarily used for its simplicity. However in the field test this method was used as individual control and measurement of demanded was not possible due to technical limitations and privacy issues.

In the case of devices supplying heat like a gas heater or a heat pump, the heat is simply given by the power of the device times the efficiency.

2

V. ICT ARCHITECTURE A P2P Market in a way comparable to [7] with market

negotiating phases on top of Hazelcast [10] was developed. The negotiating phase consists of two consecutive market rounds, a first one on a time scale of about 30 minutes and another one on a limited time scale of about 5 minutes. The first phase is meant for devices with a certain minimal operation and anti-commuting time and better performance and efficiency, such as a heat pump. All devices bid in this market round. The next phase is just meant for devices where bidding on a short time scale is such that it does not affect its operational behavior. A schematic view of the ICT architecture is shown in Figure 3.

Fig. 3. A view of the ICT architecture regarding the P2P agent representing a buffer. The Component interface ensures that communication from Java is possible between either the Matlab Simulink environment for Simulink or the Priva management system at the thermal grid of the test site.

Hazelcast written in Java serves as a platform taking care of the distributed communication typical for a P2P market [10]. The BufferAgent’s interface allows communication with either the simulation environment build in Matlab Simulink or the Priva control system.

VI. RESULTS To test the P2P market system (HeatMatcher), first

simulations were run with data representative for Durghorst during a whole year. Data for this simulation were supplied by ZONEnergie, the owner of the heating system. First the operation of the system was tested in just the Matlab Simulink environment. The Priva control principles were included by a separate component. The performance of the system was tested and the energy efficiency was analyzed. In the second

phase the HeatMatcher was connected to the Matlab Simulink environment and the Priva component omitted. By this procedure major bugs in the software were detected, solved, undesired behavior eliminated and by adding extra functionality the performance increased. It was tested and confirmed that the desired energy demand and entrance temperature to the houses was realized. Although the site allows little improvement of the energy efficiency due to the construction of the installation, even some increase of the energy efficiency was observed of about 2%.

Two field test were conducted at the site during March and April 2013. An example of the results is shown in Figure 4. Fig. 4 shows the performance of the P2P multi agent controlled management supply system during a cold night with outdoor temperature reaching -3 ˚C. During the night from about T1 = 23:30 h both stages of the heat pump are turned on until almost T2 = 9:30 h (green line). Gas fired boiler 2 (blue line) turns also on at T1 with increasing power until 03:00 h and then with decreasing power until T2 . At some occasions also gas fired boiler 1 (red line) turns on when the heat supply of both the heat pump and gas heater 2 appears to be insufficient. Until T1 stage 1 of the heat pump was turned off three times. Stage 2 of the heat pump was turned off 5 times. At 5 occasions gas boiler 2 delivered some energy. A similar behavior is observed after T2. It was confirmed that the energy supplied to the system was equal to the energy demanded by the system. The second field test was run for six consecutive days.

The heat pump supplies most part of the energy that is demanded. As the energy supplied is sufficient, considering the running time of three and another field test of five consecutive days and the absence of telephone calls of dissatisfied users, it is concluded that on general the system

Fig. 4. Operation of a multi agent controlled management system for room heating during a cold night. The stage of the heat pump is either 2, 1, or 0 (meaning “off”).

operates according to design.

The energy efficiency can be improved by avoiding turning on gas heater 2. Such an operation however was also observed during control of the Priva system. It was observed that these occasions occurred when the flow from the buffer was larger than the flow to the houses, resulting in a hot water flow to the bottom of the buffer, raising the buffer bottom temperature. So the buffer agent concludes that the load of the

3

buffer was insufficient and the system decides that an extra gas heater needs to be turned on. It is expected that this can be improved by either a higher preferred buffer temperature Tpref or by varying Tpref during the day. During this test Tpref was set at 41 ˚C, a better choice would have been 44 ˚C as the temperature derived from the heating curve during the test was 42 ˚C most of the time.

Fig 5 shows another performance test of the system operating for three consecutive days at slightly higher outdoor temperatures; decreasing to 0, 3, and 1 ˚C at each consecutive night. Gas heater 2 is not used and gas heater 1 turns on mainly during the night at low outdoor temperatures. The heat pump shows a similar behavior as shown in Fig. 4.

Fig. 6 shows for almost the same period of time the heat that is demanded by the apartments and the heat that is actually delivered. It can be seen that generally as much heat is delivered as demanded. During the day some more heat is supplied. This is caused by a required minimum of revolutions of 30% for the pumping device connected to the buffer. The agent’s intelligence was designed such that at a demand corresponding to a pumping speed of above 15%, the pump would be turned on at 30%, below 15%, it would be turned off. Apparently during the daytime testing period the demand stayed at or above a corresponding 15%, causing more heat to be delivered. At high demand during the night slightly less power is supplied. This was caused by a difference in sensor

Fig. 5. Operation of a multi agent controlled management system for room heating for three consecutive days. The stage of the heat pump is either 2, 1, or 0 (meaning “off”).

Fig. 6. The demanded and actually delivered heat to the apartments in same period of time.

4

readings. At high flow the temperature measured with a heat sensor at the water returning from the apartments was 2 ˚C higher than the temperature measured with a temperature sensor entering the buffer. This temperature drop was too large to be explained by loss of heat in the pipes. During an entire field test of 6 days no telephone calls were received from the inhabitants with complaints about their comfort.

VII. CONCLUSIONS From the simulation and field test results it is concluded

that with a multi-agent P2P market based system it is possible to control small district heat networks without infringement of user comfort. The approach with first just simulating, than simulating with the HeatMatcher and then running in a field test appeared to be fruitful, as system failure did not occur; the system was successfully deployed and in operation within one day at the test site.

The plenum in the installation caused unexpected water flows and proved to be a challenge for the P2P system. However, it was capable to deliver the right amount of heat without any disturbance in the delivery system.

Several improvements can be made. At some occasions the heat pump turns the second stage off and in the next step both the heat pump and a gas heater turn on. This counteracts the energy efficiency and influences the total systems behavior leading to some instability as can be observed at the start of the night of April 7th. This was most probable caused by a software bug leading to an unnecessary long minimum runtime of stage 1 of the heat pump.

A forecasting component is planned to be part of the HeatMatcher. This will communicate and inform each buffer in the system what it’s preferred temperature should be. At the start of this test a temperature was chosen as a best guess, however the system’s performance is rather sensitive to this value.

Demand response of the system could not be tested. To improve the energy performance it is important to shift demand in time. To carry this out, information is needed from individual houses with respect to at least indoor temperature and setpoint. Hence the traditional heating curve can be replaced by a more sophisticated way of yielding demand.

A major benefit of the HeatMatcher should be the improvement of the overall energy efficiency. The installation that was tested, leaves little room for showing this. In the next field test a solar collector will also be controlled, where it can supply a buffer and next it can be decided to transfer the heat in this buffer to either a buffer for domestic heating or the buffer for the room heating. Depending on the sizes of the buffer and a good prediction of the forecasting tool with

respect to solar irradiation this can improve the performance of the installation as intended by the development of the HeatMatcher.

Control of hybrid grids, e.g. heat and electricity should be another important step moving forward. The intention is to include part of this work in the integrated control of heat and electricity management as described in [11].

ACKNOWLEDGMENT The authors wish to thank Huib Visser of ZONEnergie for

preparation of the test site at Durghorst. Wim Kornaat, Jan Ewout Scholten and Hans Phaff for developing the Simulink model, Klaas Visscher for assisting them and Paul Booij for developing the Java HeatMatcher Matlab interface, all from TNO.

REFERENCES [1] D. Connoly, B.V. Mathiesen, P.A. Ostergaard, B. Moller, S. Nielsen,

H. Lund, U. Persson, D. Nilsson, S. Werner, D. Trier. Heat Roadmap 2050, Aalborg, HalmStad, PlanEnergi,2012

[2] J.K. Kok, B. Roossien, P.A. MacDougall, O.P. van Pruissen, G. Venekamp, I.G. Kamphuis, J.A.W Laarakkers, and C.J. Warmer, Dynamic Pricing by Scalable Energy Management Systems - Field Experiences and Simulation Results using PowerMatcher. IEEE Power and Energy Society General Meeting 2012, IEEE, 2012.

[3] O.P. van Pruissen, I.G. Kamphuis, Multi agent building study on the control of the energy balance of an aquifer. Environment, 2011 IEECB'10 - Improving Energy Efficiency in Commercial Buildings, Frankfurt, Germany, 13-14 April 2010

[4] T.W. Sandholm, T.W.: Distributional rational decision making. In: Multiagent Systems: A Modern Approach to Distributed Artificial Intelligence, pp201--258, The MIT Press, Cambridge Massachusetts, 1999.

[5] B.A. Huberman, S. Clearwater, A multi-agent system for controlling building environments, Proc. of the first international conference on multi-agent systems, pp 171-176, 1995, can be downloaded at http://portal.acm.org

[6] J.K. Kok, M.J.J. Scheepers, I.G. Kamphuis, “Intelligence in electricity networks for embedding renewables and distributed generation,” in Intelligent Infrastructures, R.R. Negenborn, Z. Lukszo, J. Hellendoorn (Eds) Springer, Dordrecht Heidelberg London New York, 2010, pp 179--209.

[7] D. Hausheer, B.Stiller, Decentralized Auction based Pricing with Peermart, IEEE International symposium on Integrated Network, 2005

[8] Priva, http://ww.priva.nl/ [9] J.K. Kok, C.J. Warmer, I.G. Kamphuis, “PowerMatcher : multiagent

control in the electricity infrastructure”, Fourth international conference on Autonomous Agents & Multi-Agent Systems, AAMAS ’05, July 25-29, 2005, Utrecht, Netherlands.

[10] Hazelcast, http://www.hazelcast.com/ [11] P. Booij, V. Kamphuis, O.P. van Pruissen, C. Warmer. Multi-agent

control for integrated heat and electricity management in residential districts, (ATES@AAMAS’13) 2013, 6-10 May, Minnesota, USA

5