Distributed Control Design for

District Heating Networks

Y.C.C. Putter
Master of Science Thesis

Delft Center for Systems and Control

Distributed Control Design for District
Heating Networks

Master of Science Thesis

For the degree of Master of Science in Systems and Control at Delft

University of Technology

Y.C.C. Putter

October 1, 2018

Faculty of Mechanical, Maritime and Materials Engineering (3mE) · Delft University of

Technology
The work in this thesis was supported by the MCS department of TNO. Their cooperation
is hereby gratefully acknowledged.

Copyright c Delft Center for Systems and Control (DCSC)

All rights reserved.
 Delft University of Technology
Department of
Delft Center for Systems and Control (DCSC)

The undersigned hereby certify that they have read and recommend to the Faculty of
Mechanical, Maritime and Materials Engineering (3mE) for acceptance a thesis
entitled
Distributed Control Design for District Heating Networks
by
Y.C.C. Putter
in partial fulfillment of the requirements for the degree of
Master of Science Systems and Control

Dated: October 1, 2018

Supervisor(s):
Dr.ir. Tamás Keviczky

Ir. Ewoud Werkman

Reader(s):
Dr.ir. Ivo Pothof

Ir. Eunice Herrera

Ir. Marlon Pijpelink

 Abstract

In The Netherlands, the current heat energy system accounts for 44% of the primary energy
usage and relies almost entirely on fossil fuels such as natural gas. To meet the Paris Climate
Agreement goals, 4th Generation District Heating (4GDH) networks are expected as sustain-
able heat energy system solution. The concept relies on optimally matching the heat energy
supply of sources such as waste heat, combined heat and power (CHP) plants and geothermal
energy, with the demand of consumers such as households or greenhouses, whilst using the
flexibility of buffers such as aquifer thermal energy storage (ATES) systems.
In this thesis, a cooperative multi-agent system (MAS) hierarchical model predictive con-
trol (HMPC) implementation is presented as smart controller for 4GDH networks, and as
alternative approach to improving TNO’s HeatMatcher (HM) algorithm with the proposed
algorithm of PowerMatcher (PM) that relies on locational marginal pricing (LMP). The
model predictive control (MPC) approach is chosen mainly due to the advantage that it op-
timizes over a prediction horizon or time span, instead of a single time step. This allows to
take into account heat energy demand predictions, time-based constraints, and the inherent
dynamic characteristics of 4GDH systems such as buffer flexibility and the variable time delay
present in the heat energy exchange.
The centralized model predictive control (CMPC) control problem is formulated as a de-
terministic, MAS, mixed-integer quadratic programming (MIQP) optimization problem and
is subsequently distributed based on the Optimal Exchange Problem formulation using the
alternating direction method of multipliers (ADMM). Hybrid system modelling theory is
applied to model the agents’ subsystems and a simplified heat energy exchange model with
constant time delay is assumed. The latter was chosen as decoupled thermal and hydraulic
equations proved to be non-linear in the valve positions and mass flow, iterative due to the
friction factor and the Reynolds number, and dependent on a variable spatial sampling to
accurately track the thermal propagation through the network.
The CMPC and HMPC algorithms are applied on an academic initial design case study to test
desired controller behaviour under perfect heat demand prediction, and on a more realistic
case study of the WarmCO2 heat grid involving 5 greenhouses with non-perfect heat demand
predictions during a summer and winter scenario.
The initial case study confirms that both algorithms perform as desired with the exception of a
small shortcoming of the hybrid modelling, and that they are similar in their optimal solutions
as expected. The same holds for the WarmCO2 case study. However, it also showcases that

Master of Science Thesis Y.C.C. Putter

ii Abstract

the deterministic optimization can become infeasible due to the time delay modelling and
non-perfect heat demand predictions, and that therefore a stochastic optimization approach
is preferred. Furthermore, good quality local optimal solutions of the NP-hard problem could
be found within a relatively short computing time limit, using the heuristic methods of the
Gurobi solver. And lastly, the importance of developing a non-cooperative MPC algorithm
to accurately represent the individual optimization goals of different stakeholders.

Y.C.C. Putter Master of Science Thesis

 Table of Contents

Abstract i

Acknowledgements xi

1 Introduction 1
1-1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1-1-1 Main Problems of Current Heat Energy System . . . . . . . . . . . . . . 2
1-1-2 Sustainable Solutions to Restructure Heat Energy System . . . . . . . . . 2
1-1-3 Overview of Current Heat Grids . . . . . . . . . . . . . . . . . . . . . . 3
1-1-4 Vision of Future Heat Grids . . . . . . . . . . . . . . . . . . . . . . . . . 4
1-2 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1-3 Related Work and Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1-4 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2 Smart Control Methods 9

2-1 Smart Control Design and Modelling Considerations for Heat Grids . . . . . . . . 9
2-1-1 Flexibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2-1-2 Multi-Agent System and Modelling . . . . . . . . . . . . . . . . . . . . . 11
2-1-3 Cooperative vs Non-Cooperative Agents . . . . . . . . . . . . . . . . . . 12
2-1-4 Centralized vs Non-Centralized Control . . . . . . . . . . . . . . . . . . . 13
2-2 HeatMatcher . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2-2-1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2-2-2 Operating Principle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2-3 Locational Marginal Pricing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2-3-1 Energy Flow Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2-3-2 Price Components and Constraints . . . . . . . . . . . . . . . . . . . . . 17
2-4 Model Predictive Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2-5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2-5-1 LMP Based on Electricity Flow Model . . . . . . . . . . . . . . . . . . . 22
2-5-2 Internal Suggestions for HM Improvements . . . . . . . . . . . . . . . . 22
2-5-3 Arguments for MPC approach . . . . . . . . . . . . . . . . . . . . . . . 23

Master of Science Thesis Y.C.C. Putter

iv Table of Contents

3 Initial MPC Design Case Study 25

3-1 Multi-Agent System Modelling . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3-1-1 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3-1-2 Simplified Energy Exchange Network . . . . . . . . . . . . . . . . . . . . 27
3-1-3 Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3-1-4 Producer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3-1-5 Consumer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3-1-6 Buffer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3-2 Optimization Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3-2-1 CMPC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3-2-2 HMPC with ADMM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3-3 Setup of Initial Design Case Scenario . . . . . . . . . . . . . . . . . . . . . . . . 42
3-4 Results and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3-4-1 Interpretation of Result Figures . . . . . . . . . . . . . . . . . . . . . . . 45
3-4-2 CMPC vs REF controller . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3-4-3 CMPC vs HMPC controller . . . . . . . . . . . . . . . . . . . . . . . . . 48
3-4-4 CMPC vs Time Delayed CMPC . . . . . . . . . . . . . . . . . . . . . . . 52
3-4-5 Time Delayed CMPC vs Time Delayed HMPC . . . . . . . . . . . . . . . 54

4 WarmCo Case Study 57

4-1 Overview of Heat Grid Characteristics . . . . . . . . . . . . . . . . . . . . . . . 58
4-2 Case Study Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
4-2-1 ProducerBuffer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
4-2-2 ConsumerBuffer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
4-3 Setup of WarmCo Case Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . 68
4-4 Results and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4-4-1 Infeasibility during Summer Scenario . . . . . . . . . . . . . . . . . . . . 71
4-4-2 Runtime Considerations and Observations . . . . . . . . . . . . . . . . . 72
4-4-3 WarmCo Agent Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

5 Conclusions and Recommendations 91

5-1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
5-1-1 4GDH . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
5-1-2 MPC vs HM and LMP Approach . . . . . . . . . . . . . . . . . . . . . . 91
5-1-3 Initial Design Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . 92
5-1-4 WarmCo Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
5-2 Recommendations for Future Work . . . . . . . . . . . . . . . . . . . . . . . . . 93
5-2-1 Network and Heat Demand Modelling . . . . . . . . . . . . . . . . . . . 93
5-2-2 Solving the MIQP Optimization Problem . . . . . . . . . . . . . . . . . . 94
5-2-3 Improving Characteristics of MPC Approach . . . . . . . . . . . . . . . . 94

A Demand Modelling 95

Y.C.C. Putter Master of Science Thesis

Table of Contents v

B Supply Modelling 99

C Network Modelling 101

C-1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
C-1-1 Assumptions for Pipe Flow and Water Properties . . . . . . . . . . . . . 101
C-1-2 Dimensionless Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
C-2 Hydraulic Modelling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
C-2-1 Total Head Loss . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
C-2-2 Pumps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
C-3 Thermal Modelling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
C-4 Modelling of District Heating Networks . . . . . . . . . . . . . . . . . . . . . . . 109

D Decomposition Methods 111

D-1 Primal and Dual Decomposition Examples . . . . . . . . . . . . . . . . . . . . . 111
D-2 Dual Decomposition Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
D-2-1 Primal Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
D-2-2 Lagrange Dual Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
D-2-3 Karush-Kuhn-Tucker conditions . . . . . . . . . . . . . . . . . . . . . . . 113
D-3 ADMM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

E Agent Constraint Matrices and Vectors 115

E-1 Agents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
E-1-1 Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
E-1-2 Producer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
E-1-3 Consumer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
E-1-4 Buffer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
E-1-5 ProducerBuffer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
E-1-6 ConsumerBuffer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
E-2 Global Coupling Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

Bibliography 123

Glossary 129
List of Acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
List of Symbols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130

Master of Science Thesis Y.C.C. Putter

vi Table of Contents

Y.C.C. Putter Master of Science Thesis

 List of Figures

1-1 Dutch statistics of current heat energy system . . . . . . . . . . . . . . . . . . . 2

1-2 Schematic of 3GDH system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1-3 Overview of district heating system generations . . . . . . . . . . . . . . . . . . 5

2-1 Overview of the energy flexibility interface (EFI) . . . . . . . . . . . . . . . . . . 10

2-2 Schematic of different control architectures . . . . . . . . . . . . . . . . . . . . 13
2-3 General HeatMatcher bidcurves . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2-4 Simple network topology with bidcurves . . . . . . . . . . . . . . . . . . . . . . 15
2-5 HeatMatcher bidcurves and global clearing price . . . . . . . . . . . . . . . . . . 16
2-6 HeatMatcher bidcurves and global clearing price with locational marginal prices . 18
2-7 Schematic of basic model predictive control trajectories . . . . . . . . . . . . . . 20
2-8 Schematic of basic model predictive controller . . . . . . . . . . . . . . . . . . . 20

3-1 Schematic of relevant variables for detailed pipe model . . . . . . . . . . . . . . 27

3-2 Schematic of simplified energy exchange model for pipe dynamics for a single edge e 28
3-3 Schematic topology of initial design case . . . . . . . . . . . . . . . . . . . . . . 42
3-4 Detailed agent topology of initial design case . . . . . . . . . . . . . . . . . . . 43
3-5 CMPC vs reference CMPC agent results for initial design case study . . . . . . . 46
3-6 CMPC vs reference CMPC cost results for initial design case study . . . . . . . . 47
3-7 CMPC vs HMPC agent results for initial design case study . . . . . . . . . . . . 49
3-8 ADMM dual variable results at k=2 . . . . . . . . . . . . . . . . . . . . . . . . 50
3-9 ADMM dual variable results at k=7 . . . . . . . . . . . . . . . . . . . . . . . . 51
3-10 CMPC vs CMPC with time delay agent results for initial design case study . . . . 53
3-11 CMPC vs CMPC with time delay cost results for initial design case study . . . . 54
3-12 CMPC vs HMPC agent results for initial design case study with time delay . . . 55
3-13 ADMM dual variables results at k=19 for initial design case study with time delay 56

Master of Science Thesis Y.C.C. Putter

viii List of Figures

4-1 Schematic of the port-zone of Biopark Terneuzen . . . . . . . . . . . . . . . . . 57

4-2 Schematic of the WarmCO2 ’s heat grid connecting Yara and the horticulture. . . 58
4-3 Detailed agent topology of WarmCO2 case study . . . . . . . . . . . . . . . . . 60
4-4 Greenhouse demand profiles of WarmCO2 case study . . . . . . . . . . . . . . . 66
4-5 CMPC vs reference CMPC runtime and optimality gap results for WarmCO2 winter
scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
4-6 HMPC q-iterations and agent runtime results for WarmCO2 winter scenario . . . 75
4-7 CMPC vs reference CMPC runtime and optimality gap results for WarmCO2 sum-
mer scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
4-8 HMPC q-iterations and agent runtime results for WarmCO2 summer scenario . . 76
4-9 CMPC vs reference CMPC agent results for WarmCO2 winter scenario, zoomed-in 79
4-10 CMPC vs HMPC agent results for WarmCO2 winter scenario, zoomed-in . . . . . 80
4-11 Reference CMPC agent results for WarmCO2 winter scenario . . . . . . . . . . . 81
4-12 CMPC agent results for WarmCO2 winter scenario . . . . . . . . . . . . . . . . 82
4-13 HMPC agent results for WarmCO2 winter scenario . . . . . . . . . . . . . . . . 83
4-14 CMPC vs reference CMPC agent results for WarmCO2 summer scenario, zoomed-in 85
4-15 CMPC vs HMPC agent results for WarmCO2 summer scenario, zoomed-in . . . . 86
4-16 Reference CMPC agent results for WarmCO2 summer scenario . . . . . . . . . . 87
4-17 CMPC agent results for WarmCO2 summer scenario . . . . . . . . . . . . . . . . 88
4-18 HMPC agent results for WarmCO2 summer scenario . . . . . . . . . . . . . . . . 89

A-1 Average weekly heat load pattern during four seasons. Left, a multi-dwelling build-
ing. Right, a public administration building with night setback control. [1] . . . . 96

B-1 Mollier hs- and Ts-chart of CHP plant. Dotted line describes the difference of a
non-CHP plant (Frederiksen & Werner 2013, p.158) [2] . . . . . . . . . . . . . . 100

C-1 Schematic of pipe cross section and buried pipe [3] . . . . . . . . . . . . . . . . 102
C-2 Pump and system curve for a variable speed pump in a closed loop pipe system;
only friction losses are considered in the system curve [4] . . . . . . . . . . . . . 105
C-3 The effect of operating a centrifugal pump at the left or right of the BEP [5] . . 106
C-4 Schematic of energy balance across differential pipe segment dx [3] . . . . . . . 107
C-5 Schematic of pipe segments s with length ∆x, where Ts2,in = Ts1,out [3] . . . . . 108
C-6 Pipe network of DH system [6]) . . . . . . . . . . . . . . . . . . . . . . . . . . 110

Y.C.C. Putter Master of Science Thesis

 List of Tables

4-1 Logical truth table (converse implication) for the successive cooling constraint . . 64

Master of Science Thesis Y.C.C. Putter

x List of Tables

Y.C.C. Putter Master of Science Thesis

 Acknowledgements

This thesis originated out of the desire to apply the knowledge gained during my Systems
and Control Master on a sustainability related topic, and to combine it with an internship.
Thanks to Ewoud Werkman and Edwin Matthijssen from TNO’s Monitoring and Control
Services (MCS) department, I was given the opportunity to investigate how to improve TNO’s
HM algorithm such that it could be applied to 4GDH networks. Thanks to dr. ir. Tamás
Keviczky from the TU Delft, the project proposal was iterated and accepted as Master’s
Thesis Project. However, it turned out that agreeing on the original project proposal was
just the beginning. The actual thesis itself proved to be a learning experience on multiple
dimensions.
Therefore, I would like to thank first and foremost my supervisors, dr.ir. Tamás Keviczky
and Ewoud Werkman, for their guidance, time and inspiration. They especially guided me in
my personal development during this research process, helped me in how to approach certain
problems whilst giving me the freedom to make my own decisions, and also facilitated meeting
people that had valuable knowledge to share or interesting case-study opportunities to offer.
For the latter, I would like to thank Marlon Pijpelink from WarmCO2 for the information
and time he provided.
Furthermore, I would like to thank everyone in the MCS department and TNO in general
with whom I could talk and discuss about both thesis and personal related things. A special
thanks goes out to Arun and Anna for their "gezelligheid" and patience when I needed to
think out loud with someone about my thesis. Not to forget, to Dick van Smirren on whom I
could trust to swing by every once in a while to make a joke but also check how I was doing.
Lastly, I am forever grateful to my friends and family who have been there when life happened.
A huge thanks to my parents for their inexhaustible moral and emotional support. Also, a
special thanks to my dear friends and peers Thijs van Winden and Marc van Vliet for the
delicious dinner evenings filled with laughter, good stories and helpful sparring moments. And
lastly, to Inge for sharing some exceptionally good times and giving me peace of mind.

Delft, University of Technology Y.C.C. Putter

October 1, 2018

Master of Science Thesis Y.C.C. Putter

 Chapter 1

Introduction

This chapter gives an overview of the production, distribution and consumption of heat energy
in the Netherlands, referred to as the heat energy system. It is motivated why the current
heat energy system needs to be restructured, and what the main approaches are to restructure
it. The potential of heat grids is presented as one of these main approaches, after which an
overview is given of the current and future heat grids. The importance of a smart controller
is mentioned after which the problem formulation is given.

1-1 Background

Reducing greenhouse gas emissions worldwide is necessary to mitigate the effects of global
climate change [7]. The Paris Climate Agreement, which entered into force on the 4th of
November 2016, coordinates this effort and serves as an instrument to shape national policies
accordingly. One of its main goals is to keep the global average temperature increase to well
below 2o C above "pre-industrial" levels [8].
In the Netherlands, the national targets are currently still set by The Energy Agreement for
Sustainable Growth [9] since 2013. However, with the ratification by the Dutch government of
the Paris Climate Agreement presented in their draft outline [10], the main national goal will
be to drastically reduce the greenhouse gas emissions with 49% to 55% in 2030 compared to
1990. Once finalized, this "Klimaatakkoord" or Climate Agreement will then include detailed
measures and approaches for each of the five sectors: Electricity, Built Environment, Industry,
Agriculture and Mobility. Covering multiple sectors, the restructuring of the national heat
energy system is expected to significantly contribute to these targets and is also an ambition
mentioned in the Energy Agreement:

"To effectively use the energy savings potential of the Dutch demand for heat
as well as the potentials of new renewable energy (including effective use of waste
heat)."

Master of Science Thesis Y.C.C. Putter

2 Introduction

1-1-1 Main Problems of Current Heat Energy System

Although it is commonly known that our electricity consumption and production needs to be
addressed for a more sustainable society, it turns out that at least 40% of the total Dutch
primary energy usage (3.340 PJ) is used for producing and distributing heat energy. This
is twice of what the electricity sector uses. Subtracting the energy losses due to conversion
and distribution, gives the final energy usage for heat (1.142 PJ) and represents the actual
heat demand [11]. Within this final energy usage, a distinction can be made per sector and
required temperature (Tlow ≤ 100o C < Thigh ). The energy mix used for supplying this final
heat energy usage consists mainly out of natural gas with a share of 80%, whereas only 5% is
met by renewable energy sources (RES) such as biomass or waste-incineration processes [12].
The rest includes fossil fuels such as coal. This pattern is similar throughout Europe [7].

Figure 1-1: Dutch statistics of primary energy usage per sector, final heat energy usage per
sector and temperature and the energy mix used for supplying this final heat energy demand [11]

The large share of natural gas may come as no surprise, as most are familiar with burning
natural gas for purposes such as cooking and heating our homes and tap water with a gas-fired
boiler. The natural gas is distributed through the extensive gas network connecting many
households, businesses and industries.
However, burning natural gas or any other fossil fuel results in greenhouse gas emissions and
is therefore not a sustainable option. It is also exergetically inefficient to burn natural gas for
producing low temperature heat (Tlow ≤ 100o C), which makes up two thirds of the total final
heat energy usage (see Figure 1-1). Furthermore, there is political pressure to further reduce
the extraction of natural gas due to substantial (financial) consequences of the earthquakes
in Groningen. And lastly, the gas network is aging, requiring a costly investment if natural
gas remains to be the most important source of energy for heat.

1-1-2 Sustainable Solutions to Restructure Heat Energy System

The Dutch Ministry of Economic Affairs elaborates the ambition stated in the Energy Agree-
ment in its letter "Warmte Visie" [13] and energy report "Transitie naar Duurzaam" [14].
The main target is twofold: first and foremost reducing the total heat demand by insulation
and increasing industry process efficiencies, and secondly lowering the required temperatures
of the heat demand. This makes it easier to meet the heat demand with more sustainable
solutions, varying per situation and region.

Y.C.C. Putter Master of Science Thesis

1-1 Background 3

Biomass is one of the sustainable solutions and can used as a solid, or converted into a
liquid or gas. Although it can replace virtually all fossil-fuel applications, it is more efficient
in applications where alternative sustainable options are scarce such as supplying the high
temperature heat demand of industry, fuel for the transport sector, or as resource for the
chemical processing industry. However, these applications alone would already result in a
high demand for biomass that can only be met with a relatively large import due to national
production capabilities being limited. On top of the import dependency, the actual sustain-
ability of using biomass is criticized. Therefore, although it can be easily supplied through
the existing infrastructure of the gas network, green gas is preferred to only provide in the low
temperature heat demand of sparsely populated areas if absolutely necessary and no other
options are available.
All-electric or electrification is suitable for newly developed residential areas or well insulated
buildings with low heat demands in general, as well as for optimized industry processes. An
example is the installation of a small heat pump instead of a gas-fired boiler in a house.
Because these solutions work on electricity rather than natural gas, the building can benefit
from the national investments made in on- and offshore wind power and other electric RES,
while having the same advantage of relying on an already large-scale distribution network.
It also ties in with the possibility for buildings to offer grid flexibility or even be completely
off-grid. This requires buildings to be able to produce and store their own energy and heat
by a currently still expensive combination of photo-voltaic (PV) panels, solar collectors, heat
pumps, batteries and thermal buffers.
Heat grids are expected to be the cheapest solution in most cases [15]. Especially for
regions which have both a considerably large supply of waste heat from horticulture and
industry, and a highly concentrated low temperature heat demand of industries and densely
populated urban areas which mainly consist of older buildings. Besides utilizing energy that
is otherwise dumped, heat grids can also tap into the fast growing potential of geothermal
energy supplying heat at either low or high temperature depending on the extraction depth.
Connecting combined heat and power (CHP) plants to the heat grid, ensures that the fossil
fuel or biomass is utilized more efficiently. In addition, the integration of thermal energy
storage systems (for instance aquifer thermal energy storage (ATES)) provide flexibility to
heat grids.

1-1-3 Overview of Current Heat Grids

A 3rd Generation District Heating (3GDH) system is explained by the general schematic shown
in Figure 1-2. Pressurized water is pumped through a closed system of supply and return
pipes which are placed together underground. The water in the pipes is therefore only an
energy carrier for the heat energy. This heat energy can be supplied or extracted from the heat
grid via heat exchangers by means of temperature differences with other counterflowing water
streams. The primary transportation network covers the longest distance from heat producer
to the heat exchange substation, from which the heat energy is actually delivered to the
consumers via the secondary distribution network. The secondary distribution network has
typically smaller pipe diameters and lower operating temperatures compared to the primary
distribution network. Information about the type of control currently used to match supply
and demand is not readily available, but a small survey identified that centralized rule based
control or human operated monitoring and control are most commonly applied.

Master of Science Thesis Y.C.C. Putter

4 Introduction

Figure 1-2: A general schematic of a current 3GDH system [16].

Single stakeholder and single heat source are the main characteristics of 3GDH grids. The
stakeholder is often an energy company, which is also the main investor and owner of the
thermal grid infrastructure. Customers connected to a thermal grid are therefore not yet able
to choose their energy provider. To ensure fair pricing, small consumers with connections
up to 100 kW are covered by the Niet-Meer-Dan-Anders (NMDA) pricing principle of the
"Warmte Wet" (Heat Law). This basically states that the price those consumers pay for heat
on average, is limited to the equivalent costs of a gas-fired boiler.
According to [12], there are currently 17 large scale heat grids (> 150 TJ) together responsible
for supplying 20 PJ of heat to 300.000 customers, making up 90% of the total heat delivered
by all heat grids. The largest transportation networks reach distances of 26 km (De Nieuwe
Warmteweg, Rotterdam) and 8 km (Diemencentrale, Almere). Most heat comes from gas
fired CHP plants with local gas fired co-generation units counting for 67%. The share of
biomass fired CHP plants and waste incineration has risen sharply from 8% to 25% between
2013 and 2015. The heat loss varies between grids serving many small (≤ 100 kW) or a few
big consumers (> 100 kW), but the average loss is estimated to be 25%.

1-1-4 Vision of Future Heat Grids

The Dutch Environmental Assessment Agency (PBL) published a report about the vision
for climate neutral heat grids [15]. By 2050, the low temperature heat demand is expected
to decrease for all sectors, except for the industry, resulting in a projected national demand
of 350 PJ. The largest demand comes from residential and non-residential buildings ( 75%).
Geothermal heat is expected to be the largest producer, growing from 3 PJ now to a share
between 25 and 75% of the total potential low temperature heat supply (297 ∼ 1352 PJ).
Aquathermal heat is expected to supply around 80-120 PJ in 2050 according to [10]. Heat
grids are expected to be the cheapest solution for the supply of circa 60 − 70% of the national
low temperate heat demand (350 PJ). The predictions made are based on assumptions such
as future (regulations on) electricity and gas prices and the practical pace of innovation in
RES.

Y.C.C. Putter Master of Science Thesis

1-1 Background 5

A 4th Generation District Heating (4GDH) system is the concept of an "open" heat grid
connecting multiple producers and consumers together, with the aim to improve the energy
savings potential and the use of RES such as geothermal heat more effectively. The main char-
acteristics as described in [17] are summarized below and a comparison with older generation
district heating systems is illustrated in Figure 1-3.

• Lower supply and return temperatures, whilst still being able to provide in the space
heating and hot domestic water needs of existing, renovated and newly designed build-
ings. This implies minimizing energy losses, by better insulation of buildings and pipes.
Also, with lower demands, cascading becomes possible where consumers are connected
in series instead of parallel.
• Maximize the potential heat and cold supply of RES, as well as the minor contributions
of individually connected buildings. E.g., lower operating temperatures make it possible
for more geothermal heat sources to be connected to a heat grid.
• Smart control to optimally match heat supply and demand, as well as the integration
with other smart energy systems to make optimal use of the synergy between heat,
electricity and gas grids.
• System-wide design and planning optimization to identify and implement suitable en-
ergy infrastructures, should no longer be based on worst case operating scenarios.

Figure 1-3: A comparison of the characteristics of all 4 generations of district heating [17]

Master of Science Thesis Y.C.C. Putter

6 Introduction

The above mentioned characteristics, with emphasis on the smart controller, play an impor-
tant role in the assumption that 4GDH are expected to be the cheapest solution for the supply
of 60 − 70% of the anticipated national low temperature heat demand. The role of the smart
controller here is twofold:

1. The operational expenditures (OPEX) can be minimized by determining optimal match-

ing schemes taking into account things like consumer heat demand predictions, the flex-
ibility available in the system such as the total buffer capacity and the efficient use of
devices to lower their operation and maintenance costs.
2. The capital expenditures (CAPEX) of a heat grid can be lowered when both the heat
grid and controller are designed iteratively with insights gained from simulations. This
could lead to smaller required pipe diameters, buffer capacities, or less controllable heat
sources whilst still being able to supply in the requested heat demand.

1-2 Problem Formulation

In short, heat grids are expected to play an important role in a sustainable heat energy
system. In order to meet this expectation, both the OPEX and CAPEX of 4GDH systems
need to be lowered to make it financially competitive compared to other heat energy system
solutions. This requires an online optimal based smart controller that is able to deal with
the characteristics of a 4GDH system, as status-quo rule-based controllers will likely face
difficulties trying to cope with the complexity of the control problem.

TNO’s desire is to improve HeatMatcher (HM), their optimal-based smart controller for
building heating and cooling systems, such that it can be used for these 4GDH systems where
network effects are no longer negligible. These network effects include losses and time delays
that occur during heat energy transport, as well as constraints such as maximum heat energy
flow and flow direction. TNO proposes to implement these effects by trying to apply the same
principles found in the locational marginal pricing (LMP) algorithm of PowerMatcher (PM),
their smart controller for electricity grids.

In this thesis, after an evaluation of the 4GDH system characteristics and the limitations of
the HM and LMP algorithms, a model predictive control (MPC) approach is proposed as
alternative instead of improving the existing HM algorithm with LMP. In order to retain the
multi-agent system (MAS) and peer-2-peer (P2P) framework advantages of HM, a coopera-
tive non-centralized hierarchical MPC is designed using the alternating direction method of
multipliers (ADMM) that is able to handle simplified network effects, deterministic demand
predictions, and mixed-integer modelling.

The optimization problem that is solved by the MPC controller, is formulated as a mixed-
integer quadratic programming (MIQP) problem with non-convex integer constraints. This is
classified as a NP (non-deterministic polynomial time) complete or NP hard problem, where
the time required to find a solution grows rapidly with increasing problem size. Because
the smart controller optimizes continuously on a hourly timestep and eventually needs to be
implemented on embedded systems, the computation time and resources are limited.

Y.C.C. Putter Master of Science Thesis

1-3 Related Work and Contributions 7

1-3 Related Work and Contributions

At TNO, the heat grid simulation software CHESS simulates temperatures and mass flows
throughout a heat grid, based on a decoupled hydraulic and thermal equation assumptions.
However, it is not designed or found suitable as a tractable predictive model for use during
online optimization. The HeatTools project computed offline optimal flow paths for a specific
building heating and cooling systems, by a brute force search of all combinations of fully
open or closed valve settings. In [18], a network model similar to CHESS but with simplified
assumptions is used during the optimization of a quadratic programming (QP) problem, to
calculate optimal thermal power production and the opening degree of valves in a bench-
mark network. In [19], the production planning for the distributed Uppsala district heating
network is cast as a non-linear programming (NLP) problem and solved with interior point
optimizer (IPOPT), with added modelling detail of the main combined heat and power (CHP)
plant. However, they assume constant or fixed time delays. Variable time delay systems are
optimized with MPC in [20] and [21], but are implicitly modelled with compensation strategies
such as the state predictor and Smith predictor. In [22], it is approximated with a piece-wise
affine (PWA) system, but the approach fails to model the integral relations between time de-
lay and mass flows correctly. To the best of the authors knowledge, no modelling methods (or
set of mixed-integer linear constraint equations) were found for variable time delay tracking
that fit inside the MIQP optimization problem formulation, and is therefore not considered.
In [23] and [24], a MAS MPC control framework is presented that additionally focuses on
dealing with the uncertainty in heat energy demand [25], whereas in this thesis a straight-
forward deterministic heat demand is assumed [26]. For their network model, a simplified
bi-directional energy exchange model is used instead of a mass flow and temperature based
model. Here, only losses and maximum line flow constraints are considered, not time delays,
and both are translated to the agent models. Because of their bi-directional flow implemen-
tation, the losses are not properly calculated for heat energy exporting agents. In this thesis,
this network model is slightly adjusted to a directional flow model, where the effects of the
losses are translated to the global constraints instead of the agent models. Also, a simple
constant time delay modelling is included in the network model.
Of the different decomposition techniques to design a non-centralized MPC algorithm (see
Appendix D), an implementation of the ADMM found in the Optimal Exchange problem
formulation [27] of is used. Furthermore, this approach shares similarities with the suggested
LMP algorithm principle found in PM [28]. Also, the mixed-integer constraints found in [23],
[24], [29] for the production capacity, ramping and minimum required on/off times are slightly
adjusted with the help of hybrid modelling theory found in [30].
For solving non-convex MIQP problems, standard algorithms that find the global optimal
solution are brute force, branch-and-bound and branch-and-cut, but take non-polynomial
time to compute, suffer from large variance in their runtimes and are generally not very
suitable for embedded applications where computing resources are limited. In [31], a heuristic
algorithm based on ADMM allowing only equality constraints, is claimed to find a fast good-
quality suboptimal solution. However, implementing the online available algorithm failed for
unknown reasons to produce the same results even for its own case-studies mentioned in the
paper. Instead, the commercial solver Gurobi is used, but the conversion of inequality to
equality constraints with the help of slack variables is kept for possible future implementation
of a heuristic solver.

Master of Science Thesis Y.C.C. Putter

8 Introduction

1-4 Outline

In this chapter, it is motivated why 4GDH systems are relevant for a sustainable heat energy
system and the importance of having a smart controller that optimally matches heat energy
demand and supply. For the remainder of this thesis, the outline is as follows:

• Chapter 2 first gives an overview of the 4GDH characteristics that a smart controller
should ideally be able to take into account. Next, the basic principles of HM, LMP and
MPC are given, and it is motivated why focus is shifted towards designing an MPC
smart controller as an alternative approach instead.
• Chapter 3 shows the initial design of a general MAS centralized MPC controller,
the models used for the agents and the simplified energy-exchange based model of an
academic heat grid. It is shown how ADMM is used for the decentralized hierarchical
MPC algorithm. An academic heat demand scenario with specifically chosen parameters
is used to test if the optimization results show expected behaviour, after which the
results are discussed.
• Chapter 4 The initial design of the smart controller is adjusted to a more realistic
case study of the WarmCO2 heat grid in Terneuzen. The modelling assumptions are
explained, and results are shown and discussed for two different scenarios where the
heat grid operates at maximum and minimum combined requested heat demand.
• Chapter 5 Here all conclusions and observations are summarized, and suggestions for
future work are given.

Y.C.C. Putter Master of Science Thesis

 Chapter 2

Smart Control Methods

In this chapter, an overview is given of relevant smart control design and modelling consider-
ations specifically for 4th Generation District Heating (4GDH) systems. Then, the basics of
two different smart control approaches are explained and compared in their ability to capture
most of the design considerations. HeatMatcher (HM) and locational marginal pricing (LMP)
as suggested by TNO are first discussed, after which model predictive control (MPC) is pre-
sented as the alternative approach.

2-1 Smart Control Design and Modelling Considerations for Heat

Grids

In this section, some smart control design considerations to model the characteristics of 4GDH
systems are explored. First, a brief overview of the available flexibility within the system is
given, such that it can be optimally used if modelled. Next, a separation of concerns approach
in the form of multi-agent system (MAS) modelling is introduced to allow for a scalable
and general solution, such that it can cope with the size and complexity of different large
scale district heating systems. Here, the obvious trade-off between modelling accuracy and
computational effort is mentioned. Lastly, the different types of collaboration and information
sharing between the MAS subsystem controllers are discussed, and how they could affect the
optimal solution, computational effort and scalability.

2-1-1 Flexibility

Flexibility is found in both heat energy supplying and consuming agents. Modelling and using
this flexibility gives the opportunity to influence the heat energy demand over time, and to
provide in a smart way.
For consumers, flexibility is found in demand-side management (DSM) measures that focus
on influencing the requested heat demand profile. The first measure, passive and therefore not

Master of Science Thesis Y.C.C. Putter

10 Smart Control Methods

related to flexibility, is to lower the overall demand by giving consumers incentives to invest
in energy efficient measures, such as applying building insulation. Other DSM measures such
as Demand Response and Interruptable Load Programme are related to flexibility, as they
achieve flattening of demand load with price signals, rewarding consumer devices to shift their
heat demand away from peak times. Consumers could also indicate a lower and upper bound
of their heat demand instead of a single setpoint, such that the offered flexibility can be used
by producers in exchange for financial compensation for the loss in comfort. Another form of
flexibility is offered by the buffers or accumulators at local and grid level (for instance aquifer
thermal energy storage (ATES)), making it possible to even out the demand load over time.
Lastly, the thermal inertia of a building can also be regarded as a thermal buffer.
For producers, the degree of flexibility or controllability depends on the physical process of
heat generation. For example, combined heat and power (CHP) plants and gas fired boilers
can supply at different output levels both in terms of supply temperature and total thermal
energy output by changing the operating mode of the production units. In comparison, solar
collectors and geothermal units differ as they have no or limited flexibility in their output
levels. They are respectively dependent on the environmental conditions or give a more or
less constant output.
For the coupling network consisting of the pipes and grid devices such as pumps and valves,
the notion of flexibility can be stretched further if variable mass flow rate in the heat grid
is also considered. Then, the same heat demand can be met by either changing the supply
temperature at the producer or by changing the mass flows with valves and pumps. By
changing the water temperature in the pipes, the grid itself can be used as a buffer which
is referred to as (un)loading. Due to the difference of propagation speeds of a change in
temperature or mass flow rate, loading and unloading can be used for well predictable heat
demand peaks, while flow rate changes are used to react quickly on smaller variations in
thermal energy demand. Both actions have a certain energy efficiency and influence the
degree of energy loss and time delay.

Figure 2-1: Four device categories of the energy flexibility interface (EFI) for the electricity sector

As an example, 4 different categories of flexibility defined for systems and devices in the
electricity sector are shown in Figure 2-1, but the same idea applies to the heat energy sector.

Y.C.C. Putter Master of Science Thesis

2-1 Smart Control Design and Modelling Considerations for Heat Grids 11

On a side note, the example of the figure comes from the energy flexibility interface (EFI),
which is an open source standard that simplifies interoperability between a device and DSM
programs. In other words, it makes an abstraction of the specifications or energy flexibility
of a energy device without inferring how the device itself is controlled or how the Smart Grid
Technology must work.

2-1-2 Multi-Agent System and Modelling

The producers, consumers, buffers and grid devices can be viewed as separate agents (rep-
resenting a subsystem) connected to each other via the pipes within the entire coupled heat
grid network, where each agent can be owned by a different stakeholder. Each agent’s actual
subsystem can be represented by a model that describes the relation of how the control inputs
affect the agent’s subsystem output or behaviour. In general, a trade-off is made between
model complexity, accuracy and computational performance. Dividing the system into sub-
systems in such a MAS approach and applying a separation of concerns principle, will result
in a more scalable control structure. This generalization makes it easier to apply it to multiple
heat grids.

Heat Demand Profile

For the consumer agents, the requested heat demand is what drives the matching algorithm.
Therefore, being able to take into account and having an accurate predictive model of the
heat demand during optimization, makes it possible to prepare for peak heat demands in
a smart fashion using the flexibility of the entire system. Prerecorded data from the heat
demand can be used directly as a profile, or instead to construct a model that generates a
day-ahead profile. Here, a distinction can be made in static and dynamic models, the degree
on which it is based on statistics or physical relations, and whether it includes uncertainty
(stochasticity) that could also be taken into account during the optimization (see Appendix
A).

Mixed-Integer Modelling

For the producer agents, controllable heat sources such as gas fired boilers are often modelled
by mixed-integer or hybrid models. These involve the use of binary variables representing
the state (off = 0, on = 1) mixed with continuous variables that are influenced by the state.
By doing this, additional constraints can be modelled to avoid optimization solutions with
high frequency on-off switching behaviour, helpful in increasing efficient use of a device and
lowering high maintenance costs. However, the type of optimization problem often becomes
non-convex due to the use of integer variables, which is generally harder to solve than convex
problems.

Coupling Network

For the coupling network, the model should ideally cover the fact that heat energy transport
is governed by mass flows and temperature differences (see Appendix C), which includes:

Master of Science Thesis Y.C.C. Putter

12 Smart Control Methods

• Mechanical Losses
Water flowing through a pipe (pipe flow) experiences mechanical losses due to pipe
friction, bends and passing through devices such as heat exchangers. Pumps and valves
are controllable grid devices used to respectively overcome and influence these losses,
and determine the flow rate and direction throughout the network. Modelling these
mechanical losses make it possible to determine the required pumping energy.
• Thermal Losses
A pipe flow with a certain flow rate and temperature, also experiences thermal losses
due to a temperature gradient with the environment and heat exchanger efficiencies.
• Variable Time Delay of Energy Transfer
Energy transfer between two points is not immediate, but has a variable time delay
influenced by fluid properties (function of the water temperature), flow rate (function
of the pump and valve positions) and path.
• Topological Constraints
The maximum energy flow through a pipe is bounded by the pumping capacity, pipe
diameter and thermal losses. Therefore, upper and lower bounds on the flow rates
and temperatures as well as constrained flow directions for each pipeline need to be
considered to ensure that the model describes the practical pipeline constraints correctly.

2-1-3 Cooperative vs Non-Cooperative Agents

Each agent’s optimization objective can be expressed by a minimization of costs and/or

maximization of utility. Basic economic laws of supply and demand state that each producer
wants to sell energy at the highest price, while consumers want to buy energy at the lowest
price. Furthermore, producers want to minimize operating costs and consumers want to
maximize their comfort. An optimal solution to the supply and demand matching can be
found, but depends on the cooperation of the agents.
When the objectives of all stakeholder’s agents are combined to a single global objective for
the entire heat grid, then the system is said to be cooperative. In this case, all the consumer
agents would share the utility equally for instance. However, when all stakeholder’s agents
are primarily focused on their own objective, the system is considered to be non-cooperative.
For instance, consumers could be indifferent to whether their neighbours are being met in
their heat demand or what the requested heat demand imposes on the producers or system
as a whole. In other words, the local optimal solutions can differ from the global optimal
solution.
Determining how supply and demand are matched with multiple producers of different stake-
holders is therefore highly depended on how different objectives are factored. For example,
how the non-cooperative fairness between producers of different stakeholders is factored in
and which type of producer is favored over others. The same holds for the location of a
producer, if the optimal solution focuses on minimizing energy losses.
Currently, heat grids have a monopoly market model, in which one producer can determine
their own market price, with the exception that consumer connections up to 100 kW are
protected by the Niet-Meer-Dan-Anders (NMDA) principle of the "Warmtewet". Future heat
grid could lean towards a purely non-cooperative market model or an oligopoly. The first

Y.C.C. Putter Master of Science Thesis

2-1 Smart Control Design and Modelling Considerations for Heat Grids 13

describes the situation where there is an abundance of producers such that the price is solely
determined by market demand. In this case, the producers can only react to the demand,
and try to produce heat energy at the lowest cost possible. Oligopoly describes the situation
where there is only a number of producers with the ability to influence the market price.
The use of a fluctuating price could be possible in these market models. In that case, the
bidding strategies of producers on short and long term as well as game theory start to play
an important role.

2-1-4 Centralized vs Non-Centralized Control

In a multi-agent system, the size of the optimal control problem tends to grow fast with
large numbers of participating agents, especially when the agent models are very detailed. If
the information of all agents in the coupled system is used in a single controller, then this
is referred to as a centralized approach. Even if this control architecture can be useful in
determining the global optimal solution for the entire system, the size of the optimization
problem can become infeasible to solve within a certain required computation time.
A non-centralized approach tries to find the same optimal solution, but now by assigning
a controller to each individual agent or agent group (for instance, by combining all heat
grid devices such as the pumps and valves) and by some sort of information communication
between the agents. In case the controllers communicate only indirectly by measuring the
effects on the shared system, then this is referred to as a decentralized approach. In case
the agent controllers can also communicate information directly with their neighbours, then
a distributed approach is in place. Furthermore, a hierarchical control approach is when the
agent controllers communicate via a higher level controller. In Figure 2-2, the differences in
control architecture are illustrated.

Figure 2-2: A schematic showing the differences in information/communication flow (arrows)

between the controllers for different control architectures. The decentralized approach has only
the solid arrows, the distributed approach includes the dashed arrows, and right a hierarchical
control approach (for both distributed and decentralized lower level communication).

The scalability improves when a non-centralized approach is used, as it is possible to solve each
subproblem in parallel reducing the computational time required to solve the entire optimiza-
tion problem. However, additional computational overhead is introduced due to information
communication between the controllers and synchronization needs to be considered. Also,
if only partial information exchange is required between controllers, then agent information
privacy can be retained which is an advantage in a system with different stakeholders.

Master of Science Thesis Y.C.C. Putter

14 Smart Control Methods

A hierarchical approach can be used when system information and control occurs at different
timescales or when it is not trivially separable among the agents. For instance, currently the
operation of the producer and the network are both controlled by one stakeholder. For future
heat grids with multiple producers of different stakeholders, the operation of the network is
likely to be decoupled from the interests of a single producer. This imposes the question
whether a truly distributive approach is possible, as network state and control influence all
agents. Instead, the network (agent group) could be seen as the higher level coordinator
in a decentralized hierarchical approach, where the consumers and producers of different
stakeholders communicate with this coupling network agent.

2-2 HeatMatcher

TNO’s HM is a suitable control candidate, but as of now it is only designed for the heating
and cooling systems of buildings, i.e. small scale networks. Therefore, improvements need
to be implemented into the existing control algorithm to take large scale network effects into
account. For this, LMP is suggested by TNO. As their preferred control candidate, this
section will first discuss why HeatMatcher was developed, how it works and what previous
suggestions for improvements are. Then, LMP is discussed in the next section.

2-2-1 Background

Most heating and cooling systems are operated by climate control systems, such as those of
Priva and Honeywell, which are based on centralized rule-based temperature control. How-
ever, the number of devices in a single system has increased, mainly caused by the desire to
make use of renewable energy sources (RES). Together with the advancements in device con-
trol options, the expanded control space tends to be a problem for traditional climate control
systems. It is difficult to design and fine-tune such systems to reach optimal performance
and the maintenance requires knowledge of all inter dependencies of control parameters. This
makes it prone to errors resulting in malfunctioning, inefficiencies, and economic losses.
As a response to these observations, HM was developed based on the main design qualities
of PowerMatcher (PM), which is a MAS approach combined with market based control [28].
Openness, the ability to easily integrate all kinds of Distributed Energy Resources (DER).
Privacy, to ensure that the owner stays in control over their devices and information. And
scalability, such that it can cope with the increasing number of connected devices and can
be deployed on large scale for future heat grids. During pilots conducted at three residential
apartment buildings and one office building, it has shown it can reduce the gas usage by
12-28% depending on the existing installation. In the first trail, the energy bill was reduced
by 18%.

2-2-2 Operating Principle

In HM, each agent represents a producer, prosumer or consumer of heat within a system, such
as heatpumps, solar panels, gasboilers, buffers and end-users. Each agent has a bidcurve, that
represents how much the agent is willing to produce or consume heat energy as a function of

Y.C.C. Putter Master of Science Thesis

2-2 HeatMatcher 15

the market price. Here, heat energy supply is defined as negative heat energy demand. To
reflect general economic supply and demand laws and have one equilibrium, the bidcurves
need to be monotonically decreasing, see Figure 2-3.

Figure 2-3: General heat demand bidcurves for four types of agents (source: EFRO final report
HM). Heat energy supply is defined as negative heat energy demand.

One or multiple virtual energy exchange markets are present, grouping agents that are phys-
ically connected to each other. For each market all the bids are gathered and combined by
adding them up during multiple rounds of different contract durations. This is because some
devices have an minimum on and off time to mitigate wear. The equilibrium price is then cal-
culated by the market auctioneer with linear programming at the point of minimal mismatch
between the demand and supply of energy, conforming to device constraints. Due to the
non-continuous operating ranges of devices, mismatch can occur. Yet, it is non-problematic
for small scale networks as it is compensated by the slowness of a heat system, the short cycle
algorithm and the minor correcting reaction in heat energy demand of the consumers.

Figure 2-4: The simplified network used to show the bidcurve principle of the HM algorithm and
the algorithm that relies on LMP. On the left side there is a red producer d1 (p) and consumer
d2 (p), and the right side there is a blue producer d3 (p) and consumer d4 (p). Each side has a
combining node vred and vblue that are also connected to each other to allow flow from one side
to the other.

Master of Science Thesis Y.C.C. Putter

16 Smart Control Methods

Figure 2-5: Bidcurves of 4 agents combined at a virtual market auctioneer, where the red
consumer demand d1 (p) and producer supply d2 (p) are close to each other in terms of node
location, as well as the blue consumer demand d3 (p) and producer supply d4 (p). The vertical
black line represents the market clearing or equilibrium price λ. Heat energy supply is defined as
negative heat energy demand.

When the equilibrium price is set, contracts are then returned to the agents. The architecture
of HM abstracts from all kinds of agent details such as device specific operation and how
bidcurves or bid functions are defined. This separation of concerns ensures that optimal
control of the device itself is left to the owner, as the device manufacturer knows best how
to define the flexibility of the device and its constraints. Scalability is ensured by the MAS
approach and because the peer-2-peer (P2P) framework does not require a central component
or registry, the computational effort can be distributed.
The intelligence of HM is not in finding the equilibrium point once the bidcurves are known,
which is a convex linear programming (LP) optimization problem, but in the process of the
agents’ bidcurve creation. Important features are the maximum, minimum, the subgradient
points and the slope, which basically results in piece-wise affine (PWA) functions. The actual
assignment of values for these features depend largely on the current state of the agent, the
∆T with the other agent states assuming fixed flow rate and grid specific demand scenarios
and strategies for desired state behaviour. This last part can be undesirable, as it requires
expert knowledge during the setup of HM.

2-3 Locational Marginal Pricing

LMP is widely used in the electricity sector and described in PM [28], on which HM is based
upon. In it, it is described how the effect of maximum line capacity, line losses, transportation
costs and network storage could be incorporated into the bidcurves of the agents. In this
section, a short summary of the working principle described in [28] is given and what the
challenges are to make it work for the thermal energy domain. The variable names used here
only apply to this section.

Y.C.C. Putter Master of Science Thesis

2-3 Locational Marginal Pricing 17

2-3-1 Energy Flow Model

First of all, the network flow is modelled by a directed graph G = hV, Ei with node set
V = {v1 , v2 , . . . , vN } and line set E = {e1 , e2 , . . . , eI }, where n = 1 . . . N and i = 1 . . . I. For
a line ei = {hi , ti , ri , ziM }, positive flow z is defined from head node hi to tail node ti with
resistance ri and maximum line flow capacity ziM .
Incidence matrix A represents the graph topology.

1

 if hi = vn
(N −1)×I
Ai,n = −1 if ti = vn (2-1)


0 otherwise

Resistance matrix R represents the resistance matrix, where

(
L×L ri if i = j
Ri,j = (2-2)
0 otherwise

The network transfer matrix H holds the relation between the subtractions dn at nodes
vn : n = 1 . . . N − 1 and line flows zi at all lines i = 1 . . . I. It can be computed with the
incidence and resistance matrix, and misses one node demand dN referred to as the swing
node d∗ .
 −1
B = AT R−1 A (2-3a)
H = R−1 AB (2-3b)
z = Hd (2-3c)

From the conservation of energy and with the introduction of total network losses L and
change in storage ∆S, it follows that the swing node demand d∗ is:
"N −1 #
∗
X
d =− dn + L + ∆S (2-4)
n=1

2-3-2 Price Components and Constraints

Each agent’s dn represents the demand at node vn with its demand function dn (p), where p
stands for the market price. As previously stated, these demand functions are monotonically
decreasing and supply is defined as negative demand.
if x ≤ y , then f (x) ≥ f (y) (2-5)

If the network is ignored, there is one general equilibrium price p∗ such that demand and
supply is matched. If the network is included, then the market based optimisation generalizes
to find a set of locational prices pn at each node n = 1 . . . N , such that:
N
X
dn (pn ) + L + ∆S = 0 (2-6a)
n=1
|zi | ≤ ziM ∀i (2-6b)

Master of Science Thesis Y.C.C. Putter

18 Smart Control Methods

These locational prices pn can be expressed as:

pn (t) = λ(t) + ηn,C (t) + ηn,L (t) + ηS (t) (2-7)
where,
λ(t) = p∗ (t) [Market Clearing Component]
I
X ∂zi (t)
ηn,C (t) = θi (t) [Line Capacity Component]
i=1
∂dn (t)
I
X ∂Li (zi (t)) ∂zi (t)
ηn,L (t) = λ(t) [Network Losses Component]
i=1
∂zi (t) ∂dn (t)
ηS (t) = λ(t)∆S (t) [Network Storage Component]
Here, the line capacity component can be both positive or negative and depends on the
incremental flow through line i caused by node n. During optimization, θi is adjusted to
ensure that the line does not overload due to all agents. The network losses component depend
on the line loss magnitude Li and the market clearing price λ, as well as the actual line flow.
The network storage component can not be attributed to a specific node and is therefore
only a function of the λ and the total change in network storage ∆S. The search space, at
time t for brevity, yielding a set of locational prices pn (2-7) then becomes (λ, θ1 , θ2 , . . . , θI ).
Eventually this will lead to a set of locational marginal prices, as shown in Figure 2-6.
I I
!
X ∂Li (zi (t)) ∂zi (t) X ∂zi (t)
pn (t) = λ(t) 1 + + ∆S (t) + θi (t)
i=1
∂zi (t) ∂dn (t) i=1
∂dn (t)

Figure 2-6: Bidcurves when LMP is applied. Heat energy supply is defined as negative heat
energy demand. Again, the red consumer demand d1 (p) and producer supply d2 (p) are close to
each other in terms of node location and have a connection to node vred , as well as the blue
consumer demand d3 (p) and producer supply d4 (p) that have a connection to node vblue . The
two combining nodes are also connected to each other e5 = {vred , vblue , r5 , z5M }. Now, besides
the market clearing price or equilibrium λ (black vertical line), there are two locational prices
pred (t) and pblue (t) (red and blue vertical lines) such that network losses, storage and capacity
constraints are now taken into account.

Y.C.C. Putter Master of Science Thesis

2-4 Model Predictive Control 19

2-4 Model Predictive Control

In this section, MPC is proposed as an alternative approach to LMP, and some of the basics
behind it will be discussed. It is proposed as alternative mainly due to advantage that MPC
computes the optimal solution over a finite prediction horizon Np or time span instead of a
single timestep. This gives a more natural framework to model the effects of a (variable) time
delay of heat energy transport as well as time-based constraints of a subsystem or device,
which are inherent to heat grids. Furthermore, it also provides a good framework to take into
account heat demand profile predictions during optimization, which should theoretically lead
to optimal use of the flexibility within the system. Lastly, MPC can also deal with errors
between predicted system behaviour and actual system behaviour.

However, it requires a different way of thinking with regard to both HM and LMP, as MPC
uses (dynamical) models of the subsystems or devices to be able to predict how certain control
inputs influence system behaviour. This makes it possible to evaluate if the control sequence
for the entire time span is optimal and satisfies the constraints. The modelling complexity
of the (sub)system is entirely up to the designer, and must be chosen such that a practical
trade-off is made between prediction accuracy, system behaviour and computational effort.
Furthermore, optimizing over a longer prediction horizon requires more computational time,
but also ensures that it can anticipate better to heat demand peaks further away in time.

Consider a dynamical system that can be described by a discretized model (2-8) that is a
function of the state x(k) ∈ X , input u(k) ∈ U and disturbance w(k) ∈ W. From here on,
the same notation will be applied as mentioned in Section 3-1-1.

x(k + 1) = f (x(k), u(k), w(k)) (2-8)

MPC uses this predictive system model, to predict system behavior or state evolution of the
actual system over a finite prediction horizon Np . At each timestep k, it measures the current
state x(k), and then searches for the optimal control sequence u∗ (k) that minimizes the cost
function J (t) (2-9) and satisfies the constraints (2-10) over the entire prediction horizon.
After the optimal control sequence is determined, only the first computed time step u∗ (k)
of u∗ (k) is actually applied to the system. The process is then repeated for the next time
steps, shifting the prediction horizon accordingly, which is also referred to as receding horizon
control, see Figure 2-7. This moving prediction horizon ensures that a mismatch between the
previously predicted x(k + 1|k) and the currently measured x(k + 1|k + 1) state due to actual
disturbances can be corrected, giving it a closed loop control structure. The other optimal
computed time steps are often time-shifted and used as initial guess at the next time step.

u∗ (k) := argmin J (x(k), u(k))) (2-9)

u(k)

s.t.

h(x(k), u(k), w(k)) ≤ 0 (2-10a)

g(x(k), u(k), w(k)) = 0 (2-10b)

Master of Science Thesis Y.C.C. Putter

20 Smart Control Methods

where,
h i
x(k) = x(k + 1|k) x(k + 2|k) . . . x(k + Np |k)
h i
u(k) = u(k|k) u(k + 1|k) . . . u(k + Np − 1|k)
h i
w(k) = w(k|k) w(k + 1|k) . . . w(k + Np − 1|k)

(a) (b)

Figure 2-7: A general schematic of a Model Predictive Control approach. On the left (a), all
MPC trajectories are shown where the optimal control sequence (cyan) is shown for a single
timestep k. The goal is for the predicted output to be equal to the reference trajectory, under
certain constraints and cost function. On the right (b), the Receding Horizon Control principle
of MPC for the time steps k, k + 1 and k + 2, which shows that only the first time step of the
entire optimal control sequence is actually implemented.

Figure 2-8: Schematic showing a basic explanation of the MPC controller, with disturbances
w(k) ∈ W omitted in the functions within the blue highlighted MPC controller box.

Y.C.C. Putter Master of Science Thesis

2-4 Model Predictive Control 21

The type of the resulting optimization problem, lineair or non-linear and convex or non-
convex, will depend on the cost function J (k), constraints h(k) and g(k) and type of vari-
ables (integer and/or continuous) used to construct the predictive system model. Again, the
complexity of the modelling is up to the designer, with the goal to find a balance between
computational effort and system prediction accuracy. However, as the mathematician R.
Tyrrell Rockafellar once said:

"In fact, the great watershed in optimization isn’t between linearity and nonlin-
earity, but convexity and nonconvexity."

Meaning that a non-convex optimization problem is harder to solve in general when there
are multiple local optimal solutions, especially when their objective value differs from the
global optimal solution. This could pose a strain on the computation resources required to
find the global optimal solution with standard brute force or cut-and-branch solvers, if the
optimization problem falls within complexity class nondeterministic, polynomial time (NP).
Heuristic methods, such as [31] and [32], often use optimization problem relaxation techniques
to try and find a feasible local optimal solution instead with good quality (close to the global
optimal solution) in less time. Proving that such a method convergences is often tied to
specific assumptions about the optimization problem type.
Lastly, if a MAS approach can be taken to divide the entire system into interacting subsystems,
of which each can be represented by their own predictive subsystem model, then it is very
advantageous to decompose the entire optimization problem into subproblems per agent. It
would then be similar to the key advantage of HM’s P2P and MAS approach, as a non-
centralized approach could be computed in parallel, making it possible to be applied to
systems with a very large number of agents. Also, information sharing between the agents
can be limited. Different decomposition techniques are available, depending on the type of
the optimization problem.

Master of Science Thesis Y.C.C. Putter

22 Smart Control Methods

2-5 Conclusion
In this section it will be elaborated why an MPC implementation is developed during the
remainder of this thesis, instead of focusing on improving the HM algorithm by adapting
the proposed LMP algorithm. It should be noted that a direct comparison will not be made
in this thesis, but that the MPC implementation serves as an example of how the control
problem can be approached differently. Therefore, most observations and arguments used
here, are only to determine if the MPC approach can at least have the potential to tackle the
problems that would also be encountered when the HM and LMP algorithms are improved.

2-5-1 LMP Based on Electricity Flow Model

First, the key problems with applying the current LMP algorithm to heat grids will be dis-
cussed. The main observation is that tLMP is based on a electricity flow model.
1. Electrical grids are passive networks, i.e., energy flows from sources to sinks are a result
of the relation between the line resistances and the supply magnitude of the sources.
In comparison, heat grids are active networks where a flow is actively directed by grid
pumps and valves to transfer energy from source to sink.
2. Electrical grids can be assumed to have instant energy transportation over the power
lines (due to speed of light), whereas heat grids suffer from a variable time delay due to
the propagation speed of a change in temperature.
3. In heat grids the thermal and mechanical (pressure) line losses Li (zi (t)) are a function
of the line flows and need to be determined iteratively together with the search for
locational marginal prices. The same applies for the resistance elements of R. If a
nominal operating flow is assumed, then the line losses and resistance elements could
be approximated as a function of the distance between two agents, eliminating the
zi (t))
dependency of a change in line flow Li (  = L (t).
i

2-5-2 Internal Suggestions for HM Improvements

Secondly, in an internal evaluation report of HM, numerous suggestions for future improve-
ments to HM were pointed out that are currently solved or in the process of being solved.
These suggestions are therefore also important to mention for MPC, as it at least needs to
be able to handle the nature of these kind of problems.
1. In the small scale building networks, situations occurred where multiple agents had
conflicting interests in the operation of pumps and valves, or where one valve could
only allow one energy loop to flow.
2. When a 30-minute contract has just been assigned it is fixed. The contract cannot be
adjusted for the duration even though the device is theoretically able to supply more
or less.
3. The iterative process for the auctioneer to take into account the different durations of
contracts make the algorithm more complex than needed.
4. The possibility of using the network as a buffer by slightly adjusting the supply and
return temperatures is mentioned (loading/unloading).

Y.C.C. Putter Master of Science Thesis

2-5 Conclusion 23

2-5-3 Arguments for MPC approach

Next, the arguments are given for choosing to implement an initial MPC controller during
the remainder instead of improving the HM with a current LMP are given.

Optimizing Over Prediction Horizon

First of foremost, a key difference is that MPC formulates the optimization problem over
an entire prediction horizon Np or time span, using models and predictions of heat demand
profiles and system behaviour. This differs from HM, as HM calculates the instantaneously
pareto-optimum price p∗ with certain supply constraints, but without taking into account
whether the contracts are also optimal with respect to future system behaviour and desired
state. I.e., HM essentially optimizes over a single time step and not over a time span.
Optimizing over the prediction horizon makes it possible to utilize the energy flexibility po-
tential more naturally, as it can take into account predicted heat demand peaks. Furthermore,
the network model, consisting of pipes, pumps, valves and heat exchangers, can be designed
from scratch such that it represents an active network with (variable) time delay. Moreover,
if it is physically modelled based on temperatures and mass flow relations, then (un)loading
of the network could be a natural result of the optimal control sequence. Also, if a mixed
integer modelling approach is taken, then time based constraints can be taken into account
during optimization, such as staying on or off for a minimum required time to mitigate wear.
Again, the complexity of the MPC models is up to the designer, but should serve to give an
acceptable balance between accurate controller performance and computation time.

Receding Horizon Principle

Due to the receding horizon principle, MPC can also be used as an online controller that can
handle errors between predicted system behaviour and actual system behaviour. Because only
the first time step is actually implemented and the optimal control sequence is re-evaluated
each time step, it does not suffer from fixed contracts over different times.

MAS and Distributed Optimization

Lastly, if a MAS modelling approach is taken, and the optimization problem is decomposed in
sub optimization problems for each agent, then it could be setup in a similar fashion as HM.
Moreover, if the network linking all other agents is viewed as a separate coordinating agent,
then this separation of concerns approach could eliminate the conflicting interests problem
between the producing and consuming devices or agents.

Master of Science Thesis Y.C.C. Putter

24 Smart Control Methods

Y.C.C. Putter Master of Science Thesis

 Chapter 3

Initial MPC Design Case Study

In this chapter, an initial model predictive control (MPC) modelling setup and a scenario
are presented, explaining the mixed-integer quadratic programming (MIQP) models and as-
sumptions used for the network, nodes, producers, consumers and buffer. To stay close to the
approach of HeatMatcher (HM), a multi-agent system (MAS) approach is applied to eventu-
ally obtain a centralized MPC and a hierarchical model predictive control (HMPC) algorithm
with the use of the alternating direction method of multipliers (ADMM).
The initial MAS modelling setup implemented in Matlab and scenario will serve as a starting
point, from which complexity can be increased in a stepwise manner. This is to adhere to one
of the desired design goals from TNO, which is to setup code in a modular (object-oriented
programming (OOP)) and generalized manner regarding the network and agent models. This
is beneficial for re-use in future projects and for allowing different heat grid typologies and
(parameter) scenarios to be tested.

3-1 Multi-Agent System Modelling

Four different agents are specified, a controllable producer that could be taken as a simple
representation of a CHP plant with an additional gas-fired boiler unit, a consumer with a
deterministic heat demand profile and perfect prediction, and lastly a buffer representing a
thermal energy storage system.

3-1-1 Notation

A specific variable notation and structuring is used, mainly for a more convenient way to
construct and update the constraints matrices and vectors for the entire prediction horizon
t = k . . . k + Np − 1 (see Appendix E), that model the agent behaviour.
A single time dependent variable is denoted with a(k), with minimum, maximum and opti-
mum denoted by respectively am , aM and a∗ (k). Furthermore, the initial value, linear cost

Master of Science Thesis Y.C.C. Putter

26 Initial MPC Design Case Study

parameter and quadratic cost parameter for each variable is denoted by respectively a0 , ar
and aR . Some variables also have other superscripts, which are used for modelling constraints.
Also, a(k + 3|k) can be used to denote the prediction for time k + 3, determined at time k.
A vector containing a grouping of variables is denoted by a tilde symbol b̃(k) and the trans-
pose by the capital superscript T (b̃T (k)). Furthermore, the Np (time expanded) vectors for
the single time dependent variable a(k) and the variable group vector b̃(k) are in bold. An
example is given below for the two binary on-off status variables vp (k) and vs (k), grouped in
the variable group vector v(k),
h i
ṽ(k) = vp (k) vs (k) (3-1a)
h i
vp (k) = vp (k) . . . vp (k + Np − 1) (3-1b)
h i
ṽ(k) = vp (k) . . . vs (k) | ... | vp (k + Np − 1) . . . vs (k + Np − 1) (3-1c)

For nested variable vectors, the same notation holds but with a delimiter symbol | between
the variables that are not grouped. An example is given for the nested variable vector of the
producer agent i = P1 (3-2a). The optimization vector θ̃ng (k) for each agentgroup, combines
the nested variable vectors of all agents that share the same agentgroup number ng (3-2c).
h i
ũP1 (k) = h̃(k) | ṽ(k) | ṽramp (k) | h̃xc (k) | s̃(k) (3-2a)
h i
ũP1 (k) = h̃(k) | ṽ(k) | ṽramp (k) | h̃xc (k) | s̃(k) (3-2b)
h i
θ̃ng (k) = ũP1 (k) | ... | ũB1 (k) (3-2c)
h i
θ̃ng (k) = ũP1 (k) | ... | ũB1 (k) (3-2d)

The same notation also holds for the corresponding capitalized constraint matrices E, and
lowercase denoted constraint vectors g. Additionally, the n × n identity matrix In , blockdiag
function blkdiag (A, . . . , Z), entrywise product ◦ (also referred to as Schur or Hadamard
product) and Kronecker product ⊗ are often used.

1 ... 0
 
 .. . . .
In =  . . .. 
0 ... 1
A . . . 0ma ×nc
 

blkdiag (A, B, C ) = 
 .. .. 
. B . 
0mc ×na ... C
     
a11 a12 a1n b11 b12 b1n a11 b11 a12 b12 a1n b1n
A ◦ B =  a21 a22 a2n  ◦  b21 b22 b2n  =  a21 b21 a22 b22 a2n b2n 
     
am1 am2 amn bm1 bm2 bmn am1 bm1 am2 bm2 amn bmn
a11 B . . . a1n B
 
 .. .. 
A⊗B = . ... . 
am1 B . . . amn B

Y.C.C. Putter Master of Science Thesis

3-1 Multi-Agent System Modelling 27

3-1-2 Simplified Energy Exchange Network

Ultimately, the desire is to control the pumps and valves in the grid. This requires to have
a physical model of the pipe flow dynamics and heat transfer in terms of pressures p(t),
mass flows ṁ(t) and temperatures T (t) (see Figure 3-1). In Appendix C, a more detailed
description is given of the equations and assumptions used for a decoupled hydraulics and
thermal equations model. However, modelling any heat grid generically on this level of detail
proved to be difficult, mainly due to the following things:

Figure 3-1: Schematic of relevant variables for detailed pipe model

• The mass flow equations are nonlinear due to the operation of valves (C-7), and iterative
due to the friction factor that depends on the mass flow (and thus also on the valve
positions) (C-3).
• The temperature equation require the tracking of a variable time delay (C-12).
• The variable mass flow ṁ(t) results in a variable time delay τ (t) of heat front propaga-
tion (C-13a). No method was found that enabled to rewrite the lower bound search for
t0 (t) to solve the integral, in a constraint that fits into a MIQP model.
• The variable mass flow ṁ(t) results in changing number of required spatial sampling
points for the accurate tracking of temperatures due to the Courant-Friedrichs-Levy
criterion (C-16).

Instead, a simplified approach is chosen based on the exchange of heat energy. Pipeline dy-
namics consisting of mass flows, variable time delays and temperatures are therefore not taken
into account. Inter dependency of the optimal result with the operation of pumps and valves
are therefore also not included in this approach. It is assumed that the heat grid operation
costs are minimal to the optimal scheduling of heat energy production and consumption. Also,
optimal setpoints of the heat energy exchange are assumed to be translatable to a lower-level
optimization problem focused on the operation of pumps and valves.

Master of Science Thesis Y.C.C. Putter

28 Initial MPC Design Case Study

Constraints

The simplified heat energy exchange model represents the heat grid by a directed graph,
consisting only of nodes and supply pipes, see in Figure 3-2. Each node is represented by an
agent i (node, producer, consumers or buffer) and the edges represent the supply pipelines.
Each agent i will have their own partial input vector consisting of the heat energy exchange
h iT
variables h̃xc,i (t) = h1ij (t) h2ij (t) . . . heij (t) , where e stands for the corresponding edge
and j 6= i for the other agent involved in the edge coupling constraint.

Figure 3-2: Schematic of simplified energy exchange model for pipe dynamics for a single edge
e

The pipeline is then modelled with a global coupling constraint (3-4a), that couples the heat
energy exchange variables of two connected agents hij (t) and hji (t). The energy loss factor and
constant heat energy transport time delay for each pipe or edge is modelled with respectively
αe and τe . Also, for each agent i, upper and lower bound constraints on the heat energy
exchange variables are needed. Here, the pre-defined direction of the grid is used to appoint
which agent is sending (hm M m M
ij = −ce and hij = 0) or receiving (hij = 0 and hij = ce ).

heji (t) + (1 − αe )heij (t − τe ) = 0 ∀e, ∀t = k . . . k + Np − 1 (3-4a)

h̃m M
xc,i ≤ h̃xc,i (t) ≤ h̃xc,i ∀i, ∀t = k . . . k + Np − 1 (3-4b)

3-1-3 Node

A Node agent (i = N ) is the most basic system element that is modelled, which only considers
the Kirchhoff relation where in- and outflow of heat energy should sum to zero. In the OOP
code setup, it also serves as the parent from which other agents inherit most of the variable
structure and methods. Therefore, the nested variable vector shown below for the Node agent,
is also to be found at the end of all other agents’ nested variable vectors ũi (k).
h iT
ũN (k) = h̃T
xc (k) | s̃T (k) (3-5)

where,
h iT
h̃xc (k) = h1ij (t) . . . heij (t)
h iT
s̃(k) = s1 (k) . . . sns (k)

Y.C.C. Putter Master of Science Thesis

3-1 Multi-Agent System Modelling 29

Here, the energy exchange variable group h̃xc (k) consists of all the required energy exchange
variables to model the pipe connections with other agents, as mentioned in the previous sub-
section of the simplified heat energy exchange model. The variable s̃(k) is used to transform
inequality constraints into equality constraints. Furthermore, each agent has additional pa-
rameters that specify the initial (3-6), minimum and maximum values for each of its variables,
denoted with the superscripts 0, m and M.

ũi (0) = ũ0i (3-6)

where
h iT
ũ0N = h0,1
ij . . . hij0,e | s01 . . . s0ns =0
h iT
m,1
ũm
N = hij . . . hm,e
ij | sm
1 . . . sm
ns =0
h iT
M,1
ũM
N = hij . . . hM,e
ij | sM
1 . . . sM
ns =0

Cost Function

For each variable of an agent i, separate linear ũri and quadratic ũR
i cost parameter vectors are
used to construct the objective cost functions Ji (k) (3-7). These cost parameters are constant
for all Np variable vectors.

Ji (k) = ũri ũi (k) + ũT R

i (k)ũi ũi (k) (3-7)
where
h i
ũrN = hr,1
ij . . . hr,e r r
ij | s1 . . . sns =0
h i
R,1
ũR
N = hij . . . hR,e R . . . sR = 0
ij | s1 ns

For each agent, no constant costs are involved on the slack variables to ensure that the
inequality constraint is properly modelled by the equality constraint. Also, no constant
costs are used for the heat exchange variables, as it assumed that a minimization of the
consumers’ energy imbalance e(k) and the producers’ operating costs is sufficient for proper
model behaviour in the centralized model predictive control (CMPC) setup. For the HMPC
setup explained later on, the ADMM works entirely on an iteration process, which determines
shared penalty costs ỹ(q) and λ̃(q) for each prediction time step t = k . . . k + Np − 1 of
each edge’s global coupling constraint. This in turn translates to a penalty cost for the
corresponding coupled variables in h̃xc,i (k) of all agents, with the goal to achieve the same
optimization result as the CMPC approach.

Constraints

The constraints for a Node agent also apply to all other agents, possibly with a variation.

Master of Science Thesis Y.C.C. Putter

30 Initial MPC Design Case Study

Nodal Balance For each agent, a variation on the nodal balance constraint is used that
models the . For a node, the Kirchhoff energy flow balance equation translates in a constraint
where the sum of in- and outflowing heat energy should be equal to zero, as it does not have the
ability to produce, consume or store energy. Remember that the global coupling constraints
already properly account for the energy loss and time delay due to heat energy transportation,
i.e., the inflowing exchange variables are therefore not affected in nodal balance equation of
an agent.
X
h̃xc (t) = 0 ∀t = k . . . k + Np − 1 (3-8)

Heat Energy Exchange Capacity Also, minimum h̃m M

xc and maximum h̃xc constraint parame-
ters are used to limit the range of the heat energy exchange variables, related to the maximum
energy flow capacity of the edge e it is connected to (3-4b).

h̃m M
xc ≤ h̃xc (t) ≤ h̃xc ∀t = k . . . k + Np − 1 (3-9)

Slack Variables For each agent i, the slack variable group s̃(k) is used to transform all
inequality constraints to equality constraints (3-10), which is not required for the Gurobi
Optimization Solver but can be helpful when heuristic optimization methods are to be imple-
mented in the future. Here, ns denotes the number of elementary inequality constraints (thus
non-repeated for Np prediction horizon), that require such a slack variable. For the sake of
readability, the inequality constraints are shown in the remainder of the thesis, instead of the
rewritten equality constraints.

a(k) ≤ 0 is equivalent to a(k) + s1 (k) = 0 s.t. s1 (k) ≥ 0 (3-10)

Furthermore, the slack variables are only lower bound constrained.

0 ≤ s̃(t) ≤ inf ∀t = k . . . k + Np − 1 (3-11)

3-1-4 Producer

For a Producer agent (i = P ), it is assumed that two separate heat production units h̃(k) are
available, where one is cheaper to operate than the other. Additionally, the on-off status of
the primary and secondary heat production units are denoted by the binary variables ṽ(k).
Lastly, the startup and shutdown of both production units are captured with the binary
ramping variables ṽramp (k).
h iT
ũP (k) = h̃T (k) | ṽ T (k) T
| ṽramp (k) | h̃T
xc (k) | s̃T (k) (3-12)

where,
h iT
h̃(k) = hp (k) hs (k)
h iT
ṽ(k) = vp (k) vs (k)
h iT
ṽramp (k) = vp,up (k) vp,down (k) vs,up (k) vs,down (k)

Y.C.C. Putter Master of Science Thesis

3-1 Multi-Agent System Modelling 31

This could for instance be used as a very simple model of a combined heat and power (CHP)
plant, that generates electricity and heat energy form its primary heat production unit hp (k)
and has the ability to fire up a gas-fired boiler as a secondary heat production unit hs (k)
in case of high demand. This heat demand is indirectly captured through its heat energy
exchange variables h̃xc (k). A brief discussion on other producer modelling considerations is
included in Appendix B.

Cost Function

The cost function is a combination of production costs and startup and shutdown costs.

JP (h̃(k), ṽramp (k)) = Cproduction (h̃(k)) + Cup,down (ṽramp (k)) (3-13)

Cproduction (h̃(k)) = h̃T (k)h̃R h̃(k) + h̃r h̃(k) + ṽ r ṽ(k)
r
Cup,down (ṽramp (k)) = ṽramp ṽramp (k)
where
h i
ũrP = h̃r | ṽ r r
| ṽramp | 0 | 0
ũR
P =0

Literature typically uses second order polynomial production costs because it fits in their
MIQP problem setup, not because it is necessary to model the actual real production costs.
In [29], the quadratic production cost and exponential startup costs are approximated with
PWA functions using auxiliary variables. In [19] and [24], the profit from selling electricity
generated by the CHP is also included. Although the current producer code setup allows
these things to be modelled in the future, in this case the production costs are assumed to
be affine (linear in h̃(k)) by setting h̃R = 0, and electrical power generation coupling and
profits are disregarded in the scope of this thesis. Lastly, the startup and shutdown costs
could represent real costs, but a fictive cost can always be used to give an incentive during
optimization to prefer solutions where startup and shutdown occur less frequently.

Constraints

Nodal Balance For the Producer agent, a variation on the nodal balance constraint is used
that models the Kirchhoff energy flow balance equation. In this instance, it includes the heat
energy production terms.
X
hp (k) + hs (k) + h̃xc (t) = 0 ∀t = k . . . k + Np − 1 (3-14)

Hybrid Production Capacity Both heat production variables h̃(k) are coupled to their re-
spective binary on/off status variables ṽ(k). Therefore, hybrid capacity limit constraints
(3-15) are used with the parameters 0 ≤ h̃m ≤ h̃M . Hybrid refers to dependence of the
continuous operating range on the state, which in this case is the on/off status.
(h̃m + ) ◦ ṽ(k) ≤ h̃(k) ≤ h̃M ◦ ṽ(k) ∀t = k . . . k + Np − 1 (3-15)

Master of Science Thesis Y.C.C. Putter

32 Initial MPC Design Case Study

To ensure that the hybrid capacity limit constraint relations (3-16) are properly modelled
during optimization,  is introduced. Only when hm m
p = 0 or hs = 0, then  is a small positive
nonzero scalar 0 <  << minimum(h̃m p, h̃m
s ). However this solution does have a small
drawback, as the magnitude of  will affect the outcome of the optimization. Also, keep in
mind that these lower bounds are only used during construction of the constraint matrices.
During optimization, the lower bounds need to be set to 0, otherwise it is not possible for the
production units to turn off.

h i
h̃m (. . . + ) ≤ h̃(k) ≤ h̃M
s ⇐⇒ [ṽ(k) = 1] (3-16a)
h i
h̃(k) = 0 ⇐⇒ [ṽ(k) = 0] (3-16b)

Hybrid Production Ramping Hybrid ramping constraints (3-17) are used with the param-
eters h̃down ≥ 0 and h̃up ≥ 0 that represent how much the production of heat is allowed
to decrease and increase in a single time step. Here, the lower and upper bounds γdown (k)
and γup (k) for the hybrid ramping constraints depend on the on/off state and whether it is
starting up or shutting down. This allows the cases when h̃up < h̃m (startup transition) and
h̃down < h̃m (shutdown transition).

−γ̃down (k) ≤ h̃(k) − h(k − 1) ≤ γup (k) ∀t = k . . . k + Np − 1 (3-17)

where

γ̃down (k) = (h̃m + h̃down ) ◦ ṽramp,down (k) + h̃down ◦ ṽramp,up (k)

γ̃up (k) = h̃m ◦ ṽramp,up (k) + h̃up ◦ ṽramp,up (k)

Binary Transient Coupling The binary ramping variable subgroups ṽramp,up (k) and ṽramp,down (k)
are used to model the startup (ṽ(k − 1) = 0 → ṽ(k) = 1) and shutdown (ṽ(k − 1) = 1 →
ṽ(k) = 0) transitions of the corresponding binary on/off status variable ṽ(k).

ṽ(k) − ṽ(k − 1) − ṽramp,up (k) + ṽramp,down (k) = 0 ∀t = k . . . k + Np − 1 (3-18a)

ṽramp,up (k) + ṽramp,down (k) ≤ 1 ∀t = k . . . k + Np − 1 (3-18b)

Minimum On (Off) Time The minimum on (off) or up (down) time constraints (3-19)
[29] can be imposed with the parameters ṽ UT ≥ 2, ṽ DT ≥ 2 for each binary on/off status
variable. For brevity’s sake, equation (3-19) only shows the constraints for the minimum up
time for a single on/off status variable v. The minimum down time constraints can be found
by substituting v UT = v DT , v U0 = v D0 and using 1 − v instead of v. Naturally, v can be
substituted for any of the binary on/off variables within ṽ(k).

Y.C.C. Putter Master of Science Thesis

3-1 Multi-Agent System Modelling 33

vU1
X
[1 − v(k)] = 0 (3-19a)
k=1
t+v UT
X
[v(n)] ≥ v UT [v(t + 1) − v(t)] ∀t = v U1 + 1 . . . k + Np − 1 − v UT (3-19b)
n=t+1
k+Np −1
X
[v(n) − (v(t − 1) − v(t))] ≥ 0 ∀t = k + Np − v UT + 2 . . . k + Np − 1 (3-19c)
n=t+1

Here, the first minimum on (off) constraint rule (3-19a) looks at how long it has previously
been on v U0 (off v D0 ) and how many timesteps it still has to remain on v U1 (off v D1 ) to
satisfy the minimum on (off) constraint. It is therefore the only constraint rule that results
in a variable constraint matrix EvUT,1 (k). The second constraint rule (3-19b) models all v UT
long consecutive time spans that fit within the Np prediction horizon time span. The third
constraint rule (3-19c) models all periods with a time span shorter than v UT , that end at the
last predicted timestep. This means that for these final timesteps, v may be on (off) for a
shorter time span than v UT (v DT ) as long as it is consecutively on (off). This is a relaxed
variation on the harder constrained implementation found in [29], where the final timespans
must be as long as the minimum on (off) time.
As a side note, it is important to mention that for heuristic optimization methods such as
[31], the first constraint rule (3-19a) needs to be removed at a optimization timestep k, if
it results in a constraint matrix EvUT,1 (k) consisting of only zeros (i.e., the first v UT time
steps are unconstrained). This is because it will lead to an extremely or even infinitely high
condition number of the overall constraint matrix, which negatively impacts the quality and
speed a heuristic method is able to find an optimal solution.

3-1-5 Consumer

For a Consumer agent (i = C), the heat energy imbalance error e(k) represents the difference
between its heat demand (defined positively) and the sum of heat energy exchange (which for
a consumer is the total inflow).
h iT
ũC (k) = eT (k) | h̃T
xc (k) | s̃T (k) (3-20)

where,
1 2
Qn e cn (trange −tn )
X
hdemand (t) = ∀t ∈ trange (krange , ts , Np , τ M ) (3-21)
n

The perfect deterministic heat demand profile is generated over the entire discretized (with
time step ts = 1 h) time range trange . This time range includes the timesteps necessary for
initial values (which is a function of the maximum time delay between two agents τ M ), and
for the prediction horizon Np at the last optimization time step of krange . It is modelled with
an exponential function (3-21) [26]. Here, n is an arbitrary number that represents how many
peaks are modelled. For each peak n, the width of the peak cn , the peak time tn and the

Master of Science Thesis Y.C.C. Putter

34 Initial MPC Design Case Study

peak magnitude Qn are required. If cn = ∞, then a constant heat demand with magnitude
Qn is modelled.

For this heat demand profile (3-21), two main assumption were made to ensure that unex-
pected optimization results during design were only an effect of mistakes made in the math-
ematical modelling of the agents, and the translation of these equations to the constraint
matrices and vectors for both the CMPC and HMPC implementations.

1. The heat demand prediction used during each optimization time step is assumed to
be perfect, meaning that there is no error between the prediction demand profile used
during optimization and the actual heat demand.
2. A deterministic demand profile without uncertainty is modelled, meaning that the op-
timization method also does not have to take into account any uncertain demand pre-
dictions during optimization, as is done in [23], [24] and [25].

As long as the other agents parameters are chosen relative to the heat demand to test certain
demand scenarios, using realistic demand profiles (in terms of absolute magnitude) is not a
necessity to asses controller performance. However, it must be revisited when the heat grid
is modelled physically in terms of mass flows and temperatures and as heat demands are
uncertain by nature this is ideally improved as soon as possible (see Appendix A). In [33],
Deerns’ Low Energy Architecture (LEA) model is explained that takes a physical approach to
model the heat demand of a building. This model takes into account weather data, building
characteristics, the building use profile and indoor climate settings.

Cost Function

The cost function is then only a quadratic function of the imbalance e(k), and not necessarily
also of the heat exchange variables as mentioned in the Node’s cost function section 3-1-3.

JC (k) = er e(k) + eT (k)eR e(k) (3-22)

where

ũrC = 0
h i
ũR
C = ẽR | 0 | 0

The reason that a quadratic cost term is required, is that using only a linear (or piece-wise
affine (PWA)) cost function on the energy imbalance ẽr > 0, ẽR = 0 leads to unwanted
results in all scenarios with multiple consumers. This is due to the constant cost gradient of
a linear cost function, it does not matter how much the energy imbalance deviates from the
origin. In other words, in a situation where the heat supply cannot meet the combined heat
demand, a linear (or PWA) cost function leads to undesired indifference between supplying
all consumers equally with shared supply resources, or only one with all supply resources.

Y.C.C. Putter Master of Science Thesis

3-1 Multi-Agent System Modelling 35

Constraints

Nodal Balance For the Consumer agent, a variation on the nodal balance constraint is
used that models the Kirchoff energy flow balance equation. In this instance, it includes the
heat energy consumption of the predicted heat demand (defined positively) and the energy
imbalance e(k) ≥ 0 which is lower bound constrained to only also allow the situation where
the heat demand cannot be met by the supply.
X
h̃xc (t) + e(t) = hdemand (t) ∀t = k . . . k + Np − 1 (3-23)

In some studies, the nodal balance is set as a hard constraint excluding an energy imbalance
variable all together, but this is under the assumption that demand can always be exactly
met by the supply. On a side note, if e(k) should represent the absolute imbalance, then the
inequality equations (3-24) are needed instead.
X
eabs (t) + h̃xc (t) ≤ hdemand (t) ∀t = k . . . k + Np − 1 (3-24a)
X
eabs (t) − h̃xc (t) ≤ hdemand (t) ∀t = k . . . k + Np − 1 (3-24b)

3-1-6 Buffer

For a Buffer agent (i = B), the only added variable is the explicitly tracked predicted state of
the buffer hb,k+1 (k), of which the dynamics are represented with a linear time invariant (LTI)
state space system (3-26). The predicted buffer state represents the energy storage level and
hηb,k+1 is the thermal energy storage efficiency coefficient of the buffer to model heat energy
loss over time. h iT
ũP (k) = hT
b,k+1 (k) | h̃ T (k) | s̃T (k)
xc (3-25)
where,
 
hb,k+1 (k) = hηb,k+1 hb,k+1 (k − 1) +
X
h̃xc (k) (3-26)

It could also be implicitly tracked by measuring the state hb (k|k) at the current time step
k, and use the LTI dynamics equation to calculate the trajectory for future time steps (3-
28). However, this approach is not chosen as it complicates the construction of the nodal
balance constraints for every agent with a buffer (see Chapter 4) and the possibility to add a
cost function on the predicted buffer state in future development. Instead, with the explicit
tracking of the buffer state, the optimal solution of the predicted state at the previous timestep
k − 1 is used as the buffer state hb (k|k) at the current timestep k (3-27).
hb (k) = h∗b,k+1 (k|k − 1) (3-27)

For completeness sake, the standard LTI system dynamics equations are included below, to
show how the predicted evolution of the buffer state can be calculated implicitly with only
information about the state at the current timestep k and the control inputs at all timesteps,
which are here the heat energy exchange variables.
l
X
hb (k + 1 + l|k) = Al+1 hb (k|k) + Al−i B h̃xc (k + i|k) ∀l ∈ [0 . . . Np − 1] (3-28)
i=0

Master of Science Thesis Y.C.C. Putter

36 Initial MPC Design Case Study

where
A = hηb,k+1
B = hηb,k+1 I 1×number of elements in h̃xc (k)

Which in matrix notation looks like

      
hb (k + 1|k) A1 B 0 ... 0 h̃xc (k|k)

 hb (k + 2|k)  
  A2 


 A1 B B ... 0 

h̃xc (k + 1|k) 

.. = ..  h (k|k) +  .. .. ..   ..

 b ..  

 .
 
  . 

 . . . . 

.


hb (k + Np |k) AN p ANp −1 B ANp −2 B . . . B h̃xc (k + Np − 1|k)

Cost Function

There could be a cost for energy stored in the buffer (on state hb,k+1 (k)), but for now it is
reasoned that the buffer storage efficiency parameter hηb,k+1 already functions as a mechanism
to influence the degree in which the buffer is used. Therefore, the cost term on the state is
omitted.

JC (k) = hrb,k+1 hb,k+1 (k) + hT R

b,k+1 (k)hb,k+1 hb,k+1 (k) (3-29)

where

ũrC = 0
ũR
C =0

Constraints

Nodal Balance For a Buffer agent, a variation on the nodal balance constraint is used
that models the Kirchhoff energy flow balance equation. In this instance, the node now has
the possibility to store and retrieve energy from the buffer. Therefore, the constraint is also
capturing the predicted buffer dynamics at the same time including the heat storage efficiency
factor hηb,k+1 .

hb,k+1 (k) = hηb,k+1 (hb,k+1 (k − 1) +

X
h̃xc (k)) ∀t = k . . . k + Np − 1 (3-30)

Buffer Capacity Furthermore, the minimum and maximum energy storage in the buffer is
constrained by the parameters 0 ≤ hm M
b,k+1 ≤ hb,k+1 .

hm M
b,k+1 ≤ hb,k+1 (t) ≤ hb,k+1 ∀t = k . . . k + Np − 1 (3-31)

Y.C.C. Putter Master of Science Thesis

3-2 Optimization Problems 37

3-2 Optimization Problems

3-2-1 CMPC

The overall general CMPC MIQP optimization problem for each timestep that is optimized
k ∈ krange , is then simply a combination of the Np expanded cost functions Ji (k), constraints
matrices Ẽi (k) and vectors g̃i (k) and variable vectors ũi (k) of all agents within the entire
system, such that every agents’ agentgroup number ng = 1. Because all inequality constraints
are translated to equality constraints with the help of slack variables, (3-32b) can be dropped
as well as the superscipts that denote the difference between the inequality and equality
constraint matrices and vectors.
X X X
min Jng (θ̃ng (k)) = Ji (ũi (k)) = JC (ũi (k)) + JP (ũi (k)) ∀k ∈ krange (3-32a)
θ̃ng (k) ∀i ∀i=C ∀i=P
s.t.
Ẽnineq
g
θ̃ng (k) ≤ g̃nineq
g
(k) (3-32b)
Ẽneqg (k)θ̃ng (k) = g̃neqg (k) (3-32c)
Ẽxc,ng θ̃ng (k) = g̃xc,ng (k) (3-32d)

where
h i
θ̃ng (k) = ũT
P1 (k) | ... | ũT
N1 (k) | ... | ũT
C1 (k) | ... | ũT
B1 (k) | ...
 
Ẽng = blkdiag ẼP1 (k), . . . , ẼN1 , . . . , ẼC1 , . . . , ẼB1 , . . .
h iT
T (k), . . . , g̃ T, . . . , g̃ T, . . . , g̃ T, . . .
gng = g̃P1 N1 C1 B1

UT,1
Most constraint vectors and a single constraint matrix EP,v (k) (see section 3-1-4) are dy-
namic, meaning that these need to be updated after each optimization time step. However,
the other constraint matrices and vectors can be generated and stored once offline before the
optimization, and then retrieved during the optimization to save computing resources (see
Algorithm 1).
The optimization problem is MIQP because both continuous and binary variables are used
(mixed-integer), quadratic cost functions are used for the consumers, and all constraints are
mixed-integer linear. The non-convexity of the optimization problem, where there are multiple
optimal solutions instead of one, is due to the minimum on/off time integer constraints of the
producer, covered in the total inequality constraints (3-32b). Also, it can be non-convex due
to a poor choice of linear cost parameters, where for instance both production units of the
producer are equally weighted. Furthermore, the global equality constraints are separated
from the total combined equality (and inequality) constraints of all agents, as they span
the heat energy exchange variables h̃xc,i (k) of all agents. Lastly, to let the optimization
problem know that only the first time step of the optimal solution for each variable is actually
implemented, all other predicted time steps are additionally weighted with a factor 0.9 on
their corresponding (linear and quadratic) cost parameter.

Master of Science Thesis Y.C.C. Putter

38 Initial MPC Design Case Study

3-2-2 HMPC with ADMM

As these global coupling constraints are a function of the h̃xc,i (k) between two coupled agents
for each edge, primal and dual decomposition techniques [34] could be used to construct a
non-centralized MPC approach. Primal decomposition (resource allocation) is better suited
for problems with coupling variables, e.g., when two agents share the same variable. Here,
it is possible to achieve feasible solutions at every iteration [35]. Dual decomposition (price
coordination) is better suited for problems with coupling constraints, as is the case in this
MPC optimization problem setup.
However, standard primal and dual decomposition methods are only applicable for convex cost
functions and constraints (where strong duality holds). ADMM is an augmented Lagrangian
(dual decomposition) method, that works around this strict convexity requirement and will
therefore be used instead, as the MPC optimization problem setup is non-convex and has
coupling constraints. More specifically, the (unscaled) Optimal Exchange problem formulation
is used as starting point, which is a simplified version of the generalized Sharing Problem
formulation that can be solved with ADMM [27].
In this thesis, the coupling constraints of the Optimal Exchange problem formulation are im-
proved such that the higher level coordinator has knowledge of the connectivity matrices Ki
of each agent, which is essentially a separation of the total global coupling constraint matrix
Ẽxc and vector g̃xc (k) per agent’s heat energy exchange variables h̃xc,i (k). The HMPC con-
troller therefore also includes the same information about transportation losses and constant
time delays for each edge. Of course, it could be possible for the agents to have knowledge
over their own connectivity matrix, but this approach is not chosen. This is because it is
expected that a more detailed network model (preferably based on temperatures and mass
flows) results in far more complex global coupling constraints which are likely depended on
the grid valves and pumps. As discussed in Section 2-5, the operation of these grid devices
are preferably not assigned to a single agent, but instead to a higher level coordinator that
has full knowledge of the heat grid.

Y.C.C. Putter Master of Science Thesis

3-2 Optimization Problems 39

ADMM Interpretation & Considerations

With the q-iterations of the ADMM procedure (see Algorithm 2), the optimization problem
is thus decomposed in sub optimization problems per agent(group), where each agent(group)
minimizes their own (combined) cost function, subject to their own (combined) constraints,
and using only their own (combined) variable vector (3-34). However, for the ADMM method
the agent specific cost function is now expanded with two ADMM cost functions that only
apply to the agent’s energy exchange variables h̃xc,i (k). The first added ADMM cost function
(3-34b) is comparable to the locational pricing principle of locational marginal pricing (LMP),
and the second cost cost function (3-34c) is a proximal regularization term, that gives the
agent a direction on how to adjust its h̃xc,i (k) to agree with the other coupled agent on their
specific global coupling constraints.
Once all agent(group)s have found their optimal solution, only their energy exchange variables
∗(q)
h̃xc,i (k) are sent to the higher level coordinator that updates the dual variables ỹ (q+1) (k)
and λ̃(q+1) (k) (3-35) (e.g., variables that are introduced during the dual decomposition of
the original primal problem consisting of the primal variables). These dual variables are
essentially iteratively updated cost parameters used in the two added ADMM cost functions.
Once these dual variables are updated by the higher level coordinator, they are sent back to
the agent during the next (q = q + 1) iteration, such that the procedure is repeated. The
iterative information exchange of variables between the agents and the higher level coordinator
is what refers to the Alternating Direction part of ADMM. Eventually this iterative procedure
stops, when the ỹ (q) (k) has converged below a certain threshold value q  > 0(≈ 0) or when
the maximum allowed number of iterations q M is reached (to prevent getting stuck due to
oscillating behaviour in finding an optimal solution). Also, a minimum number of iterations
is imposed with q m > 1.
Here, ỹ (q) (k) (3-35a) is a measure of the global coupling constraint error for each participating
∗(q−1)
agent, with the previous set of optimal h̃xc,i (k) of all agent(group)s, and for each edge as
well as for the entire Np prediction horizon. Important to note, in our network implementa-
tion, only two agents are coupled to each other via an edge, which is why the factor 21 is used.
This implies that with this equation of ỹ (q) (k), each agent is incentivized to solve half of the
global coupling constraint error. The second dual variable λ̃(q) (k) (3-35b) is essentially an
integrator on ỹ (q) (k) with stepsize ρ. This can be interpreted as the locational price of LMP,
but with the difference that it is the locational price for each edge instead of each node.
The tuning of this ADMM parameter ρ, will affect the number of iterations and oscillating
behaviour of the convergence. Increasing ρ increases the speed at which the dual variables con-
verge, but increase the chance of oscillating convergence behaviour. An auto-tuning method
based on heuristics that is largely indifferent on the initial setting of the parameter ρ is
described in [36].

Master of Science Thesis Y.C.C. Putter

40 Initial MPC Design Case Study

Algorithm 1: MPC Optimization

Data: Global constraint matrices Ẽxc,ng , Ki
1 begin
2 for k = krange do
3 foreach agentgroup ng do
4 Warm Starting of nested group optimization vector θ̃n0g (k), where for each
single variable the previous optimal Np vector is shifted with one time step
k and appended with a 0
h i
h0p (k) = h∗p (k + 1|k − 1) . . . h∗p (k + Np − 1|k − 1) 0

5 end
6 foreach agent i do
7 Update the constraint matrices Ei (k) and vectors gi (k), where
Agent.Update only takes the first time step of the previously calculated
optimal control sequence. Also, static matrices and vectors stored in
memory are retrieved during this update.
h i
Ẽi (k), g̃i (k) = Agent.Update (ũ∗i (k|k − 1))

Combine the agents’ Np expanded variable vectors as well as the global

coupling constraints matrices and vectors that belong within the same
agentgroup
 
Ẽng (k) = blkdiag Ẽ1 (k), . . . , Ẽi (k), Ẽxc,ng
h iT
g̃ng (k) = g̃1T (k), . . . , g̃iT (k), g̃xc,n
T
g
(k)

8 end
9 if all (ng == 1) then
10 CMPC optimization
 

θ̃n∗g (k) := argmin Jng (θ̃ng (k)) =

X
Ji (ũi (k)) (3-33)
θ̃ng (k) ∀i∈{ng =1}

11 s.t.
(3 − 32b), (3 − 32c) and (3 − 32d)

12 else
13 HMPC optimization with ADMM, see Algorithm 2
14 end
15 Retrieve the agents’ optimal control sequence ũ∗i (k) from θ̃ng
∗ (k).

16 Store ũ∗i (k) for Agent.Update() method during next k-iteration.

17 end
18 end

Y.C.C. Putter Master of Science Thesis

3-2 Optimization Problems 41

Algorithm 2: HMPC optimization, q-iterations for ADMM algorithm for single

timestep k
Data: ADMM parameters ρ, q m , q M , q e
1 begin
2 Initialize q = 1, apply warm-start for ỹ 1 (k) and λ̃1 (k) (see Algorithm 1)
 
while q < q m or (q < q M and any ỹ (q−1) (k) > q  ) do
 
3
4 foreach Agentgroup ng and Agent i do
• Using fixed dual variables ỹ (q) (k) and λ̃(q) (k), each agentgroup
solves its local optimization problem with added ADMM cost functions.

• This requires each agent to update its constant agent cost vector
with the additional ADMM cost-terms.

θ̃n(q+1)
g
(k) := argmin (Jng (θ̃ng (k), h̃xc,ng (k)) . . . (3-34a)
θ̃ng (k),h̃xc,ng (k)

+λ̃(q)T (k)K̃ng h̃xc,ng (k) . . . (3-34b)

ρ   2
+ ỹ (q) (k) + K̃ng h̃xc,ng (k) − h̃(q) (k) ) ) (3-34c)

xc,ng
2 2

subject to Ẽng (k)θ̃ng (k) = g̃ng (k)

 
where K̃ng = blkdiag K̃1 , . . . , K̃i

5 end
6 for Higher-level Coordinator do

• Receives only h̃q+1

xc,i (k) from all agents i (of all agentgroups ng ).

• Update the ADMM dual variables y (q) (k) and λ(q) (k).

1X (q+1)
ỹ (q+1) (k) = Ki h̃xc,i (k) (3-35a)
2 i
λ̃(q+1) (k) = λ̃(q) (k) + ρỹ (q+1) (k) (3-35b)

7 end
8 Update q = q + 1
9 end
10 Assign optimization results of last q-iteration as optimal solution.

θ̃n∗g (k) = θ̃nq g (k) (3-36)

11 end

Master of Science Thesis Y.C.C. Putter

42 Initial MPC Design Case Study

3-3 Setup of Initial Design Case Scenario

For the initial MPC design case, a scenario was chosen with one producer, one node, three
consumers and one buffer agent. The connections between the agents can be seen in Figure 3-3.
The purpose of this fictive heat grid was to test if the MPC controller implementations could
show controller performance or behaviour that corresponded with pre-defined expectations.
These expectations are based on parameter settings for the optimization, topology and all
agents. For the optimization parameters, a prediction horizon of Np = 8 time steps is taken,
where each time step is 1 hour, and only a single day is simulated. For the ADMM parameter
settings ρ = 0.4, q  = 0.005 were used and q m = 3 and q M = 50 for the scenario without time
delay and q m = 5 and q M = 200 with time delay. Furthermore, for the HMPC controller only
the producer is in a different agentgroup from the rest.

Figure 3-3: Schematic topology of initial design case used for designing the MPC controllers.
The agents are 1 producer, 1 node, 1 buffer and 3 consumers where the edges or pipelines are
denoted with ei . The predictive models of the agents belong to the corresponding agentgroups
ng . For CMPC, all are in the same agentgroup ng = 1 and no higher level coordinator is used.
For HMPC, only the producer is in a different agentgroup from the rest. Here, the higher level
coordinator of the ADMM algorithm is only concerned with the global coupling constraint for
edge e1

Network, Node and Buffer Parameters

Each consumer is connected directly (via the node) to the producer, and indirectly (via the
buffer). The maximum heat energy flow capacity from the node to each consumer is capped
(cM M M
2 = c3 = c4 ≈ 11 MW) such that only a base heat demand can be supplied. The other
edge capacities, (cM M M M M
1 , c5 , c6 , c7 , c8 ) are unconstrained. The loss factors αe = 0.1 are equal
for each edge. The constant time delays τe are disregarded at first, after which a constant
time delay is modelled only for the edge between the producer and the node τ1 = 2 h. The
buffer efficiency parameter is set at hηb,k+1 = 0.95 to model minor heat loss over time and can
be considered unconstrained in maximum storage level hM b,k+1 . The node has no parameter
settings different from those that were already mentioned in its modelling section (3-1-3).

Y.C.C. Putter Master of Science Thesis

3-3 Setup of Initial Design Case Scenario 43

Figure 3-4: Detailed agent topology of initial design case showing all variables for all agents. Each
agent is represented by a black box, where the nodal balance can be determined by the direction
of the variable arrows. If it point towards the agent’s node, then it is added. However, for the heat
exchange variables the conventional arrow direction is shown, with the solid lines representing the
actual edge flow direction and the dashed lines representing hij (k) ≤ 0. Furthermore, all heat
exchange
 variables are shown with
 the agent names,
 instead of their corresponding node numbers
P N C1 C2 C3 B = 1 2 . . . 6 .

Producer Parameters

Below are the variable vector, initial values, linear cost parameters, quadratic cost parameters
and special agent parameters for the producer.
h iT
ũP1 (k) = h̃T (k) | ṽ T (k) T
| ṽramp (k) | h̃T
xc (k) | s̃T (k)
h h i h i h i iT
ũ0P = 0 0 | 1 1 | 1 0 1 0 | 0 | 0
h h i h i h i i (3-37)
ũrP = 5 7 | 3 3 | 1 1 1 1 | 0 | 0
ũR
P =0

where
h i
P̃iparameters = h̃m , h̃M , h̃up , h̃down , ṽ UT , ṽ DT
h iT h iT h iT h iT h iT h iT 
P̃1parameters = 0 0 38 14 10 20 10 20 3 3 3 3

Master of Science Thesis Y.C.C. Putter

44 Initial MPC Design Case Study

Consumer Parameters

Below are the variable vector, initial values, linear cost parameters, quadratic cost parame-
ters and special agent parameters for the consumers. Important to note is that the energy
imbalance error is only lower bounded em = 0.
h iT
ũC (k) = eT (k) | h̃T
xc (k) | s̃T (k)
h iT
ũ0C = 0 | 0 | 0
h i (3-38)
ũrC = 0 | 0 | 0
h i
ũR
C = 25 | 0 | 0
where
h i
C̃iparameters = h̃Q c t
demand , h̃demand , h̃demand
h iT h iT h iT 
C̃1parameters = 8 5 inf 5 0 13
h iT h iT h iT 
C̃2parameters = 9 5 inf 6 0 14
h iT h iT h iT 
C̃3parameters = 10 20 5 inf 1 7 0 8 15

Desired Controller Behaviour

With these parameters, the following controller behaviour is expected to test the results:

• The main goal is to supply the requested heat demand, thus minimize the ẽi (k).
• All consumers should be equally supplied, relative to their magnitude of heat demand
due to the equal quadratic cost parameter eR
i = 25 ∀i = C.

• The direct lines should be preferred over the indirect lines, as the latter consist of a
path of 3 edges instead of 2 which results in a greater heat loss over the entire path.
• The primary heat production unit h̃s (k) should always be prioritized in suppling heat
demand, as it is the cheapest. However, it maximum heat production is not enough to
provide more than the combined base heat demand.
• The secondary heat production unit h̃s (k) should only turn in to help supplying the
peak heat demand. However it is chosen such that the producer’s combined maximum
heat production is insufficient for combined peak heat demand of all consumers.
• Therefore, the buffer stored energy is required to help supplying the peak heat demand
due to the producer’s shortage of maximum combined heat production.
• The buffer should be loaded just before the peak heat demand, to minimize the destruc-
tion of stored heat energy over time due to the buffer efficiency.
• The minimum on (off) constraints of the producer should not be violated.
• The HMPC controller performance should be identical to the CMPC controller.

Y.C.C. Putter Master of Science Thesis

3-4 Results and Discussion 45

3-4 Results and Discussion

Before results are discussed, the interpretation of the main figures is explained (e.g., Figure 3-
5).

3-4-1 Interpretation of Result Figures

For each of these type of figures, each agent is displayed in its own subfigure. In each agent’s
subfigure, the first predicted time step of each variable’s optimal solutions for all time steps
is shown ũ∗i (k) ∀k ∈ krange . The driving force behind the optimization is of course the heat
demand of each consumer agent, which is always displayed as a black solid line. Please notice
the difference in magnitude of the y-axis between the the agents. Heat demand parameters
were chosen such that the lowest and highest heat demand profiles are assigned to respectively
the first and third consumer. Please also notice that the heat energy exchange variables hij (k)
and hji (k) of two coupled agents are shown with the same color and marker. Furthermore,
the solid and dashed lines are used to make a comparison of optimal solutions between for
instance two controllers or two scenarios. If the optimal trajectories are exactly equal, then
both lines coincide and only the solid line remains visible.

3-4-2 CMPC vs REF controller

Desired CMPC Controller Behaviour

Firstly, in Figure 3-5, it can be seen that the CMPC controller solutions (solid) shows the
described desired controller behaviour, which tells us that the models are properly constructed
and updated. Also, the heat energy transportation loss and maximum capacity of the edges
are properly taken into account at each agent. However, it should be noted that the minimum
on time for the secondary heat production unit (hs (k)) is violated at the 8th hour. A closer
look reveals that this is not actually the case, and instead hs (8) = , which is a example of
the expected drawback from the hybrid capacity constraint modelling.

Similar Optimal Error Trajectories

Secondly, in Figure 3-5 as well, a comparison between the CMPC controller (solid) is made
with a reference controller (dashed) that optimizes over the entire time span all at once,
e.g., a prediction horizon that covers the entire optimization time span krange or trange . It is
assumed that this controller is able to find the best optimal solution, as it has full knowledge.
The comparison is made to see if the receding horizon principle of the CMPC creates big
differences in the optimal solution it finds at each timestep. It is noticed that there is virtually
no difference in each of the consumer’s energy imbalance e(k) trajectory. Also, there’s no
difference in how the heat demand is being supplied over time via both the direct and indirect
heat energy exchange variables of each consumer.

Master of Science Thesis Y.C.C. Putter

46 Initial MPC Design Case Study

Figure 3-5: Results comparison of CMPC (solid) vs Reference CMPC (dashed) results, no time
delay

Y.C.C. Putter Master of Science Thesis

3-4 Results and Discussion 47

Different Optimal Heat Production Trajectories

Although most of the trajectories at the other agents are also similar, a few interesting
differences can be seen. The most relevant differences are seen in both the primary hp (k) and
secondary heat hs (k) production trajectories at the producer. For hp (k), a difference is seen at
the third hour. This results in a higher heat exchange h1,2 (k) to the node, which in turn sends
it to the buffer h2,6 (k). Here, the buffer state hb (k) trajectory for the reference controller leads
one timestep, until the 8th hour where the first peak demand occurs at the third consumer.
Because the third consumer’s direct heat exchange variable h5,2 (8) = 10 = hM 5,2 is already at
its maximum capacity, the peak has to be handled via the indirect line via the buffer h5,6 (k).
Looking at the buffer state, because the reference case has stored more energy in advance,
it requires less heat from the producer at that time step compared to the CMPC controller
(solid). The CMPC controller makes this up with a higher hs (k) because hp (k) is already at
its maximum production.

Between the 13th and 17th hour, only a difference in hs (k) can be seen. Here, the areas
under the trajectories seem equal. This time, the reference controller makes use of the buffer
to be able to shut off earlier. However, it is unexpected from the reference controller that
it starts loading the buffer earlier to a higher state, as this results in slightly more overall
heat loss due to buffer the efficiency. To investigate, the optimal solution costs for both
controllers are compared in Figure 3-6 which show that the optimal solution costs for the
CMPC is surprisingly slightly lower than the reference controller. This could be an effect of
the non-convexity as well as the fact that the CMPC optimizes more often and makes use of
the warm-start each timestep, which could push it towards a different solution.

Figure 3-6: Cost comparison of CMPC (solid) vs REF (dashed)

Master of Science Thesis Y.C.C. Putter

48 Initial MPC Design Case Study

3-4-3 CMPC vs HMPC controller

In Figure 3-7, a comparison between the CMPC (solid) and HMPC (dashed) controllers is
made to test whether the HMPC produces equal or similar results. The buffer state hb (k)
trajectories and primary heat production hs (k) trajectories are mostly identical. However,
the consumers’ energy imbalance error is less optimal compared to CMPC.
For the secondary heat production unit hs (k), differences are seen around the 2nd, 7th, 10th
and 13th hour. For the 7th hour, the same hybrid coupling capacity constraint drawback
occurs but the CMPC decides that the beginning time step should be equal to , whereas the
HMPC decides that this should occur at the end. Because of the latter, the HMPC is required
to stay off for three time steps afterwards which explains the 13th hour (which still falls in the
prediction horizon when t = 7 h). A closer look on the coupled on off binary variable vs (k)
during the q-iterations (see Figure 3-9), show that the same solution of CMPC was found at
the 8th, 9th and 10th q-iteration, but that it jumped back to this current solution.
Furthermore, the producer’s heat exchange variable h1,2 (k) at t = 0 h (or k = 2) is equal, while
the receiving node shows a difference. This implies that the ADMM algorithm has not yet
converged, but was required to stop due to the maximum allowed q-iterations (see Figure 3-
8). However, physically this means that the pipe flow is not modelled correctly. Moreover,
oscillating behaviour can be seen (Figure 3-8)) in the convergence of the dual variable y(k+2).
Both are an example of the importance of tuning the ADMM stepsize parameter ρ correctly.
Although it could have been chosen to use a different ρ for each prediction time step, this
would only make the initial tuning more cumbersome.

Y.C.C. Putter Master of Science Thesis

3-4 Results and Discussion 49

Figure 3-7: Results comparison of CMPC (solid) vs HMPC (dashed) results, no time delay

Master of Science Thesis Y.C.C. Putter

50 Initial MPC Design Case Study

Figure 3-8: The ADMM q-iterations of the ADMM dual variables and coupled binary status for
k=2 (t = 0 h). On the left side, the Np predictions for the dual variables y(k) and λ(k) are
shown. On the right, the coupled binary variables of the producer vp (k) and vs (k) are shown. It
can be seen that y(k) still has not converged after the maximum 50 q-iterations as well as the
oscillating behaviour y(k + 2) for the k+2 (t = 2 h) predicted timestep. This can also be seen
at vs (k + 2).
Y.C.C. Putter Master of Science Thesis
3-4 Results and Discussion 51

Figure 3-9: The ADMM q-iterations of the ADMM dual variables and coupled binary status for
k = 7 (t = 5 h). Shows that vs (k + 2) finds the same solution of CMPC at the 8th, 9th and 10th
q-iteration for (turning on at t = 7 h), but that it jumped back to its ending solution (turning on
at k+3, e.g., t = 8 h).

Master of Science Thesis Y.C.C. Putter

52 Initial MPC Design Case Study

3-4-4 CMPC vs Time Delayed CMPC

In Figure 3-10, a CMPC optimal solution comparison is made between a scenario without
(solid) and with (dashed) a time delay between the producer and node agent τ1 . It is noted
that the time delay is properly modelled in the global coupling constraints by looking at the
producer’s h1,2 (k) and node’s h2,1 (k) energy exchange variable for the first few time steps.
Moreover, the same can be seen by looking at the consumers’ energy imbalance error e(k)
that is equal to the heat demand for the first two timesteps, due to an empty buffer hb (k) = 0
and the time delay in heat exchange from the node h2,j=C (k) = 0.
Also, when the producer is up and running you can clearly see that the secondary heat
production hs (k) has almost the exact same profile around the 8th hour, but shifted exactly
a time delay difference. Due to the minimum off time, it is required to stay off for three time
steps, which is one of the indicators why it cannot compute the same optimal solution (but
simply time shifted with the time delay) as for the scenario without the time delay.
Furthermore, quite different trajectories in terms of the consumers’ energy imbalance error
ei (k) and buffer state hb (k) are to be seen, where the time delayed scenario seems to result
in larger errors. However, build ups in the consumers’ energy imbalance error ei (k) can be
noticed before each peak heat demand.

Y.C.C. Putter Master of Science Thesis

3-4 Results and Discussion 53

Figure 3-10: CMPC without time delay (solid) vs CMPC with a time delay τ1 = 2 between the
producer and node (dashed).

Master of Science Thesis Y.C.C. Putter

54 Initial MPC Design Case Study

3-4-5 Time Delayed CMPC vs Time Delayed HMPC

In Figure 3-12 (and 3-11, 3-13), a comparison between the CMPC (solid) and HMPC (dashed)
is made when for both the same scenario with time delay is used. The first thing to notice
is that the time delay is properly modelled for HMPC controller as well, by looking at the
same things noticed in the previous comparison. This means that the constant connectivity
matrices K̃i (k) and (update required) constraint vectors are computed and updated correctly.
Furthermore, it is important to note that the maximum number of q-iterations was increased
to 200, because the convergence took longer overall. This has to do with the fact that for the
first two predicted time steps, the node has to solve for his energy exchange variable h2,1 (k)
while the producers energy exchange h2,1 (k − 2) is already fixed (e.g., unaffected by the dual
variables because the energy was sent two time steps earlier). Moreover, it is surprising to
see that the HMPC controller actually succeeds better in minimizing the consumers’ energy
imbalance error ei (k).

Figure 3-11: Cost comparison of CMPC (solid) vs HMPC (dashed) for time delayed scenario.
First time steps should be disregarded because the ADMM dual variables have not converged
enough.

Y.C.C. Putter Master of Science Thesis

3-4 Results and Discussion 55

Figure 3-12: CMPC vs HMPC, both time delayed

CMPC (solid) vs HMPC (dashed) comparison, with a time delay τe1 = 2 between the
producer and node.

Master of Science Thesis Y.C.C. Putter

56 Initial MPC Design Case Study

Figure 3-13: The ADMM q-iterations of the ADMM dual variables and coupled binary status
variables for k = 19 (t = 16 h), showing large oscillating behaviour.

Y.C.C. Putter Master of Science Thesis

 Chapter 4

WarmCo Case Study

The European initiative Multi Utility Providing (MUP) is a systems concept that aims to
match the residual and waste commodity streams of industry processes such as CO2, water,
heat and electricity between companies, through an underground pipe system to reduce the
use of other transportational modalities. This is also known as smart-linking and the concept
shares many of the characteristics of a 4th Generation District Heating (4GDH) system. The
MUP concept is part of the European program Ports Adapting To Change (PATCH) and also
applied in the zone that connects the ports of Borsele, Vlissingen and Terneuzen. In this zone
a coll, the main stakeholders are 12 energy and resource intensive companies that form the
Smart Delta Resources and the horticulture in Zeeuws Vlaanderen, that are part of Biopark
Terneuzen as well as the, see Figure 4-1.

Figure 4-1: Schematic of the port zone and the (color-coded) smartlinks [37]. For clarity, the
fertilizer manufacturer Yara Sluiskil, energy company WarmCO2 and the horticulture have been
highlighted in respectively red, orange and green on the right most platform.

Master of Science Thesis Y.C.C. Putter

58 WarmCo Case Study

For the horticulture, the use of a CHP system or conventional gas-fired boiler is often not
economically efficient as the energy costs are around 30% of the total operational expenditures
(OPEX) due to high gasprices. The horticulture tries to reduce their OPEX, but also to
improve the sustainability of their energy demand. This is done by transitioning away from
fossil fuels and instead focus on alternative solutions such as geothermal heat, waste heat and
reducing heat loss with for example the use of dual shading screens.
At fertilizer manufacturer Yara Sluiskil, the production of predominately nitric acid and
ammonia also results in waste heat and CO2 as byproducts which are normally cooled and
and emitted in the environment. WarmCO2 is a small scale energy company that originated
from the MUP concept and is a joint initiative by Yara and port authority North Sea Port,
both part of the Smart Delta Resources platform. WarmCO2 takes this waste heat and CO2
from Yara and delivers it to the nearby horticulture in the zone. The price of the waste heat for
the horticulture is substantially lower compared to the use of conventional installations such
as a gas fired boiler or combined heat and power (CHP) system. Also, not being dependent
on natural gas as a heat source means that the uncertainty of the gas prices is no longer
relevant. Instead, long-term contracts ensure a low fixed price for the horticulture.

4-1 Overview of Heat Grid Characteristics

In Figure 4-2, a schematic is shown of the heat grid of WarmCO2 that runs between Yara
and the horticulture. From now on, the waste heat production of Yara and the buffer and
cooler of WarmCO2 are referred to as the producer and each connected greenhouse company
as a consumer.

Figure 4-2: Schematic of the WarmCO2 ’s heat grid connecting Yara and the horticulture.

The producer has a thermal buffer of 51 MW and two heat production capacities, one with
a constant output of 40 MW coming from the and a controllable output of 36 MW. The

Y.C.C. Putter Master of Science Thesis

4-2 Case Study Setup 59

transport of heat is done via a heat grid that start at circa 90o C and loses temperature due
to heat exchanger efficiencies and transportation supply pipeline where it cools down to 83o C
before it reaches the heat exchanger of the last consumer.
It also has a cooling capacity consisting of two separate stages; air cooling (14.2 MW) that
brings down the temperature from 87 to 66o C and water cooling (19 MW) to bring it down
from 66 to 38o C at a maximum flow rate of 600 m3 /h. They respectively consist of 6 and
5 cooling units, each equipped with 2 and 3 ventilators driven by separate motors of which
one has a variable rotation speed (frequency modulation control) and the rest have a fixed
rotation speed. For the air cooling specifically, each cooling unit also has controllable venting
louvers that can be opened or closed on a 15 second time-scale to either maximize or minimize
the heat exchange with the outside air.
The consumers connected to the heat grid have a heat demand and a thermal buffer. Via the
climate control computer, a consumer communicates a requested heat demand over a time
period with WarmCO2 such that the heat grid pumps and valves can be operated accordingly.
Furthermore, if the requested heat demand is above a certain breakpoint value, the consumer
has to pay a higher price to compensate for the extra load on the producer.

Optimization goals

The incentives to optimize the matching between heat demand and supply are different for
the stakeholders and operating scenarios. During periods of high heat demand (winter), it
is in the consumers interest to use their buffers as efficient as possible to stay in the lower
price tariff for the requested heat demand. The higher price tariff is meant as compensation
for the additional OPEX made for using the second controllable production unit. It is paid
regardless of whether the second controllable production unit is actually used or not. It is
therefore also in the producer his interest to use his thermal buffer as efficiently as possible,
such that it can handle the total consumer demand with only the constant heat production.
During periods of low heat demand (summer), the producer could have a surplus of heat
supply when the constant heat production is larger than the total heat demand, and the
buffer storage is already at its limit. In this case, the cooling units needs to be used, resulting
in additional OPEX for the producer. It is therefore in his interest to use its thermal buffer,
and possibly also those of the consumers, as efficient as possible to minimize the use these
cooling units.

4-2 Case Study Setup

In this setup, the main characteristics of the case study are translated to fit within the
assumptions of the existing framework already built in Chapter 3. Two new agent types
are first introduced, the ProducerBufer and the ConsumerBuffer. Next, the focus lies on
how the previously stated optimization goals of the case study for the two different heat
demand scenarios can be achieved by the model predictive control (MPC) and decentralized
or distributed model predictive control (DMPC) approach. Also, for the DMPC approach,
each consumer will be regarded as a single agent, increasing the number of heat exchange
variables that are communicated with the higher level coordinator. Lastly, the effect of

Master of Science Thesis Y.C.C. Putter

60 WarmCo Case Study

introducing a fixed nominal time delay and a mismatch between prediction and actual heat
demand is used, to investigate how this effects the optimization. In Figure 4-3, a schematic
is shown of the setup and all of the important variables per agent.

Figure 4-3: Setup of the WarmCO2 ’s heat grid, with the main continuous variables per agent
besides the coupling heat exchange variables hxc (k). Positive flow direction is denoted by the solid
lines for the coupling heat exchange variables hxc (k). The producer has a primary constant heat
production hp (k), secondary variable heat production hs (k), air cooling hc1 (k), water cooling
hc2 (k) and a thermal buffer hb (k). Each consumer has a thermal buffer hb (k) and heat demand
hd (k).

4-2-1 ProducerBuffer

For the ProducerBuffer agent (in this chapter also referred to as the producer), the complete
variable vector is shown below.
h i
ũPB (k) = hb,k+1 (k) | h̃(k) | ṽ(k) | ṽramp (k) | vz (k) | h̃xc (k) | s̃(k) (4-1)

where,
h i
h̃(k) = hs (k) hc1 (k) hc2 (k)
h i
ṽ(k) = vs (k) vc1 (k) vc2 (k)
h i
ṽramp (k) = vs,up (k) vs,down (k) vc1,up (k) vc1,down (k) vc2,up (k) vc2,down (k)

Cost Function

The cost function is a combination of production, cooling and startup and shutdown costs.
As mentioned in Section 3-1-4, only linear cost parameters are used, and the startup and
shutdown costs could represent real costs, a fictive cost can always be used to prefer optimal

Y.C.C. Putter Master of Science Thesis

4-2 Case Study Setup 61

solutions where startup and shutdown occur less frequently.

JP B (k) = Cproduction (hs (k)) + Ccooling (hc1 (k), hc2 (k)) + Cup,down (ṽramp (k))
Cproduction (h̃(k)) = hR hs (k)2 + hrs hs (k) + vsr vs (k) (4-2a)
Ccooling (h̃(k)) = hR 2 r r
c1 hc1 (k) + hc1 hc1 (k) + vc1 vc1 (k) (4-2b)
+ hR 2 r
c2 hc2 (k) + hc2 hc2 (k) + r
vc2 vc2 (k)
r
Cup,down (ṽramp (k)) = ṽramp ṽramp (k) (4-2c)

where
h i
ũrPB = hrb,k+1 (k) | h̃r (k) | ṽ r (k) | ṽramp (k) | 0 | 0 | 0
ũR
PB = 0

Constraints

Most constraints are similar to those of the Producer agent discussed in Section 3-1-4.

Nodal Balance However, with the addition of the buffer hb,k+1 (k) for the producer, the
nodal balance changes where the sum of production, cooling, in and outflow is now allowed
to be nonzero due to the capacity to store heat in or retrieve heat from the buffer. The same
modelling assumptions for the buffer also apply here (see Section 3-1-6). Again, because the
global constraints already take into account direction of flow and thereby line losses properly,
the in- and outflows hxc (k) can be simply added or subtracted from the buffer level, without
having to adjust for line losses in this nodal balance equation (4-3).
 X 
hb,k+1 (t) = ηb hb,k+1 (t) + hp (t) + hs (t) − hc1 (t) − hc2 (t) + h̃xc (t) (4-3)
∀t = k . . . k + Np − 1

Buffer Capacity The buffer capacity is constrained by the parameters 0 ≤ hm M

b,k+1 ≤ hb,k+1 .

hm M
b,k+1 ≤ hb,k+1 (t) ≤ hb,k+1 ∀t = k . . . k + Np − 1 (4-4)

Buffer (Dis)charge Furthermore, the buffer discharge is constrained, or the rate at which
up
heat can be extracted from (hdown
b,k+1 ≥ 0) or added to (hb,k+1 ≥ 0) the buffer. It is similar to
the up and down ramping constraints of the production variables h̃(k).
up
−hdown
b,k+1 ≤ hb,k+1 (t) − hb,k+1 (t − 1) ≤ hb,k+1 ∀t = k . . . k + Np − 1 (4-5)

Production Capacity Heat production is split into two parts, the uncontrollable primary
heat production hp (k) and the controllable secondary variable heat production hs (k). The
uncontrollable primary heat production is essentially implemented as a thermal heat supply
profile similar to the exponential equation for the heat demand profile of a consumer (see

Master of Science Thesis Y.C.C. Putter

62 WarmCo Case Study

Section 3-1-5) to allow for different profiles to be loaded. In this case, a flat profile is created
by using the following specific parameters Qn = 40 MW and cn = ∞, where tn is irrelevant.
1 2
Qn e cn (trange −tn )
X
hp (t) = ∀t ∈ trange (krange , ts , Np , τ M ) (4-6)
n

The secondary (controllable) variable heat production hs (k) is coupled to its binary on/off
status vs (k). Therefore, hybrid capacity limit constraints are used with the parameters 0 ≤
hm M m
s ≤ hs . Again, only when hs = 0 then  is a small positive nonzero scalar.

(hm M
s + )vs (t) ≤ hs (t) ≤ hs vs (t) ∀t = k . . . k + Np − 1 (4-7)

Production Ramping Also, hybrid ramping constraints (4-8) are used with the parameters
hds ≥ 0 and hus ≥ 0. Again, the lower and upper bounds γdown (k) and γup (k) for the ramping
constraints are variable due to their hybrid nature and are dependent on the binary variables
vs (k), vs,down (k) and vs,up (k).

−γs,down (t) ≤ hs (t) − hs (t − 1) ≤ γs,up (t) ∀t = k . . . k + Np − 1 (4-8)

where

γs,down (k) = (hm d d

s + hs )vs,down (k) + hs vs (k)
γs,up (k) = hm u
s vs,up (k) + hs vs (k)

Binary Transient Coupling The binary variables vup (k) and vdown (k) denote the ramping
of the corresponding binary on/off status variable v(k), to represent startup and shutdown.

ṽ(k) − ṽ(k − 1) − ṽramp,up (k) + ṽramp,down (k) = 0 ∀t = k . . . k + Np − 1 (4-9a)

ṽramp,up (k) + ṽramp,down (k) ≤ 1 ∀t = k . . . k + Np − 1 (4-9b)

Minimum On (Off) Time Also, the same minimum on and off time constraints as mentioned
in (3-19) can be imposed with the parameters ṽ UT ≥ 2, ṽ DT ≥ 2 for each binary on/off status
variable. Again, in (4-10), only the constraints for the minimum up time are shown. The
minimum down time constraints can be found by substituting v UT = v DT , v U0 = v D0 and
using 1 − v instead of v. Here, v can be substituted for any of the binary on/off variables
within ṽ(k), so they can also apply to the cooling.

vU1
X
[1 − v(k)] = 0 (4-10a)
k=1
t+v UT
X
[v(n)] ≥ v UT [v(t + 1) − v(t)] ∀t = v U1 + 1 . . . k + Np − 1 − v UT (4-10b)
n=t+1
k+Np −1
X
[v(n) − (v(t − 1) − v(t))] ≥ 0 ∀t = k + Np − v UT + 2 . . . k + Np − 1 (4-10c)
n=t+1

Y.C.C. Putter Master of Science Thesis

4-2 Case Study Setup 63

Hybrid Cooling Capacity, Ramping and Minimum On (Off) Time A continuous cooling
range is considered for both the total air cooling capacity hc1 (k) and total water cooling
capacity hc2 (k), where total refers to the combined effort of all cooling units. This assumption
is made due to the variable rotation speeds of one of the ventilator engines for the air and
water cooling units. And also because the venting louvers of the air cooling units can be
controlled on a time step scale of 15 seconds whilst the optimization time step is on a hourly
time step scale. Specifically for this case study, this means that if the cooling is turned on,
the capacity limit parameters are hm m M M
c1 = hc2 = 0, hc1 = 14.2 and hc1 = 19 MW. Notice that
cooling is also defined positively, due to the choice of heat flow direction for the nodal balance
(4-3) as can be seen in Figure 4-3. Then, for the cooling variables (hc1 (k) for air cooling and
hc2 (k) for water cooling) the same constraints for the hybrid capacity (4-7), hybrid ramping
(4-8), and minimum on and off times (4-10) are applied.
(hm M
c1 + )vc1 (t) ≤ hc1 (t) ≤ hc1 vc1 (t) ∀t = k . . . k + Np − 1 (4-11a)
(hm
c2 + )vc2 (t) ≤ hc2 (t) ≤ hM
c2 vc2 (t) ∀t = k . . . k + Np − 1 (4-11b)
−γc1,down (t) ≤ hc1 (t) − hc1 (t − 1) ≤ γc1,up (t) ∀t = k . . . k + Np − 1 (4-11c)
−γc2,down (t) ≤ hc2 (t) − hc2 (t − 1) ≤ γc2,up (t) ∀t = k . . . k + Np − 1 (4-11d)

Hybrid Successive Cooling Next, an approximation is made to model the fact that the air
and water cooling units function at successive temperature ranges, as explained previously.
The water cooling unit should only be allowed to turn on (vc2 (k) = 1), if the air cooling unit is
running at maximum cooling capacity (hc1 (k) = hM c1 ). To model this in general, the following
logical statement is used that can be transformed into two mixed-integer linear inequality
constraints [30].
(
f (x(k)) ≤ M (1 − δ(k))
[f (x(k)) ≤ 0] ⇐⇒ [δ(k) = 1] (4-12a)
f (x(k)) ≥  + (m − )δ(k)
where
m = max f (x(k)) (4-12b)
x(k)∈X

M = min f (x(k)) (4-12c)

x(k)∈X

Here, f (x(k)) is a function of a continuous variable x(k) ∈ R, δ(k) ∈ {0, 1} is a binary variable
and  in this context is a small positive scalar ( ≈ 0), typically the machine precision, to
transform the original non strict inequality [f (x(k)) < 0] into a strict inequality [f (x(k)) ≤ 0].
However, it is assumed that in practice the case 0 < f (x(k)) <  cannot occur due to
the finite number of bits used for representing real numbers on a computer. In the actual
implementation, this  was set to 0 without any noticeable problems, but it is still shown in
(4-12) and (4-13) for completeness sake. The two inequality constraints can be rewritten such
that only the variables are on the left side.
f (x(k)) + M δ(k) ≤ M (4-13a)
−f (x(k)) + (m − )δ(k) ≤ − (4-13b)

Now, theh logical statement

i needed to model the successive use of the air and water cooling
reads as hc1 (k) ≥ vzbp ⇐⇒ [vz (k) = 1], where the breakpoint value for the auxiliary binary

Master of Science Thesis Y.C.C. Putter

64 WarmCo Case Study

variable to turn on is set to vzbp = hMc1 . This can then be substituted into the generalized
constraints 4-13 to result in the actual constraints used.

−hc1 (t) + (−hm bp m

c1 + vz )vz (t) ≤ −hc1 ∀t = k . . . k + Np − 1 (4-14a)
hc1 (t) + (−hm
c1 + vzbp − )vz (t) ≤ vzbp − ∀t = k . . . k + Np − 1 (4-14b)

By using a separate parameter for the breakpoint value hm bp M

c1 ≤ vz ≤ hc1 , it is possible to allow
the water cooling to turn on earlier. The reason that the logical statement uses an additional
auxiliary binary variable vz (k) instead of the on/off status of the water cooling vc2 (k), is to
allow the situation for hm
c2 > 0. However, this then requires one more constraint to couple
the auxiliary binary variable vz (k) and the water cooling on/off status vc2 (k) according to
a converse implication relation vz (k) ←− vc2 (k), which is shown in the logical truth table
(Table 4-1).

vz (t) − vc2 (t) ≤ 0 ∀t = k . . . k + Np − 1 (4-15)

Table 4-1: Logical truth table (converse implication) for the successive cooling constraint

vz (k) vc2 (k) vz (k) ←− vc2 (k)

1 1 T
1 0 T
0 1 F
0 0 T

Y.C.C. Putter Master of Science Thesis

4-2 Case Study Setup 65

4-2-2 ConsumerBuffer

For the ConsumerBuffer agent (in this chapter also referred to as the consumer), the nested
variable vector is shown below.
h i
ũCB (k) = hb,k+1 (k) | e(k) | vz (k) | hz (k) | h̃xc (k) | s̃(k) (4-16)

For the actual heat demand profile hd (k), each consumer and scenario is based on data from
WarmCO2 concerning the heat demand of the 5 largest consumers in the time frame between
1 January 2017 and 1 January 2018. For the two scenarios of highest and lowest aggregated
heat demand (of the 5 largest consumers), a weekly profile on a hourly time scale is used.
To obtain these two heat demand profiles for each consumer, some filtering was required on
the data provided by WarmCO2 . Firstly, the data is averaged to convert it from a 5 minute
to a hourly time scale. Secondly, the days with missing hours per consumer are identified
and filtered out for all consumers. Lastly, a moving average filter is applied to this processed
dataset, to find 7 consecutive days where the aggregated heat demand is either at a maximum
or minimum value, see Figure 4-4.
One of the main assumptions of the heat demand will now be changed:

1. The heat demand prediction profile hd (k) used during optimization will now be a com-
bination of the actual hd,act (k) and requested hd,req (k) heat demand profiles as seen in
Figure 4-4). This means that a different non-perfect heat demand prediction is assumed
for the prediction horizon at each optimization time step. Here, a linear decreasing fac-
tor f Np ×1 = [1 . . . 0.5 . . . 0] is used such the actual heat demand profile is dominant in
the first half of the prediction horizon, and the requested heat demand profile at the
last half of the prediction horizon.

hd (k) = f ◦ h̃d,act (k) + (1 − f ) ◦ h̃d,req (k) (4-17)

2. Still, a deterministic demand profile without uncertainty is modelled, meaning that the
optimization method does not take into account any uncertainty bounds of the demand
predictions during optimization.

Master of Science Thesis Y.C.C. Putter

66 WarmCo Case Study

Figure 4-4: Weekly demand profiles of the 5 largest consumers used for the two different scenarios
of highest and lowest total (requested) heat demand, obtained by filtering recorded data from
WarmCO2 in the time period between 1 January 2017 and 1 January 2018.

Y.C.C. Putter Master of Science Thesis

4-2 Case Study Setup 67

Cost Function

Besides the usual quadratic cost parameter on the imbalance e(k), the heat exchange variables
are now also priced. This allows a two-tiered or piece-wise affine (PWA) cost function for heat
demand consumption which has the form of (4-18b). Here, h̃+ xc (k) denotes the sum of only
bp
the positive heat energy exchange variables (inflows) and vz is the breakpoint value where
the price jumps from r1 to r2 . Also, a linear cost is used to disfavor the 2nd tariff region.

JCB (k) = Cimbalance (e(k)) + Cimport (h̃+

xc (k)) + Ctariff2 (vz (k))
Cimbalance (e(k)) = eR e(k)2 (4-18a)
(
r1 h+
xc (k) if h+ bp
xc (k) ≤ vz
Cimport (h̃+
xc (k)) = +
(4-18b)
r2 hxc (k) if h+ bp
xc (k) > vz
Ctariff2 (vz (k)) = vzr vz (k) (4-18c)

where
h i
ũrC = 0 | 0 | vzr | r2 − r1 | r1 | 0
h i
ũR
C = 0 | ẽR | 0 | 0 | 0 | 0

Constraints

Nodal Balance As the consumer now also include a buffer hb,k+1 (k), the nodal balance
changes as it has the ability to store and extract energy from the buffer. As usual, it includes
the predicted heat demand (defined positively) and the energy imbalance e(k) ≥ 0 which is
lower bound constrained to allow situations where less heat energy is supplied than asked for.
 
hb,k+1 (k + 1) = hηb,k+1 hb,k+1 (k) − e(k) − hd (k) +
X
hxc (k) ∀t = k . . . k + Np − 1
(4-19)

Buffer Capacity The buffer capacity is constrained by the parameters 0 ≤ hm M

b,k+1 ≤ hb,k+1 .

hm M
b,k+1 ≤ hb,k+1 (t) ≤ hb,k+1 ∀t = k . . . k + Np − 1 (4-20)

up
Buffer (Dis)charge The (dis)charge is constrainted by the parameters (hdown
b,k+1 ≥ 0 ≤ hb,k+1 ).

up
−hdown
b,k+1 ≤ hb,k+1 (t) − hb,k+1 (t − 1) ≤ hb,k+1 ∀t = k . . . k + Np − 1 (4-21)

Two-Tiered Pricing The cost function of the consumer is a PWA function (4-18b) and can
be rewritten in a single linear equation by first introducing a binary variable vz (k) using the
same approach outlined in (4-12). But now the approach is applied to capture the relation
[h+ bp
xc (k) >= vz ] ⇐⇒ [vz (k) = 1] resulting in the two linear mixed integer inequality

Master of Science Thesis Y.C.C. Putter

68 WarmCo Case Study

constraints used for each consumer. Here, hm+ M+

xc and hxc denote the sum of the minimum and
maximum parameter values of only the positive heat exchange variables (inflows).

−h+ m+ bp
xc (k) + (−hxc + vz )vz (k) ≤ −hm+
xc ∀t = k . . . k + Np − 1 (4-22a)
h+ M+ bp
xc (k) + (−hxc + vz − )vz (k) ≤ vzbp −  ∀t = k . . . k + Np − 1 (4-22b)

Now the cost function can be written as

J(h+ + +
xc (k), vz (k)) = r1 hxc (k) + (r2 − r1 )vz (k)hxc (k) (4-23)

Lastly, the product of the binary and continuous variable at the end of (4-23) can be further
simplified by introducing another (continuous) auxiliary variable hz (k). To model this in
general, 4 mixed-integer linear inequality constraints are needed [30], where f (x(k)) is a
function of a continuous variable x(k) ∈ R, δ(k) the binary auxiliary variable and z(k) the
continuous auxiliary variable.



z(k) ≤ M δ(k)

z(k) ≥ mδ(k)

z(k) = δ(k)f (x(k)) (4-24a)
z(k) ≤ f (x) − m(1 − δ(k))



z(k) ≥ f (x) − M (1 − δ(k))



where
m = max f (x(k)) (4-24b)
x(k)∈X

M = min f (x(k)) (4-24c)

x(k)∈X

Rewriting such that all variables are on the left hand side, and substituting (4-23) in (4-24)
results in the constraints and cost objective used for each consumer.

J(h+ +
xc (k), vz (k), hz (k)) = r1 hxc (k) + (r2 − r1 )hz (k) (4-25a)
s.t.
−hM+
xc vz (k) + hz (k) ≤ 0 ∀t = k . . . k + Np − 1 (4-25b)
hm+
xc vz (k) − hz (k) ≤ 0 ∀t = k . . . k + Np − 1 (4-25c)
+ m+
−hxc (k) − hxc vz (k) + hz (k) ≤ −hm+xc ∀t = k . . . k + Np − 1 (4-25d)
h+ M+
xc (k) + hxc vz (k) − hz (k) ≤ M+
hxc ∀t = k . . . k + Np − 1 (4-25e)

4-3 Setup of WarmCo Case Scenarios

For the WarmCO2 case study, the purpose of the more realistic heat grid was to test how
the hierarchical model predictive control (HMPC) controller deals with a considerably larger
prediction horizon, more agents affected by the alternating direction method of multipliers
(ADMM) algorithm, and with non-perfect heat demand predictions. Most of the constraint
modelling was already verified to work properly in the previous chapter, so only the additional

Y.C.C. Putter Master of Science Thesis

4-3 Setup of WarmCo Case Scenarios 69

constraints have to be tested. Of course, in general the optimization goals stated in Section 4-
1 should be met. However, it is hard to measure optimality differences without a reference
controller (or recorded performance data) to compare it to. Therefore, a centralized model
predictive control (CMPC) reference controller with the same prediction horizon is used, but
with the difference that it assumes a perfect heat demand prediction (hd (k) = hd,act (k).

Also, due to non-disclosure agreement (NDA) contracts signed between WarmCO2 and the
horticulture, detailed information about for instance the breakpoint value for the prices agreed
upon were not available. Instead, the breakpoint values were based on the hectares of the each
greenhouse consumer. The same applied to the size of the buffers, the buffer efficiency and
the up and down ramping parameters. The prediction horizon of Np = 24 time steps is taken,
where each time step is 1 hour, and the chosen summer and winter weeks are simulated. For
the ADMM parameter settings ρ = 0.4, q  = 0.005, q m = 10 and q M = 500. Furthermore, all
agents are in a different agentgroup except for the Producer and Node.

Network, Buffer and Node Parameters

Each consumer is connected directly (via the node) to the producer (see Figure 4-3). The
maximum heat energy flow capacity from the node to each consumer is capped at (cM 2 =
cM
3 = cM = cM = cM ≈ 44 MW). The maximum edge capacity from the producer to the
4 5 6
node is taken, (cMe1 = 100 MW). The loss factors αe = 0.1 are equal for each edge. Only a
constant time delay is modelled for the edge between the producer and the node τe1 = 4 h.
The buffer efficiency parameter of the producer hηb,k+1 = 0.995 is set slightly higher than
those of the consumers hηb,k+1 = 0.975, assuming that the larger buffer is more efficient. For
each greenhouse consumer, the maximum rate at which heat can be extracted hdown b,k+1 from or
added to hupb,k+1 their buffer is based on the hectares of the respective greenhouse consumer.
M up
The maximum buffer capacity hb,k+1 = 8hb,k+1 is assumed to be 8 hours times the maximum
rate at which heat that can be extracted from the buffer. The node has no parameter settings
different from those that were already mentioned in its modelling section (3-1-3) and will be
excluded from the main result figures.

ProducerBuffer Parameters

Below are the variable vector, initial values, linear cost parameters, quadratic cost parameters
and special agent parameters for the producer.

h iT
ũPB (k) = hb,k+1 (k) | h̃(k) | ṽ(k) | ṽramp (k) | vz (k) | h̃xc (k) | s̃(k)
h h i h i h i iT
ũ0P = 0 | 0 0 0 | 1 0 0 | 1 0 1 0 1 0 | 0 | 0 | 0
h h i h i h i i
ũrP = 0 | 7 3 5 | 1 1 1 | 1 1 1 1 1 1 | 0 | 0 | 0
ũR
P =0
(4-26)

Master of Science Thesis Y.C.C. Putter

70 WarmCo Case Study

where
parameters
h i
up η
P˜B i = h̃m h̃M h̃down h̃up ṽ DT ṽ UT hm M down
b,k+1 hb,k+1 hb,k+1 hb,k+1 hb,k+1
            
0 38 10 10 3 3
parameters
P˜B 1 = 0 14.2 100 100 3 3 0 408 51 51 0.995
            
0 19 100 100 3 3

ConsumerBuffer Parameters

Below are the variable vector, initial values, linear cost parameters, quadratic cost parameters
and special agent parameters for the consumers. Initially, the energy imbalance error is lower
bounded to em = 0.
h iT
ũCB (k) = hb,k+1 (k) | e(k) | vz (k) | hz (k) | h̃xc (k) | s̃(k)
h iT
ũ0CB = 0 | 0 | 0 | 0 | 0 | 0
h i (4-27)
ũrCB = 0 | 0 | 100 | 1 | 1 | 0
h i
ũR
CB = 0 | 50 | 0 | 0 | 0 | 0

where
h i
up up
C̃iparameters = ṽzbp , h̃m M down
b,k+1 , h̃b,k+1 , h̃b,k+1 , h̃b,k+1 , h̃b,k+1

˜ parameters
h i
CB 1 = 10 0 78 9.75 9.75 0.975
˜ parameters
h i
CB 2 = 17 0 111.6 13.95 13.95 0.975
˜ parameters
h i
CB 3 = 6.5 0 40.8 5.1 5.1 0.975
˜ parameters
h i
CB 4 = 5 0 32 4 4 0.975
˜ parameters
h i
CB 5 = 18 0 120.8 15.1 15.1 0.975

Desired Controller Behaviour

With these parameters, the following controller behaviour is desired to test the results:

• The main goal is to supply the actual heat demand hd,act (k), and thus minimize the
energy imbalance error ẽi (k).

• All consumers should be equally supplied, relative to their magnitude of heat demand
due to the equal quadratic cost parameter eR
i = 25 ∀i = CB.

• For each scenario, the stated optimization goals (Section 4-1) should be achieved. In
summary, during winter the consumers want to minimize the case where they are in the
second price tariff vz (k) = 1. And during summer, the producer wants to minimize the
case where the constant heat production hp (k) must be cooled with the cooling units
hc1 (k) and hc2 (k) .

Y.C.C. Putter Master of Science Thesis

4-4 Results and Discussion 71

• The water cooling unit hc2 (k) is only allowed to run when the air cooling unit is at
maximum cooling capacity hc1 (k) = hM c1 .

• The consumers’ auxiliary binary variable vz (k) should only be allowed to turn on when
(the sum of) the positive heat energy exchange variables is greater than the breakpoint
value h+ bp
xc (k) > vz .

• Each buffer is only allowed to extract and add heat to the buffer at a specified rate.

• The HMPC controller performance should preferably be identical to the reference CMPC
controller in terms of minimized energy imbalance error e(k).

4-4 Results and Discussion

Before the agent results are shown for each scenario, the two main important findings are
first discussed that affected the eventual settings of the optimization problems. These main
findings were not found during the initial design case study in the previous chapter because
now:

• the prediction horizon Np is substantially higher 24 >> 8, which resulted in a much

bigger optimization problem.
• the optimization is done over a week instead of a day, which affected the total runtime
and therefore the time it took to be able to evaluate the effect of a cost parameters
setting on the optimal solution.
• non-perfect instead of perfect heat demand predictions are used during optimization.

4-4-1 Infeasibility during Summer Scenario

In the summer scenario, the CMPC optimization model was infeasible to solve, i.e., no optimal
solution could be found at time step k = 35 when the lower bound for the error was set to
zero em = 0. This has to do with a combination of things.

• The 4 h time delay between the producer and the node causes the first 4 time steps of
the node’s incoming heat exchange variable hNP (k) to be fixed at the next optimization
time step.
• At each time step, the optimal solution is computed for a different non-perfect heat
demand prediction profile. To elaborate, the non-perfect heat demand prediction profile
at time step k + 1 is no longer simply a shifted version of the non-perfect heat demand
prediction profile at k. (4-17).
• As mentioned before in Section 4-2-2, the uncertainty of these non-perfect heat demand
predictions are not taken into account during optimization.
• The optimal solution will try to maximize the usage of the buffers in this scenario, to
take advantage of the heat loss caused by the buffer efficiency parameters hηb,k+1 .

Master of Science Thesis Y.C.C. Putter

72 WarmCo Case Study

During the 4 previous optimization time steps, the buffers were filled to such a degree that the
optimal solutions did not include a margin of error for the heat demand prediction profiles
of each ConsumerBuffer hd (k) in the next time step optimization. This resulted that the
ConsumerBuffers’ Nodal Balance constraints (4-19) were part of an irreducible inconsistent
subsystem (IIS). This means that if one of these constraints was allowed to be violated, the
optimization problem would be feasible.
Multiple approaches are possible to solve this infeasibility. One of the approaches is to detect
that the optimization has failed, after which a relaxed optimization problem is optimized
instead where the variables’ lower and upper bound, and/or the model constraints are allowed
to be violated. The penalty cost for violating one of these constraints then determines the
next best solution. For instance, one could allow the ConsumerBuffers’ maximum buffer
capacity to be violated hMb,k+1 . As a side note, starting from version 8.0, the Gurobi Matlab
interface includes this possibility where these computations were done with version 7.5.2.
Therefore, a simpler approach was taken instead where the lower bound of the Consumer-
Buffers’ energy imbalance error is allowed to be negative em = −∞ during the original opti-
mization. This would always result in a feasible optimization problem under all of the mod-
elling assumptions taken in this thesis. More importantly, the previous relaxation approach
is not chosen as the author argues that the focus should be on improving the optimization
method such that it takes into account the uncertainty during optimization, and to construct
an accurate heat demand prediction model.

4-4-2 Runtime Considerations and Observations

In the winter scenario for the CMPC controller, certain time steps suffered from ever evolving
optimization runtimes, i.e., it was stuck trying to find the global optimal solution. This
showcases the characteristics of a nondeterministic, polynomial time (NP) hard problem of
the (non-convex) mixed-integer quadratic programming (MIQP) optimization problem. More
specifically, with a prediction horizon of Np = 24, the optimization model comprised of 3192
continuous and 360 binary variables amounting to 2360 possible combinations that needed to
be evaluated in a branch-and-bound approach (Gurobi) to find the global optimal solution.
To remedy this problem, a maximum allowed computing time limit for the optimization was
set to 100 s, with the exception that at least one feasible (local) optimal solution should
always be found.
The same runtime limit was used for the CMPC and HMPC controllers, and one can argue
if this is a fair comparison. The HMPC controller would then get the same runtime limit for
each of the smaller sub optimization problems. However, this assumptions was taken, because
it is assumed that these subproblems could in theory be computed in parallel, which is one of
the main advantages of the HMPC over the CMPC approach. The HMPC then only has to
deal with additional computational overhead introduced by the iterative ADMM algorithm
of the higher-level coordinator.
In the figures below (Figure 4-5 and 4-7), the optimality gap and the required computing time
are shown for the perfect (REF) and non-perfect heat demand prediction (CMPC) controllers
at each optimization time step k. The global optimal solution is found within the allowed
runtime if the optimality gap is 0%. Of course, these runtime and optimality gap results
depend heavily (amongst other things) on the:

Y.C.C. Putter Master of Science Thesis

4-4 Results and Discussion 73

• available computing resources (Intel i5-6300 CPU at 2.40 GHz with 8GB RAM).
• expertise at which models are translated to equations and are expressed in code.
• pre-solving techniques (used by Gurobi) to reduce the set of constraints, and improve
the condition number of the constraint matrices.
• heuristic methods (used by Gurobi) to find the local optimal (or incumbent) solutions.

Convergence of ADMM y Variables

In the other figures (Figure 4-6 and 4-8), the number of q-iterations and total runtime per
agent(group) over all these q-iterations are shown for the HMPC controller. A measure for the
HMPC controller similar to the optimality gap, would be to show the ADMM dual variable
y (qend ) (k) at the last iteration, especially when the maximum number of q-iterations is reached
qend = q M = 500. However, with this setup of 5 lines affected by the ADMM algorithm and
a prediction horizon of Np = 24, there are 120 y variables to show which is hard to capture
in a single image. Instead, one could look at the main result figures comparing the CMPC
and the HMPC controllers to get a sense of how well the ADMM algorithm achieves a similar
optimality gap of the CMPC controller.
Furthermore, the main observation will be shared for the ADMM dual variables y HMPC in
case the maximum number of q-iterations was reached. What happens is that in this case, the
time delay causes the first τe y variables of the repeated elementary global coupling constraint
to be non-converged. This is because normally, two agents would solve for half of the problem
simultaneously. But with the time delay, the first τe predicted time steps are fixed, which
causes the receiving agent to converge slower than normal.

Winter Runtime Observations

In the winter scenario (maximum combined heat demand), it can be seen (Figure 4-5) from
time steps k = 22 to k = 29, that it succeeds in finding a local optimal solution close to the
global optimal solution, but in an exponentially increasing required runtime. From k = 30
to k = 35 and from k = 44 to k = 94, the optimality gap both increases and decreases
significantly reaching using up the entire maximum allowed runtime. It therefore also results
in a generally long optimization time.
Furthermore, for the HMPC controller (Figure 4-6), a similarity in number of q-iterations
and CMPC runtime is seen. Also, it can be seen very clearly that most of the time used for
optimization is by the consumers. It is thought that this is due to the fact that in the winter
scenario, each consumer’s binary variable vz modelling the two-tiered pricing mechanism is
now relevant during optimization. Note that for the HMPC approach, this results in very
long combined optimization times if computed in serial.

Summer Runtime Observations

In the summer scenario (minimum combined heat demand), it can be seen (Figure 4-7) that
the maximum allowed runtime is never used, and that computation time even stays well below
a second. Furthermore, it also finds the global optimal solution within the runtime used.

Master of Science Thesis Y.C.C. Putter

74 WarmCo Case Study

Looking at Figure 4-8, it can be clearly seen that almost all of the optimization time is used by
the producer. It is thought that this is due to fact that in the summer scenario, the producer
has to use the cooling units, which introduces the relevance of three binary variables of the
producer during optimization vc1 (k), vc2 (k) and vz (k). Additionally, it is interesting to see
that due to the producer, the HMPC approach requires much more time than the CMPC
controller.

Y.C.C. Putter Master of Science Thesis

4-4 Results and Discussion 75

Figure 4-5: Winter scenario, CMPC (solid) and reference (dashed) results for runtime and
optimality gap. Notice that the first timesteps are initialized due to the time delay, and therefore
not optimized.

Figure 4-6: Winter scenario,HMPC results of number of q-iterations and runtime per agent at
each optimization step k.

Master of Science Thesis Y.C.C. Putter

76 WarmCo Case Study

Figure 4-7: Summer scenario, CMPC (solid) and reference (dashed) results for runtime and
optimality gap.

Figure 4-8: Summer scenario, HMPC results of number of q-iterations and runtime per agent
at each optimization step k.

Y.C.C. Putter Master of Science Thesis

4-4 Results and Discussion 77

4-4-3 WarmCo Agent Results

For the agent result comparisons of CMPC vs REF (Figure 4-9 and 4-14) and CMPC vs
HMPC (Figure 4-10 and 4-15), only the most interesting time window per scenario is selected
from the entire time span. The results over the entire time span are shown directly afterwards,
but then only for a single controller (REF, CMPC and HMPC per figure. The author notes
in advance that it is highly possible that some interesting features are missed by accident and
therefore not mentioned during the evaluation of the results.

Winter Scenario

Looking at the results globally in the figures that cover the entire time span (Figure 4-11, 4-12
and 4-13), the desired controller behaviour as specified earlier is verified. Also, the trajectories
of the optimal solution of the HMPC controller are again very close to those of the CMPC
controller, as desired. One main exception that showcases undesired controller behaviour, is
that the energy imbalance error ei (k) at each consumer reaches large (possibly unacceptable)
values that increase when non-perfect heat demand prediction is applied, and is even worse
for the HMPC controller due to the iterative ADMM algorithm and fixed step-size parameter
ρ. The comparison results will therefore be discussed using the time window where the errors
are large.

Second price tariff region vz (k) In the winter scenario, the goal was to minimize the case
where consumers are in the second price tariff vz (k) = 1. It can be seen in Figure 4-9,
where the controller with non-perfect (CMPC) and perfect (REF) heat demand prediction
are compared, that the number of times the red binary line vz (k) = 1 is very limited, or
even not occurring at all (CB4). This specific result is obviously very dependent on the set
cost function parameter vzr = 100 relative to the other cost function parameters, and the
breakpoint values vzbp . Although a linear cost of 100 seems high, it should be noted that the
value vz (k) is binary and can only be 1 at maximum. Therefore, setting a quadratic cost on
this variable would also be useless.

Buffer state hb,k+1 (k) The consumer buffer trajectories hb,k+1 (k), also in Figure 4-9, are
charged right for the peak demand hd,act (k), and kept as empty as possible during low demand
to avoid heat destruction due to the loss caused by the efficiency parameter hηb,k+1 < 1.

Non-perfect vs Perfect Heat Demand Prediction The difference in the buffer (un)loading
hb,k+1 (k) trajectory (and other trajectories as well) between the two controllers (Figure 4-
9), comes from the effect of using a non-perfect heat demand prediction hd (k) for CMPC
controller. Its predicted heat demand is sort of flattened or averaged, not seeing the full
peaks ahead of hd,act (k) during optimization due to the linear combination with hd,req (k), and
therefore loading the buffers less than actually necessary (compared to the REF trajectories).

Energy Imbalance Error e(k) Furthermore, the most important thing to observe (Figure 4-9)
is that the energy imbalance error of the agents grows for both controllers to large (unaccept-
able) values at the peak demand. Again, this specific result is also very dependent on the

Master of Science Thesis Y.C.C. Putter

78 WarmCo Case Study

set cost function parameter eR i = 50, relative to the other cost function parameters. If it is
highly increased, then it is suppressed in a higher degree. However, it does clearly show an
interesting feature, where the goal to supply all consumers equally is not properly covered
with the current constraints and quadratic cost function parameter setting. Instead, they are
equally not supplied in their heat demand. This means that for a consumer with a higher
heat demand (CB2, CB5), it gets effectively more than a consumer with a lower heat demand
(CB3, CB4), for which it is possible that it receives effectively almost nothing.
To combat this, one could use custom quadratic cost parameters per agent eR i that are based
on the estimated relation of the average heat demand between the consumers. Another
possibility would be to model the energy imbalance error as a ratio of the absolute energy
imbalance error divided by the heat demand. However, this would make the optimization
non-linear in the constraints, which is something that should be avoided if possible.
It also shows the need for a non-cooperative MPC approach, because now all agents only
try to optimize for the global objective. Instead, the agents will then be able to optimize
for their own local objective, which as discussed can be different from the global objective.
This requires a big redesign of the current MPC implementation (and assumptions) where
game-theory should be taken into account.

Producer Behaviour Looking at the producer, it is noticed that the cooling units are off at
all times hc1 (k) = hc2 (k) = 0, the secondary controllable heat production unit hs (k) is almost
always at full running at full production capacity, and that the buffer hb,k+1 (k) is charged as
well to help meet in the peak heat demand of the consumers.

Y.C.C. Putter Master of Science Thesis

4-4 Results and Discussion 79

Winter Scenario CMPC vs REF Comparison

Figure 4-9: Agents’ winter scenario results comparison over selective time span, between CMPC
controllers with perfect (REF, dashed) and non-perfect (CMPC, solid) heat demand prediction.

Master of Science Thesis Y.C.C. Putter

80 WarmCo Case Study

Winter Scenario CMPC vs HMPC Comparison

Figure 4-10: Agents’ winter results comparison over selective time span, between CMPC (solid)
and HMPC (dashed) controllers both with non-perfect heat demand prediction

Y.C.C. Putter Master of Science Thesis

4-4 Results and Discussion 81

Winter Scenario REF over Entire Time Span

Figure 4-11: Agent results of REF (solid) with perfect heat demand prediction, over entire time
span of winter scenario.

Master of Science Thesis Y.C.C. Putter

82 WarmCo Case Study

Winter Scenario CMPC over Entire Time Span

Figure 4-12: Agent results of CMPC (solid) with non-perfect heat demand prediction, over entire
time span of winter scenario.

Y.C.C. Putter Master of Science Thesis

4-4 Results and Discussion 83

Winter Scenario HMPC over Entire Time Span

Figure 4-13: Agent results of HMPC (solid) with non-perfect heat demand prediction, over entire
time span of winter scenario.

Master of Science Thesis Y.C.C. Putter

84 WarmCo Case Study

Summer Scenario

Looking at the results globally in the figures that cover the entire time span (Figure 4-16,
4-17 and 4-18), the desired controller behaviour as specified earlier is verified. Also, the
trajectories of the optimal solution of the HMPC controller are again very close to those of
the CMPC controller, as desired. Here, the cooperative MPC approach works best, as there
is only a single producer, and the main optimization goals are heaviliy focussed on this single
producer.
One exception that showcases undesired controller behaviour, is that the energy imbalance
error ei (k) at each consumer reaches small negative values when non-perfect heat demand
prediction is applied, and is worse for the HMPC controller due to the iterative ADMM
algorithm and fixed step-size parameter ρ. The comparison results will therefore be discussed
using the time window where the buffers are being charged to full capacity.

Second Price Tariff Region vz (k) In the summer scenario, it should be the case that all
consumers should be able to stay within the first price tariff region vz (k) = 0. This is verified
by looking at all figures for the summer scenario, regardless of the controller or time span.

Buffer State hb,k+1 (k) The consumer buffer trajectories hb,k+1 (k), seen in for instance Fig-
ure 4-9 and 4-10, are sometimes discharged for the small peak demands hd,act (k). The in-
teresting thing that is seen is that all buffers, including the producer’s buffer, are kept at
maximum capacity to make use of the free destruction of heat energy due to the loss caused
by the efficiency parameter hηb,k+1 < 1. This is probably to limit the costs of operating the
cooling units hc1 (k) and hc2 (k) whilst coping with the uncontrollable constant heat production
hp (k).

Negative Energy Imbalance Error e(k) Furthermore, the most important thing to observe
(Figure 4-10) is that the energy imbalance error of the agents reaches small negative values
for the controllers with non-perfect heat demand prediction at the peak demand. Again, this
specific result is very dependent on the cost function parameter eR i = 50 used, relative to
the other cost function parameters. If it is highly increased, then it is suppressed in a higher
degree. Physically, a negative energy imbalance error implies that more heat is delivered
to the consumer than asked for. This effect is a result previously discussed, where the lower
bound was ignored em = −∞ to allow for feasible solution to be computed when non-perfect heat
demand predictions were used. Again, this shows the need for a stochastic MPC approach
together with an accurate model for heat demand prediction.

Producer Behaviour Looking at the producer (at for instance Figure 4-16), it is noticed
that the cooling units hc1 (k) and hc2 (k) are indeed successively turned on at all times, and
that the secondary cooling unit is kept off as often as the controllers could. The secondary
controllable heat production unit hs (k) is always turned off. Lastly, the producer’s buffer
hb,k+1 (k) is the one that is discharged to help in the small peaks instead of the consumers’
buffer.

Y.C.C. Putter Master of Science Thesis

4-4 Results and Discussion 85

Summer Scenario CMPC vs REF Comparison

Figure 4-14: Agents’ summer scenario results comparison over selective time span, between
CMPC controllers with perfect (REF, dashed) and non-perfect (CMPC, solid) heat demand pre-
diction.
Master of Science Thesis Y.C.C. Putter
86 WarmCo Case Study

Summer Scenario CMPC vs HMPC Comparison

Figure 4-15: Agents’ summer results comparison over selective time span, between CMPC (solid)
and HMPC (dashed) controllers both with non-perfect heat demand prediction

Y.C.C. Putter Master of Science Thesis

4-4 Results and Discussion 87

Summer Scenario REF over Entire Time Span

Figure 4-16: Agent results of REF (solid) with perfect heat demand prediction, over entire time
span of summer scenario.

Master of Science Thesis Y.C.C. Putter

88 WarmCo Case Study

Summer Scenario CMPC over Entire Time Span

Figure 4-17: Agent results of CMPC (solid) with non-perfect heat demand prediction, over entire
time span of summer scenario.

Y.C.C. Putter Master of Science Thesis

4-4 Results and Discussion 89

Summer Scenario HMPC over Entire Time Span

Figure 4-18: Agent results of HMPC (solid) with non-perfect heat demand prediction, over entire
time span of summer scenario.

Master of Science Thesis Y.C.C. Putter

90 WarmCo Case Study

Y.C.C. Putter Master of Science Thesis

 Chapter 5

Conclusions and Recommendations

5-1 Conclusions

In this section, the main conclusions found during the thesis are summarized and are approx-
imately sorted by order of the chapters.

5-1-1 4GDH

• 4th Generation District Heating (4GDH) systems are expected to be an important sus-
tainable heat energy system solution compared to the current gas-grid and other heat
energy system solutions such as full-electric and bio-gas filled gas grid.
• An optimal based smart controller for 4GDH is required to minimize the operational
expenditures (OPEX) and capital expenditures (CAPEX) such that 4GDH also becomes
financially competitive to the other heat energy system solutions.
• Preferably, the optimal smart controller can take into account flexibility, predictions,
stochasticity of heat demand and system predictions, physical network effects such as
variable heat eergy transport time delay, mixed-integer modelling, and a non-cooperative
and non-centralized multi-agent system (MAS) approach.

5-1-2 MPC vs HM and LMP Approach

• Currently, HeatMatcher (HM) is not able to take any network effects into account.
The proposed algorithm that relies on the current locational marginal pricing (LMP)
principle is based on an electricity flow model. This is inherently different from a heat
flow model. Also, the optimization problems are formulated as an linear programming
(LP) problem, which limits the modelling complexity.
• Furthermore, the main advantage of the model predictive control (MPC) approach is
that it optimizes over a prediction horizon, instead of a single time step which gives the

Master of Science Thesis Y.C.C. Putter

92 Conclusions and Recommendations

modelling freedom to take into account (uncertain) heat demand predictions, time-based
constraints, (variable) time delays etc.
• Here, a trade-off has to be made between the system prediction accuracy and the com-
plexity of the resulting optimization problem type that has to be (optimally) solved.

5-1-3 Initial Design Case Study

• A network model based on the decoupled hydraulic and thermal equations, results in
a non-linear time variant model that is to the author’s best of knowledge, not im-
plementable within the modelling freedom that a mixed-integer quadratic program-
ming (MIQP) problem formulation provides. Therefore the models are not based on
temperatures, mass flows and pressures.
• Instead, a simplified heat energy exchange model is assumed and implemented, which is
able to properly take into account maximum line flow constraints, linear line flow losses,
and constant heat energy transport delay for both the centralized model predictive
control (CMPC) and hierarchical model predictive control (HMPC) controllers.
• The main constraint to match heat energy supply and demand is relaxed to always find
feasible solutions, by introducing an energy imbalance error ei (k) for each agent that is
minimized. This also allows to use this MPC approach to iterate on the minimum re-
quired installation specifications during heat grid design such as the maximum capacity
per line, the buffer size and heat production unit capacities. This is beneficial in the
search for lowering the CAPEX.
• At least a quadratic cost instead of linear or piece-wise affine (PWA) cost on the energy
imbalance error is needed, to ensure that all consumers can be equally supplied.
• No cost on the buffer state hb,k+1 (k) is required, because the efficiency parameter hηb,k+1
modelling the buffer storage loss over time already functions as a mechanism to only
use the buffer slightly before peak demand, to decrease unnecessary heat destruction.
• The capacity and ramping constraints found in literature are slightly improved using
hybrid modelling techniques, to be able to more precisely model the constraints per
discrete state (on, off, starting up and shutting down). A flaw of the hybrid capacity
constraint is found.
• The minimum on and off time constraints are subdivided into three smaller constraint
sections, of which the ending sub constraints are relaxed to allow more feasible binary
solutions.
• The dual decomposition of the Optimal Exchange problem formulation using the ADMM
algorithm is used as basis, resulting in the HMPC controller that produces similar results
to CMPC controller as desired.
• The convergence and oscillating behaviour is highly dependent on the ADMM step size
parameter ρ. The higher the step size, the faster it initially converges but the higher the
chance the algorithm shows oscillating behaviour, i.e., continuously switches between
previously found local optimal solutions.

Y.C.C. Putter Master of Science Thesis

5-2 Recommendations for Future Work 93

5-1-4 WarmCo Case Study

• Hybrid modelling techniques successfully model the two-tiered cost function of a green-
house consumer, and the successive use of the cooling units.
• The combination of using non-perfect heat demand predictions, in a system with a con-
stant time delay, and a deterministic optimization approach, can result in an infeasible
optimization problem if the energy imbalance error is lower bounded (em = 0).
• A maximum runtime limit had to be imposed to ensure that the nondeterministic,
polynomial time (NP) hard problem did not get stuck trying to find the global optimal
solution by negligible improvements of the optimality gap.
• This causes, together with a nonzero time delay for a line τe > 0 affected by the ADMM
algorithm, that the first τe ADMM dual variables y are often non-converged when the
maximum allowed q-iterations is reached.
• The winter scenario shows that most computation time and effort is found at the con-
sumers, and the summer scenario shows the opposite. Furthermore, the winter scenario
takes considerably more computation time than the summer scenario, possibly caused
by the increased complexity of the now relevant binary variables of the consumers vz (k).
• The winter scenario shows that the buffers are only used for helping supply the heat
demand peaks, and the summer scenario shows that the buffers are used at maximum
capacity to make full advantage of the free heat destruction due to the buffer efficiency
parameter.

5-2 Recommendations for Future Work

In this section the main recommendations for future work are given, approximately sorted by
subject.

5-2-1 Network and Heat Demand Modelling

• As mentioned in the WarmCo case study: to obtain an accurate heat demand model
based on temperatures, occupancy, building usage etc., to improve the non-perfect pre-
diction profiles used during the (deterministic) optimization. The better the accuracy,
the smaller the error between the predicted heat demand h̃d (k) and the actual h̃d,act (k)
and the closer the CMPC and HMPC controller performance gets to the reference
CMPC controller.
• Make a translation from the optimal heat energy exchange results (setpoints) h̃xc,i (k)
to a lower level grid operation optimization problem. This could then try to actu-
ally achieve the setpoint by optimizing the valves and pumps setpoints using a higher
detailed model of the grid based on physics equations (mass flow, temperature).
• Model the variable time delay or apply an approximation method in the heat energy ex-
change model for more accurate modelling of the heat transportation characteristics. If
this is possible, then going towards a simple temperature and mass flow based modelling
approach is the logical next step.

Master of Science Thesis Y.C.C. Putter

94 Conclusions and Recommendations

5-2-2 Solving the MIQP Optimization Problem

• The winter scenario of the WarmCo case showed the importance of applying heuristic
methods for finding a good quality local optimal solution in restricted runtime. However,
the cost and possibly the computational requirements for using the commercial solver
Gurobi (which uses many state-of-the-art heuristics) can be too expensive to use on the
embedded systems. It is therefore possibly required for TNO to use an open source solver
instead or to develop their own MIQP solver based on heuristics found in literature.
• Apply optimization starting point heuristics, to simulate a more explorative/greedy
solver that tries different and feasible combinations of the binary values first.
• Explore the possibility to combine the advantages of the MPC approach, with the
main advantage of HM’s bidcurve principle that currently requires only a single two-
way information exchange between the agents and the higher level coordinator. This
could possibly reduce or eliminate the large number of ADMM q-iterations, if more
information is shared.
• Otherwise, implementing an auto tuning of step size ρ for each of the separate global
coupling constraints affected during the ADMM algorithm, to be less dependent on
convergence quality and speed of the initially chosen value.
• Make the HMPC algorithm more robust, in the sense that hard constraints are relaxed
after the case when no feasible optimal solution can be found (within a certain limited
runtime).

5-2-3 Improving Characteristics of MPC Approach

• Design a stochastic MPC approach, to take into account the uncertainty of the heat
demand prediction during optimization. This would then possibly result in more robust
buffer (un)loading trajectories.
• Design a non-cooperative MPC approach, to be able to fully support scenarios with
multiple stakeholders where the local objective of each agent can differ from the global
objective. This requires to study game-theoretic approaches.
• Design a time-based hierarchical coordinator that discretizes the system models with a
larger time step ts >> 1 h, such that the heat demand peaks can be anticipated with this
time-based higher level coordinator. Then, the controller that model the subsystems
with a smaller time step, can use the setpoints instead of having to use a large prediction
horizon Np . This decreases the optimization problems for both time-step discretized
system models optimizers.
• Get detailed cost function parameters to ensure that the optimization is as close as
possible to the actual financial optimization objective.

Y.C.C. Putter Master of Science Thesis

 Appendix A

Demand Modelling

The demand profile varies per building, occupancy and environmental conditions such as
weather and solar irradiation. The latter basically implies the geographical and the time
dependency of the demand, ranging from weekly to seasonal time scales. The demand profile
can often be divided in a space heating and domestic hot water component, and the ratio
between the two differs with the seasons. It also has a deterministic and stochastic component.
If the demands of individual buildings are aggregated at substations, then this could influence
the ratio between the stochastic and deterministic component.

For a residential building, the basic demand profile features are the peak in the morning, the
dip at daytime, during weekends the delayed peaktime in the morning and the slightly higher
demand at daytime and during colder months the higher demand. For large residential or
office buildings the heat loss differs per room due to its location and thermal inertia, and for
that reason the building’s overall demand is often modelled.

In [1], typical heat loads were identified with the use of one year of hourly heat meter readings
from 141 substations in two district heating networks, and two of them can be seen in Figure A-
1. Another example is the publicly available space heating and domestic hot water [38] profile
generator from Quintel Etmoses. These recorded or averaged demand profiles can be taken
as a reference prediction, but the limitation with this kind of approach is that the demand
profile is not a function of measurable variables such as the outdoor temperature or usage
profile.

Master of Science Thesis Y.C.C. Putter

96 Demand Modelling

Figure A-1: Average weekly heat load pattern during four seasons. Left, a multi-dwelling building.
Right, a public administration building with night setback control. [1]

Instead, models are generated that represent the relation between input variables (tempera-
ture, time, usage) and the output (heat demand or supply temperature setpoint). A distinc-
tion can be made in static and dynamic models, the degree on which it is based on physical
relations, and whether it includes stochasticity. Data is then used to finetune the parame-
ters of the model, to achieve a good fit (evaluated with for example RSME, VAF) but avoid
overfitting, such that it is general enough for other possible datasets or scenario’s.
Static models estimate the current output by using only current input data, of which two
examples are shown in equation (A-1a) [26][22]. Both have a constant term to model base
demand and a nonlinear term to model the peak demand. The second example differs as it
includes a direct and linear dependency of the outside temperature. Current climate control
systems mostly use a straightforward affine relation between outside temperature and heat
demand or supply temperature setpoint.
1 1
(t−t1 )2 (t−t2 )2
y(t) = β0 + β1 e c1 + β2 e c2 (A-1a)
1
(t−t1 )2
y(t) = β0 + β1 T (t) + β3 e β4 (A-1b)

Dynamic models estimate the current output by also using previous input and output data.
One approach is to use a black box modelling method, which excludes explicit knowledge
about the underlying physical relations between input and output. It is possible to include
noise (stochastic behaviour) with methods such as the Box Jenkins model, shown in equa-
tion A-2, in which q represents the shift operator and the capital letters the polynomials
describing the dynamics of the input u(t) and noise e(t). When additional assumptions are
made on the dynamics between inputs and noise, the Box Jenkins model can result in for
example ARX, ARMAX and OE [39] models. It is also possible to use other modelling
techniques such as applying recurrent neural networks [40].

B(q) C(q)
y(t) = u(t) + e(t) (A-2)
F (q) D(q)

If the modelling is based partially or fully on physical relations, then it is referred to as

respectively grey or white box modelling. One of the examples is Deerns’ LEA model which

Y.C.C. Putter Master of Science Thesis

 97

is based on the dynamics of the indoor air temperature, specific heat, density and volume of
the air as well as the demand, internal production and environmental gain or loss of thermal
energy. It is described in more detail in [33] and [23], of which the latter applied a stochastic
version of it with the use of Markov Chains to create different demand profile scenarios.
dTin
cp,a ρa Va = hd + hint + henv (A-3)
dt

Master of Science Thesis Y.C.C. Putter

98 Demand Modelling

Y.C.C. Putter Master of Science Thesis

 Appendix B

Supply Modelling

Regarding heat supply, it was identified in chapter 1 that combined heat and power (CHP)
plants and gas fired boilers are currently dominant, but that the vision lies in geothermal
energy as dominant producer and making effective use of excess industry heat that is otherwise
wasted. Also, thermal energy storage both on large scale (ATES) and small scale (located
at consumers) play an important role in the heat grid. Modelling the characteristics of these
systems can be done on different levels of details, but should give a representation of the
operation costs, dynamics and controllability or flexibility as referred to by TNO.
Abstract models of CHP plants or controllable producers are usually described by variables
that represents the on/off state v(k) and the produced thermal energy variable h(k). For
CHP specifically, this can include the coupling between heat and electricity production [24].
The constraints are used to model the dynamics with minimum on-off times and the limits
on production capacity and ramping. The startup, shutdown and production costs are either
linear, PWA or quadratic ([29] and [18]). PWA functions of h(k) can represent the different
discrete operating modes of thermal energy production. Buffers can be represented by a
simple discrete time dynamic x(k + 1) = η(x(k) + uin (k) − uout (k)), with buffer storage
efficiency parameter η. Uncontrollable producers such as waste heat, geothermal or solar
can be captured by the same reasoning outlined in the demand modelling, depending on the
process characteristics.
Detailed models of CHP plants consider the thermodynamics of the processes, see Figure B-
1. It is an approach often found in literature that focuses on the energy and cost-efficiency
optimization of production planning in current heat grids ([39], [2] and [19]). Because of the
physical modelling, they also give insight in the dependencies of mass flow, supply and return
temperature on the efficiency.
In general, the efficiency of all types of producers can be optimized by lowering the supply
and return temperatures. In [39], it is is stated that the power to heat ratio for CHP plants
is increased in particular when the supply temperature is lowered, instead of the return
temperature. Lower supply temperatures also decrease the thermal energy losses and make
it easier for other low-temperature sources to supply heat. To still provide in the thermal

Master of Science Thesis Y.C.C. Putter

100 Supply Modelling

energy demand at the substations, the volumetric flow rate has to increase. In an exisiting
grid, the pipe diameters cannot be changed, so consequently the mass flow rate has to increase.
This lowers the time delay and also counters the heat exchanger performance drop due to
accumulation of small particles inside. But at the cost that it requires more pumping energy
and affects the pressure levels at every substation.

Figure B-1: Mollier hs- and Ts-chart of CHP plant. Dotted line describes the difference of a
non-CHP plant (Frederiksen & Werner 2013, p.158) [2]

Y.C.C. Putter Master of Science Thesis

 Appendix C

Network Modelling

Because heat energy transfer equations are governed by mass flows and temperature differ-
ences and pipe specifications, a physical model of a heat grid network could describe how they
affect the thermal losses, mechanical losses and variable time delay. To limit the theoretical
scope, only a basic decoupled hydraulic and thermal equations model is explored to yield
some insights of the complexities that arise with such an approach.

C-1 Preliminaries

The assumptions required for this basic decoupled hydraulic and thermal equations model
will be stated, and verified with specifications of Eneco’s heat grid "Leiding over Noord" in
Rotterdam [16]. Also, the most relevant dimensionless numbers found in the equations are
briefly explained.

C-1-1 Assumptions for Pipe Flow and Water Properties

The main assumptions are incompressible, turbulent, steady, fully developed and one dimen-
sional pipe flow fully filled with water. Furthermore, it is assumed that the time delay of a
change in flow rate, or the propagation speed of a pressure change, is negligible due to the
speed of sound of water (1481 m/s at 20o C. Lastly, it is assumed that friction losses are not
converted into heat.
Water properties can be looked up at various resources such as [41] for precise numbers as
they change with pressure and temperature. It may be justified for some properties to take
mean values as they change only slightly with temperature, such as the thermal conductivity
κw ≈ 0.65 W/mK, specific heat Cp,w ≈ 4.2 kJ/kgK and density ρw ≈ 980 kg/m3 (taken at
approximately 325-350 K) [42]. Incompressible flow is assumed because water density varies
slightly with pressure.
Here, water flows at a maximum speed of Vmax = 3.2 m/s under a maximal pump pressure
of pmax = 22 bar. The pipe geometry of the primary transportation network consist of an

Master of Science Thesis Y.C.C. Putter

102 Network Modelling

inner diameter of Dinner = 0.70 m of steel with a thickness of ts = 0.01 m covered by a

polyurethane insulation thickness of tPUR = 0.08 m and a polyethylene layer of tPE = 0.01 m.
The thermal conductivity κ W/mK of the materials can be estimated with online data [41].
The primary network loop has a total length of Ltotal = 16.8 km and the pipes are assumed
to be at a depth of d = 1 m. The supply and return temperatures for the transportation
and distribution network are respectively Tts = 100 − 120, Ttr = 58 − 68, Tds = 70 − 90 and
Tdr = 40 − 60 degrees Celsius.

Figure C-1: Schematic of pipe cross section and buried pipe [3]

C-1-2 Dimensionless Numbers

Dimensionless group numbers that are relevant for momentum- and convective heat transfer
of pipe flows are given [43][44][45].
ρDV DV 4ṁ
Re = = = (C-1a)
µ v πDµ
v µCp
Pr = = (C-1b)
α κ
hc L
Nu = (C-1c)
κ
VL
P e = ReP r = (C-1d)
α

The Reynolds number Re is the relative importance of convective versus molecular trans-
port of momentum and is used to determine whether a flow is laminar or turbulent Re > 4000.
With the specifications, the values are in the order of Re ≈ 1 · 106 . Turbulent flow is unsteady
by nature due to eddies, but the fluctuations of the velocity and pressure are averaged to
assume steady flow. Furthermore, the thermal entrance region is relatively short compared
to laminar flow, such that the transverse temperature distribution becomes "fully developed".
Together with the significant importance of pipe roughness  for turbulent flow according to
[44, sec. 4.7.1] and [46], these assumptions will determine which heat transfer and pressure
loss correlations are applicable.

Y.C.C. Putter Master of Science Thesis

C-2 Hydraulic Modelling 103

The Prandtl number P r is the ratio between momentum and thermal diffusivity and is
only a function of the fluid properties at given temperature and pressure. For water it ranges
from 7 at 20o C to 1.45 at 120o C.

The Nusselt number N u is the ratio between convective and conductive heat transfer
across the boundary, thus perpendicular to the flow direction. It can be approximated with
Gnielinski correlation for turbulent flow (assuming a constant pipe surface temperature and
only valid for 3000 ≥ Re ≥ 5 · 106 and 0.5 < P r < 2000 with P r 6= 1).
 
f
8 (Re − 1000)P r
N¯u =  1/2 (C-2)
f
1 + 12.7 8 (P r2/3 − 1)

The Friction Factor f is the Darcy-Weisbach friction factor (not to be confused with the
Fanning friction factor) and is used to determine the pressure loss due to pipe flow. It depends
on the Reynolds number Re and relative pipe roughness /D, which can be estimated with
table 6.1 [45, p. 381]. The friction factor can be determined by the Moody diagram or
Colebrook-White equation C-3a, but are not convenient to use as it must be solved iteratively
with for example the Newton-Rhapson method. In [47] an extensive overview of all non-
iterative approximation friction factors is given, with the Goudar-Sonnad equation being the
most complex but also the most accurate one that covers all relative pipe roughness values.
For ease of use the Haaland method ( ≤ 0.05 mm) [48] is given in equation C-3b.
1  2.51
 
√ = −2 log + √ (C-3a)
f 3.7D Re f
" 1.11 #
1  6.9
√ = −1.8 log + (C-3b)
f 3.7D Re

The Peclet number P e is similar to the Reynolds number, as it gives the ratio of advection
(convection) to diffusive transport (conduction) of thermal energy. With the specifications
P e >> 1, and thus conduction of thermal energy in the flow direction can be neglected.

C-2 Hydraulic Modelling

Hydraulic modelling is used to analyze fluid flow, decoupled from thermal energy exchange.
The law of conservation of mass states that the pipe mass inflow equals the mass outflow
C-4a. Simplifying it for equal and constant diameter pipes, results to the statement that the
incoming flow rates should equal the outgoing flow rates at a node. A straightforward relation
between pipe geometry and mass flow is given in equation C-4b.
ṁin = ṁout (C-4a)
π
A = D2
4
Q = AV
π
ṁ = ρQ = ρ D2 V (C-4b)
4

Master of Science Thesis Y.C.C. Putter

104 Network Modelling

The (extended) Bernoulli’s equation (C-5) is a conservation of mechanical energy for fluid
flow. Expressed in terms of heads, it shows the relation between the flow work per unit mass
p V2
ρg , the kinetic energy 2g and potential energy z of fluid at upstream (. . .)1 and downstream
(. . .)2 , together with additional sources ∆hpump or losses ∆htot (V ) of mechanical energy.
! !
p V2 p V2
+ +z + ∆hpump = + +z + ∆htot (V ) (C-5)
ρg 2g 1
ρg 2g 2

For closed loop piping systems, the extended Bernoulli’s equation basically reduces to (C-6).
This equation shows that the pump is only required to overcome the dynamic pressure losses.
As the total head loss ∆htot (V ) is a function of the flow velocity V , the operating point of
the pump and valve positions will determine the mass flows through the pipe network.

∆hpump = ∆htot (V ) (C-6)

As a side note, the static pressures in the network are irrelevant for flow rate calculation.
However, sufficient static pressure ensures a positive pressure in the pipelines under system
operation and fluctuating temperatures, such that the chance of air drawing in through micro
gaps of fittings is decreased and vaporization (if temperatures reach boiling point) is avoided.
Both effects could negatively affect the life-span and efficiency of components such as pumps.
Static pressure is created with open or closed expansion vessels for small network, or by
pressure holding systems for large networks.

C-2-1 Total Head Loss

The total head loss ∆htot (V ) in a pipe consists of major losses caused by heat exchangers
hxc , pipe friction hf , and minor losses hm that are caused by a multitude of situations such
as pipe entrance or exits, expansions or contractions, bends, fittings and (throttling) valves.
X
∆htot = hxc + hf + hm
V 2 
fL X

= hxc + + K (C-7)
2g d

The major losses are calculated with the Darcy-Weisbach equation C-8 which requires the
friction factor f of equation C-3a or C-3b. The head loss of heat exchangers depend on the
type and size installed. Most commonly used heat exchangers are plate heat exchangers.

L ρV 2 L 8ṁ2
∆p = f =f 5 2 (C-8)
D 2 D π ρ

The minor loss coefficient K can be looked up in tables from manufacturers or approximated
with [45, section 6.9]. Although the minor losses could be neglected, the effect of (partially
xvalve = (0, . . . , 1)) closed valves in particular could be substantial (even infinite).

Kvalve,max V 2
hvalve = (C-9)
xvalve 2g

Y.C.C. Putter Master of Science Thesis

C-2 Hydraulic Modelling 105

C-2-2 Pumps

The type of pumps installed in district heating systems, are centrifugal pumps. These pumps
are designed in such a way that they are able to deliver the minimum required pressure at
the furthest located substation at a certain desired flow rate and energy efficiency.
The pump curve is a graphical representation of the produced pressure (or head) as a function
of the produced flow rate. The system curve for a closed loop pipe system, relates the system’s
total head loss (dynamic pressure loss only) as the flow rate. By superimposing the two curves
and finding the intersection point, the operating point of the pump can be determined, see
Figure C-2. A pump is manufactured such that the operating point falls within circa 80−110%
of the best efficiency point (BEP).

Figure C-2: Pump and system curve for a variable speed pump in a closed loop pipe system;
only friction losses are considered in the system curve [4]

Different flow rates can be achieved by using throttling valves or bypass lines for a constant
speed pump, or by changing the rotational shaft speed of a variable speed pump. In [4] it is
stated that the latter is superior regarding energy efficiency, reliability and life cycle costs.
This is also studied in [49], which compares the differences in the design and efficiency of
a distributed variable speed pumping (DVSP) system to a conventional central circulating
pump (CCCP) system.
For variable speed pumps, the affinity laws relate the change in rotational shaft speed (N )
to the change in flow rate (Q), added pump head (H) and the shaft power (P ), assuming
constant efficiency (ηpump ).
2 3
Q1 N1 ∆hpump,1 N1 P1 N1
 
= = = (C-10)
Q2 N2 ∆hpump,2 N2 P2 N2

Master of Science Thesis Y.C.C. Putter

106 Network Modelling

However, this assumption can be invalid as the system curve often does not perfectly coincide
with the iso-efficiency lines of the pump curve, as can be seen in Figure C-2. This can be the
case due to mechanical/electrical design limitations of the pump, even if the system and pump
curves are perfectly modelled. Additionally, the system curve changes for every possible set
of valve positions. Therefore, the pump is often oversized to ensure proper performance for
the worst-case system curve in a trade-off with the pump’s efficiency.

ρ g H(Q) Q
ηpump = (C-11)
Pin

Either way, changing the flow rate comes with a cost and also has constraints. For constant
speed pumps controlled with throttling valves, the cost is in the form of the amount of wasted
mechanical energy and the constraint is that the throttling valves have to be closed for at
least 10%. For variable speed pumps, the cost is in the form of electrical energy consumed and
how lifespan is affected by operating at the left or right of the BEP. Constraints are imposed
to operate close within the BEP region to avoid overheating, vibration and cavitation.

Figure C-3: The effect of operating a centrifugal pump at the left or right of the BEP [5]

C-3 Thermal Modelling

According to [3], to model the temperature front propagation in pipe flow, most district
heating grid studies use models that can be classified into three groups: transient, steady
state and pseudo-transient. All groups make use of the one dimensional partial difference
equation C-12 of the energy balance depicted in Figure C-4, but neglect one or more terms
besides the third. It is not mentioned specifically if the use of the one dimensional partial
differential equation is also valid for turbulent flow, but this will be assumed to be the case.
The terms can be described as the:

Y.C.C. Putter Master of Science Thesis

C-3 Thermal Modelling 107

1. gains due to internal energy of the fluid

2. enthalpy of the fluid
3. rate of conductive heat transfer within the fluid (negligible due to P e >> 1)
4. rate of convective heat transfer from soil surface to fluid

∂T ∂T ∂2T T −T
ρCp A + ṁCp = kA 2 + s 0 (C-12)
| {z ∂t} | {z ∂x} | {z∂x } | R
{z }
1 2 3 4

Figure C-4: Schematic of energy balance across differential pipe segment dx [3]

The transient model only neglects the third term, and provides relatively accurate results
for variable flow where the time delay does not have to be tracked, but is computationally
intensive.
− AC 1R0 ρ (t−τ (t))
 
Tout (t) = Ts (t) + Tin (t − τ (t)) − Ts (t) · e p (C-13a)
Z t
4ṁ(θ)
L= dθ (C-13b)
t0 (t) πD2 ρ
τ (t) = t − t0 (t) (C-13c)

The steady state model is computationally faster by neglecting the first term, resulting in
equation C-14. It assumes constant mass flow ṁ, where the output temperature of a segment
∆x = L and the time delay are simplified in the following:
− ṁCL R0
 
Tout = Ts + (Tin − Ts ) · e p (C-14a)

πD2 ρL
τ= (C-14b)
4ṁ

Master of Science Thesis Y.C.C. Putter

108 Network Modelling

Psuedo-transient combines the steady state model with a variable time delay td to include the
tracking of the fluid in time with variable flow ṁ(θ). At each time step t, ṁ(t) and Ti (t) are
fixed leading to a τ (t) and Tout (t) with the steady state model equation C-14. To calculate
the heat loss at time t, the previous Tout,τ (t) is used. However, variable flow rate makes the
modelling complicated and possibly unsuitable for online optimization methods. This may
require to assume a constant flow rate and time delay.

Tout,τ (t) = Tout (t − τd ) (C-15a)

Q̇h (t) = ṁ(t)Cp (Tin (t) − Tout,τ (t)) (C-15b)

If Ts and R0 are not taken as a function of x, a simple relation between heat loss, pipe
specifications and mass flow can be assumed. Else, the pipe can be divided into segments ∆x
which must adhere to the Courant-Friedrichs-Levy criterion, where ∆t is the simulation time
step:
4ṁ(t)∆t
Co (t) = ≤1 (C-16)
πDρ∆x

Figure C-5: Schematic of pipe segments s with length ∆x, where Ts2,in = Ts1,out [3]

The thermal resistance per unit length R0 of the fourth term can be approximated by equa-
tion C-17, under the assumption that the thermal conductivity of the materials are not a
function of temperature or time. Also, the effect of thermal interaction between the return
and supply pipe, modelled with detail in [50], is neglected for now
 
rb r
   
rc
1 ln ra ln rdln 1 rb
R0 = + + + c
+ (C-17)
¯
2πra hc 2πkab 2πkbc 2πkcd Sks

Although the outer surface of the pipe is not isothermal due to the insulation layers, it is
assumed for now such that the shape factor can be approximated by the following relation
[44, p. 153].
2π
S=   (C-18)
−
cosh 1 rd
a

Only the first term of R0 depends on the pipe flow speed due to the average heat transfer
coefficient h¯c . It can be estimated using the Nusselt number equations C-1c and C-2, together

Y.C.C. Putter Master of Science Thesis

C-4 Modelling of District Heating Networks 109

with the specifications of the previous section C-1. A rough hand calculation of the relative
effect of this first term on the total thermal resistance R0 , under different flow speeds (0.032 <
V < 3.2), estimates a negligible contribution in the order of < 1·10−3 . The thermal resistance
can therefore be approximated as a function of only the pipe specifications, neglecting the
mass flow dependency.

C-4 Modelling of District Heating Networks

To calculate the flows and pressures in pipe networks where pipes are connected in series and
parallel, Kirchoffs voltage and current laws apply for steady flows. It basically states that
the sum of all flows of line l connected to any junction or node n must be zero (continuity
equation), and the net pressure change around any closed loop must be zero (conservation of
potential). If two flows of different temperature enter a node, the node temperature can be
calculated by the mass flow weighted average [51].

The resulting computation of a network with loops consists of a non-linear system of algebraic
equations. In many cases it is linearized, to reduce computational effort in a trade-off with
modelling accuracy. A fast method is preferred over a very accurate one, as it is likely that
the actual optimization of demand and supply will be an iterative process before it comes to
a solution.

• Demand and supply of thermal energy affect mass flows and supply temperatures.

• These determine the time delays and energy losses between suppliers and consumers.

• This process is iterated until demand and supply are optimally matched with network
effects taken into account.

The Hardy Cross method is often used to solve the linearized loop equations, which is an
iterative flow rate error correction method and starts with assumed flows in the predefined
direction. A more computational efficient numerical solution compared to the Hardy Cross
method is described in [6] and tested on the loop configuration of the DH system depicted
in Figure C-6. These approaches only consider the hydraulic solution, and not the thermal
solution. In [52], a graph theoretical simulation approach with decoupled hydraulic and
thermal solution is presented, similar to the theory outlined in previous sections. A simulation
software of TNO called CHESS [53], aims to model the dynamic behaviour of thermal systems
and is more or less based on the same assumptions stated in this study. In PowerMatcher [28],
only energy exchange is modelled based on graph theory with line resistance and capacity.
Furthermore, in [18] the possibility of dynamical programming is stated but not elaborated.
Mass flows and pressures for a few values of opening degrees of the valves are computed off-
line during simulation. Then, during optimization, mass flows and pressure are determined
on-line using a linear interpolation of the solutions.

Master of Science Thesis Y.C.C. Putter

110 Network Modelling

(a) Typical DH system configuration (b) Loop configuration of a DH system

Figure C-6: Pipe network of DH system [6])

Y.C.C. Putter Master of Science Thesis

 Appendix D

Decomposition Methods

Primal decomposition (resource allocation) is better suited for problems with coupling vari-
ables. Here, it is possible to achieve feasible solutions at every iteration. Dual decomposition
(price coordination) is better suited for problems with coupling constraints. Indirect decom-
position can be used to translate a primal decomposition to a dual decomposition. How-
ever, caution is required because the following examples are only applicable for convex cost
functions and constraints (where strong duality holds). The alternating direction method
of multipliers (ADMM) is an augmented Lagrangian method (dual decomposition method),
that works around the strict convexity requirement and is therefore in this thesis problem
setup, as it is non-convex due to the integer variables and constraints. Most information here
is gathered from the following sources. Only a very brief and incomplete summary of the
available information is shown, gathered from [27][34][54][55] and [56].

D-1 Primal and Dual Decomposition Examples

Primal Problem:
p(x) = min f1 (x1 , y) + f2 (x2 , y) (D-1a)
x1 ,x2 ,y

s.t. x1 ∈ X 1 (D-1b)
2
x2 ∈ X (D-1c)
y∈Y (D-1d)

Primal Decomposition: Master problem controls the coupling variable y and fixes it for
the subproblems (φi )
min φ1 (y) + φ2 (y) (D-2a)
y∈Y

φ1 (y) = min f1 (x1 , y) (D-2b)

x1 ∈X 1
φ2 (y) = min f2 (x2 , y) (D-2c)
x2 ∈X 2

Master of Science Thesis Y.C.C. Putter

112 Decomposition Methods

Dual Decomposition: Translating the primal problem, by introducing local versions yi of

y and coupling constraint y1 = y2 . Then introduce the the Lagrange dual function to make
the problem separable.
p(x) = min f1 (x1 , y1 ) + f2 (x2 , y2 ) (D-3a)
x1 ,x2 ,y1 ,y2

s.t. y1 = y2 (D-3b)
T
L(x1 , x2 , y1 , y2 , λ) = f1 (x1 , y1 ) + f2 (x2 , y2 ) + λ (y1 − y2 ) (D-3c)

Then the master and subproblems gi (λ) are

max g1 (λ) + g2 (λ) (D-4a)
λ
g1 (λ) = inf (f1 (x1 , y1 ) + λT y1 ) (D-4b)
x1 ,y1

g2 (λ) = inf (f2 (x2 , y2 ) + λT y2 ) (D-4c)

x2 ,y2

D-2 Dual Decomposition Problem

Now focusing on dual decomposition for a general convex primal problem.

D-2-1 Primal Problem

Taking the primal problem, where x is called the primal variable, J(x) is the cost function,
gi and hi are respectively the i-th inequality and equality constraints, and x must lie in the
feasible set X .
p(x) = min J(x) (D-5a)
x
s.t. gi ≤ 0, i = 0 . . . I (D-5b)
hi = 0, i = 0 . . . E (D-5c)
x∈X (D-5d)

D-2-2 Lagrange Dual Problem

Lagrange Dual function, basically moves the constraints to the cost function (penalty like idea)
with Lagrange (only equality constraints) or KKT multipliers λi and vi (dual variables). The
infimum is the greatest lower bound of a subset S of a partially ordered set T.
I
X E
X
L(x, λ, v) = J(x) + λi gi (x) + vi hi (x) (D-6)
i=1 i=1
gd (λ, v) = inf (L(x, λ, v)) (D-7)
x

The KKT multipliers λi and vi are basically soft linear approximations of the infinitely hard
displeasure functions, i.e. set indicator functions I_ (gi (x)) and I0 (hi (x)).
( (
0 if gi (x) ≤ 0 0 if hi (x) = 0
I_ (gi (x)) = I0 (hi (x)) = (D-8)
∞ gi (x) > 0 ∞ otherwise

Y.C.C. Putter Master of Science Thesis

D-3 ADMM 113

Dual problem is always concave, even if the primal problem is non-convex.

I E
!
X X
d(x, λ, v) = max gd (λ, v) = max inf J(x) + λi gi (x) + vi hi (x) (D-9)
λ,v λ,v x
i=1 i=1
λi ≥ 0 (D-10)
vi ≥ 0 (D-11)

Where the following always holds for any feasible x̃ (weak duality d∗ ≤ p∗ or strong duality
d∗ = p∗ ):
 
gd (λ̃, ṽ) = inf L(x, λ̃, ṽ) ≤ L(x̃, λ̃, ṽ) ≤ J(x̃) (D-12)
x

D-2-3 Karush-Kuhn-Tucker conditions

Karush-Kuhn-Tucker conditions are necessary, but only sufficient for convex problems. Non-
convex problems require additional constraint qualifications (regularity conditions).
I E
∇J(x∗ ) + λi ∇gi (x∗ ) + vi ∇hi (x∗ ) = 0
X X
(D-13a)
i=1 i=1
gi (x∗ ) ≤ 0 (D-13b)
∗
hi (x ) = 0 (D-13c)
λ∗i ≥0 (D-13d)
λ∗i gi (x∗
)=0 (D-13e)

D-3 ADMM

ADMM is derived of the method of multipliers shown below, which uses an augmented Lan-
grangian with penalty parameter ρ, to workaround the hard requirement that the problem has
to be convex in order for it to converge. However, the problem is that this is not decompose-
able into subproblems, and that is where the ADMM comes in, which is shown in Algorithm 2.

min J(x) (D-14)

x∈X
s.t. Ax = c (D-15)
ρ
Laug (x, λ) = J(x) + λT (Ax − c) + ||(Ax − c)||22 (D-16)
2

Master of Science Thesis Y.C.C. Putter

114 Decomposition Methods

Y.C.C. Putter Master of Science Thesis

 Appendix E

Agent Constraint Matrices and

Vectors

E-1 Agents

Only the unique constraints will be shown per agent. The assignment of slack variables to
the inequality constraints will be omitted, and instead the original inequality constraints will
be shown. Here, nxc denotes the number of heat exchange variables for an agent.

E-1-1 Node

Nodal Balance (3-8)

   
11×nxc 0 ... 0 0
 0

11×nxc ... 0 
 0
 
 . ..  h̃ (k) =  .  (E-1)
 . .. .. xc .
 . . . .

 .
0 0 . . . 11×nxc 0

Heat Energy Exchange Capacity (3-9)

   M
In 0 ... 0 h̃xc
 xc  M
 0 Inxc ... 0 
 h̃xc 
 . ..  h̃ (k) =  .  (E-2a)
 . .. ..  xc  . 
 . . . .   . 
0 0 . . . Inxc h̃M
xc
−h̃m
   
−Inxc 0 ... 0 xc
 m
 0

−Inxc ... 0 
 −h̃xc 
 . ..  h̃ (k) =  .  (E-2b)
 . .. ..  xc  . 
 . . . .   . 
0 0 . . . −Inxc −h̃m
xc

Master of Science Thesis Y.C.C. Putter

116 Agent Constraint Matrices and Vectors

Remember that all other agents have a variation on the Nodal Balance constraints and have
the Heat Energy Exchange Capacity constraints.

E-1-2 Producer

The constraints for a Producer agent can be written as follows.

Nodal Balance (3-14)

     
11×2 0 ... 0 11×nxc 0 ... 0 0
1×2 ... 11×nxc ...
 0 1 0   0 0 0
     

 h̃(k) +  ..
 . ..  ..  h̃ (k) =  .  (E-3)
 . .. .. 
.. ..  xc .
 . . . .   . . . .  .
0 0 . . . 11×2 0 0 . . . 11×nxc 0

Hybrid Capacity (3-15)

−h̃M
     
I2 0 ... 0 0 ... 0 0
0

I2 . . . 0 
 0

−h̃M . . . 0   0
 
.  h̃(k) +   ṽ(k) ≤  .. 
.
.. ..  .. .. .. ..   
.
. . . .. 
  . . . .  .
0 0 . . . I2 0 0 . . . −h̃M 0
(E-4a)
h̃m + 2×1
     
−I2 0 ... 0 0 ... 0 0
 0

−I2 . . . 0  
 0 h̃m + 2×1 . . . 0 
 0
 
 ṽ(k) ≤  . 
 h̃(k) + 
 . ..  .. ..
 . .. .. 
.. .. .
 . . . .   . . . .

 .
0 0 . . . −I2 0 0 . . . h̃m + 2×1 0
(E-4b)

Hybrid Ramping (3-17)

       
I2 0 ... 0 Ã 0 ... 0 B̃ 0 ... 0 h̃(k − 1)
−I2 I2 ... 0 0 Ã . . . 0  0 B̃ . . . 0  0
       
 
 .
 . .. ..  h̃(k) +  ..
..  
.. ..  ṽ(k) +  ..
..  
.. ..  ṽ
..  ramp (k) ≤  .. 
 . . . . . . . . . . . . 
 .


0 . . . −I2 I2 0 0 . . . Ã 0 0 . . . B̃ 0
(E-5a)
       
−I2 0 ... 0 C̃ 0 ... 0 D̃ 0 ... 0 −h̃(k − 1)
 I2 −I2 . . . 0  0 C̃ . . . 0  0 D̃ . . . 0  0
       
 
 .
 . .. ..  h̃(k) +  ..
..  
.. ..  ṽ(k) +  ..
..  
.. .. ..  ṽramp (k) ≤ 
  .. 
 . . . .  . . . . . . . .  .


0 . . . I2 −I2 0 0 . . . C̃ 0 0 . . . D̃ 0
(E-5b)

Y.C.C. Putter Master of Science Thesis

E-1 Agents 117

where
" #

up
 −hup
p 0
à = diag −h̃ =
0 −hup
s
" #
  −hm 0 0 0
B̃ = diag −h̃m ⊗ [1 0] = p
0 0 −hms 0
" #

down
 −hdown
p 0
C̃ = diag −h̃ =
0 −hdown
s
" #
  0 −hm down 0
p − hp 0
D̃ = diag −h̃m − h̃down ⊗ [0 1] =
0 0 0 −hm
s − hdown
s

Binary Transient Coupling (3-18)

     
ã 0 ...
0 b̃ 0 . . . 0 ṽp (k − 1)
−ã ã ...
0 0 b̃ . . . 0 0
     
 
 .
 . .. ..  v
..  p (k) +  .. . .
 ṽ
. . ..  ramp
 (k) =  ..  (E-6a)
 . . .. . . . . 
 .


0 . . . −ã ã 0 0 . . . b̃ 0
   
c̃ 0 . . . 0 1
0 c̃ . . . 0 1
   
.
. .. . .  ṽramp (k) ≤ 
 .. 
 (E-6b)
. . . . .. 
 .
0 0 . . . c̃ 1
d˜
     
0 ... 0 ẽ 0 . . . 0 ṽs (k − 1)
 ˜ ˜
−d d . . . 0 0 ẽ . . . 0 0
  
   
 vs (k) +  .. . . ṽ
..  ramp (k) = .. (E-6c)
 . .. 
.. .. 
.   
 .
 . . . . . . .. .

 .


0 . . . −d˜ d˜ 0 0 ... ẽ 0
f˜ 0 . . .
   
0 1
 0 f˜ . . . 0 1
  

 ṽramp (k) ≤  ..  (E-6d)
. .. 
 . .. .  
. . .. . .
0 0 ... f˜ 1

where
h i
ã = 1 0
h i
b̃ = −1 1 0 0
h i
c̃ = 1 1 0 0
h i
d˜ = 0 1
h i
ẽ = 0 0 −1 1
h i
f˜ = 0 0 1 1
(E-7)

Master of Science Thesis Y.C.C. Putter

118 Agent Constraint Matrices and Vectors

Minimum on (off) times (3-19) This minimum up time constraints are shown for a single
binary v(k) variable.
h i
U1 U1
−11×v 01×Np −v v(k) = −v U1 (E-8)
UT
−v UT v UT − 1 −11×v −1
   
0 ... 0 0
UT UT 1×v UT −1
 0 −v v −1 −1 ... 0 0
   

 v(k) =  .  (E-9)
 .. .. .. .. .. ..

.
 . . . . . . .


UT −1
0 ... 0 −v UT UT
v − 1 −11×v 0
UT −2
   
0 . . . 0 −v UT + 1 v UT − 2 −11×v 0
. . .. .
.. ... .. .. .

. . .
. .  v(k) =  . 

(E-10)
0 . . . −3 2 −1 −1 0
   

0 ... 0 −2 1 −1 0

The minimum down time constraints are also shown for a single binary v(k) variable.
h i
D1 D1
11×v 01×Np −v v(k) = 0 (E-11)
DT
1 − v DT 11×v −1
 DT   
v 0 ... 0 v DT
DT
 0 v DT 1 − v DT 11×v −1 ... 0 v DT 
   

 v(k) =  .  (E-12)
 .. .. .. .. .. ..

 . 
 . . . . . .  . 


0 ... 0 v DT 1−v DT 1 1×v DT −1 v DT
DT −1
   
0 . . . 0 v DT − 1 2 − v DT 11×v v DT − 1
. .
. .. ... .. .. ..   .. 
. . . . 
 v(k) =

 . 
 (E-13)
0 . . . 3 −2 1 1 3
   
  
0 ... 0 2 −1 1 2

E-1-3 Consumer

The constraints for a Consumer agent can be written as follows.

Nodal Balance (3-23)

     
1 0 ... 0 11×nxc 0 ... 0 hd (k)
0 1 . . . 0
 
 0

11×nxc ... 0 


 hd (k + 1) 

 e(k) +  ..
.  h̃ (k) =  ..  (E-14)
. .. . . ..  
.. .. ..  xc
. . . .  . . . . 

 .


0 0 ... 1 0 0 . . . 11×nxc hd (k + Np − 1)

E-1-4 Buffer

The constraints for a Buffer agent can be written as follows.

Y.C.C. Putter Master of Science Thesis

E-1 Agents 119

Nodal Balance (3-30)

 
hb,k+1 (k − 1)

 0 

E1 hb,k+1 (k) − Ẽ2 h̃xc (k) = η 
 ..  (E-15)
.

 
0
where
   
1 0 ... 0 ã 0 . . . 0
−η 1 . . . 0  0 ã . . . 0 
   
E1 = 
 .. .. .. . Ẽ2 =  .. . . . .
  .. 
 . . . .. 
  . . . .

0 . . . −η 1 0 . . . 0 ã
h i
η= hηb,k+1 ã = η 1×nxc

Buffer Capacity (3-31)

hM
   
1 0 ... 0 b,k+1
hM
0 1 . . . 0
  
 b,k+1 
.
. .. . . hb,k+1 (k) ≤  .. 
  (E-16)
. . . . .. 
  . 

0 ... 0 1 hM
b,k+1
−hm
   
−1 0 . . . 0 b,k+1
−hm
 0 −1 . . . 0 
  
 b,k+1 
 .
 . .. .. .  h b,k+1 (k) ≤  ..  (E-17)
 . . . .. 


 .


0 . . . 0 −1 −hm
b,k+1

E-1-5 ProducerBuffer

The constraints for a ProducerBuffer agent are mostly similar to the constraints of the Pro-
ducer Agent, the other constraints can be written as follows.

Nodal Balance (4-3)

 
hb,k+1 (k − 1) + hp (k)

 hp (k + 1) 

E1 hb,k+1 (k) − Ẽ2 h̃(k) − Ẽ3 h̃xc (k) = η 
 ..  (E-18)
.

 
hp (k + Np − 1)
where
     
1 0 ... 0 ã 0 . . . 0 b̃ 0 . . . 0
−η 1 . . . 0  0 ã . . . 0  0 b̃ . . . 0
    

E1 = 
 .. .. .. . Ẽ2 =  .. . .

. . Ẽ3 =  .. . .

. .
 . . . .. 
 . . . . .. 
 . . . . .. 

0 . . . −η 1 0 . . . 0 ã 0 0 . . . b̃
h i h i
η = hηb,k+1 ã = η −η −η b̃ = η 1×nxc

Master of Science Thesis Y.C.C. Putter

120 Agent Constraint Matrices and Vectors

Buffer (Dis)charge (4-5)

hb,k+1 (k − 1) + hup
   
1 0 ... 0 b,k+1
−1 1 . . . 0
  
 hup
b,k+1


 .
 . .. .. ..  hb,k+1 (k) ≤ 
  ..  (E-19)
 . . . .  .


up
0 . . . −1 1 hb,k+1
−hb,k+1 (k − 1) + hdown
   
−1 0 . . . 0 b,k+1
 1 −1 . . . 0 
  
 hdown
b,k+1


 .
 . .. ..  h
..  b,k+1 (k) ≤  ..  (E-20)
 . . . .  
 .


0 . . . 1 −1 hdown
b,k+1

Hybrid Successive Cooling (4-14) and (4-15)

     
1 0 ... 0 a 0 ... 0 −hmc1
 m
0 1 . . . 0 0 a . . . 0 −hc1 
   
−
 .. . . . .  hc1 (k) + 
 .. . . . .  vz (k) ≤ 
 .. 
 (E-21a)
. . . . .. 
 . . . . .. 
  . 
0 ... 0 1 0 ... 0 a −hmc1
     
1 0 ... 0 b 0 ... 0 vzbp − 
 bp
0 1 . . . 0 0 b . . . 0 vz − 
    
.
. .. . . hc1 (k) +  .. . . . . ..  vz (k) ≤  .. 
    (E-21b)
. . . . .. 
 . . . .  . 

0 ... 0 1 0 ... 0 b vzbp − 
     
1 0 ... 0 1 0 ... 0 0
0 1 . . . 0 0 1 . . . 0 0
     
.
. .. . .  vc2 (k) − 
 .. . . . .  vz (k) ≤ 
 .. 
 (E-21c)
. . . . .. 
 . . . . .. 
 .
0 ... 0 1 0 ... 0 1 0

where

a = vzbp − hm
c1
b = vzbp − hM
c1 − 

E-1-6 ConsumerBuffer

Nodal Balance (4-19)

 
hb,k+1 (k − 1) − hd (k)

 hd (k + 1) 

E1 hb,k+1 (k) − E2 e(k) − Ẽ3 h̃xc (k) = η  ..  (E-22)
.
 
 
hd (k + Np − 1)

Y.C.C. Putter Master of Science Thesis

E-1 Agents 121

where
     
1 0 ... 0 η 0 ... 0 ã 0 ... 0
−η 1 . . . 0 0 η . . . 0 0 ã . . . 0 
    

E1 = 
 .. .. .. Ẽ2 =  .. . .
..  
Ẽ3 =  ..
. . ..  
.. .. .. 
 . . . . . . . . . . . . 
0 . . . −η 1 0 ... 0 η 0 0 . . . ã
h i
η = hηb,k+1 ã = η 1×nxc

Two-Tiered Pricing (4-22) and (4-25)

+
 
11×nxc
   
a 0 ... 0 0 ... 0 −hm+
xc
+  m+ 
11×nxc
 
0 a . . . 0  0 ... 0 −hxc 
  
 +
 vz (k) −   h̃xc (k) ≤  (E-23a)
. 
. .. .. ..   ... .. .. ..  .. 

. . . .  . . . 
  . 
0 ... 0 a 0 ... 0 11×nxc
+
−hm+
xc
+
 
11×nxc
   
b 0 ... 0 0 ... 0 vzbp − 
+
11×nxc vzbp − 
 
0 b . . . 0  0 ... 0
    
 +
.
. .. . .  v (k) + 
z  . .. .. ..  hxc (k) ≤ 
 ..  (E-23b)
. . . . .. 
  ..
 . . . 
  . 

0 ... 0 b 0 ... 0 11×nxc
+
vzbp − 
 
g̃
g̃ 
 
+
Ẽ1 vz (k) − Ẽ2 hz (k) − Ẽ3 h̃xc (k) ≤ 
 .. 
 (E-23c)
.
g̃

where

a = vzbp − hm+
xc
b = vzbp − hM+
xc − 
 
0
 0 
g̃ =  m+ 
 
−hxc 
hM+
xc
d˜ 0 . . . 0
     
c̃ 0 . . . 0 ẽ 0 . . . 0
0 c̃ . . . 0
   ˜
0 d . . . 0

0 ẽ . . . 0
 
Ẽ1 =  . . Ẽ2 =  . . Ẽ3 =  . .
. . . . . ...  . . . . . ...  . . . . . ... 
     
 ..   ..   .. 
0 . . . 0 c̃ 0 ... 0 d ˜ 0 . . . 0 ẽ
−hM+
     
xc 1 0
 hm+  −1  0 
d˜ =  
 xc 
c̃ =  m+  ẽ =  
   
 −hxc   1  −1
hM+
xc −1 1

Master of Science Thesis Y.C.C. Putter

122 Agent Constraint Matrices and Vectors

E-2 Global Coupling Constraints

Considering
 h line e = 1 i
a single that couples the first heat energy exchange variable of
1 2
agent i h̃xc,i (k) = hij (k) hik (k) with the first heat energy exchange variable of agent
 h i
j h̃xc,j (k) = h1ji (k) h3jk (k) , using the constant time delay τe , linear loss factor αe and
prediction horizon Np .

h1ji (t) + (1 − α1 )h1ij (t)(t − τ1 ) = 0 ∀t = k . . . k + Np − 1 (E-24)

 
g̃i
0
 
1 1
Ki h̃xc,i (k) + Kj h̃xc,j (k) ≤ 
 .. 
 (E-25)
.
0

where
" #
0τe ×Np −τe 0τe ×τe h i
Ki1 = ⊗ 1 0
(1 − αe )INp −τe 0Np −τe ×τe
h i
Kj1 = INp ⊗ 1 0
h iT
g̃i1 = h1ij (k − τ1 ) . . . h1ij (k − 1)

Furthermore, the following holds

 
Ki = blkdiag Ki1 , Ki2
 
Kj = blkdiag Kj1 , Kj2
h i
Ẽxc (k) = Ki Kj

Y.C.C. Putter Master of Science Thesis

 Bibliography

[1] H. Gadd and S. Werner, “Heat load patterns in district heating substations,” Applied
energy, vol. 108, pp. 176–183, Apr. 2013.

[2] L. Laakkonen, T. Korpela, J. Kaivosoja, M. Vilkko, Y. Majanne, and M. Nurmoranta,

“Predictive supply temperature optimization of district heating networks using delay
distributions,” Energy Procedia, vol. 116, pp. 297–309, Jun. 2017.

[3] J. Duquette, A. Rowe, and P. Wild, “Thermal performance of a steady state physical
pipe model for simulating district heating grids with variable flow,” Applied Energy, vol.
178, pp. 383–393, Jun. 2016.

[4] Europump & the Hydraulic Institute. (2004, May) Variable Speed
Pumping: A Guide to Successful Applications; Executive Sum-
mary. [Online]. Available: https://www.energy.gov/eere/amo/downloads/
variable-speed-pumping-guide-successful-applications-executive-summary. [Accessed
May. 13, 2017].

[5] K. Pfaffendorf, “The Untold Story of Pump Performance Curves & Reli-
ability,” Aug. 2014, [Online]. Available: https://blog.craneengineering.net/
pump-performance-curves-reliability. [Accessed May 13, 2017].

[6] V. D. Stevanovic, S. Prica, B. Maslovaric, B. Zivkovic, and S. Nikodijevic, “Efficient

numerical method for district heating system hydraulics,” Energy Conversion and Man-
agement, vol. 48, no. 5, pp. 1536–1543, Jan. 2007.

[7] "Core Writing Team", R. K. Pachauri, and L. Meyer, Climate Change 2014: Synthesis
Report, L. Meyer, Ed. IPCC, 2014.

[8] UNFCCC. Conference of the Parties (COP), “Adoption of paris climate agreement,”
United Nations Office at Geneva, Dec. 2015, [Online]. Available: https://unfccc.int/
sites/default/files/english_paris_agreement.pdf. [Accessed May 12, 2017].

Master of Science Thesis Y.C.C. Putter

124 Bibliography

[9] SER, “The Agreement on Energy for Sustainable Growth: a policy in practice,” 2015,
[Online]. Available: https://www.energieakkoordser.nl/~/media/files/energieakkoord/
publiciteit/agreement-on-energy-policy-in-practice.ashx. [Accessed May 3, 2017].

[10] Klimaatakkoord, “Voorstel voor hoofdlijnen van het Klimaatakkoord,” Jul. 2018, [On-
line]. Available: https://www.klimaatakkoord.nl/documenten/publicaties/2018/07/10/
hoofdlijnen-compleet. [Accessed July 20, 2018].

[11] Good! (2017) Nationaal Warmtenet Trendrapport. [Online]. Available: https://atriensis.

nl/_userdata/files/Nationaal_Warmtenet_Trendrapport_2017_april_2017.pdf. [Ac-
cessed May 3, 2017].

[12] M. Marijke, R. Matton, R. Segers, J. Vroom, and A. M. Kremer, Monitoring warmte

2015. ECN and CBS, April 2017, no. ECN-E–17-018.

[13] Economische Zaken and H. H. Kamp, “Kamerbrief Warmtevisie,” Apr. 2015, [On-
line]. Available: https://www.rijksoverheid.nl/documenten/kamerstukken/2015/04/02/
kamerbrief-warmtevisie. [Accessed May 16, 2017].

[14] Ministerie van Economische Zaken, “Energierapport: transitie naar duurzaam,” Jan.
2016, [Online]. Available: https://www.rijksoverheid.nl/documenten/rapporten/2016/
01/18/energierapport-transitie-naar-duurzaam. [Accessed March 24, 2017].

[15] N. Hoogervorst, Toekomstbeeld Klimaatneutrale Warmtenetten in Nederland. Den Haag:

Uitgeverij PBL, 2017.

[16] Eneco, “Leiding over Noord: Warmte gemaakt uit... Afval!” 2017, [Online]. Available:
http://projecten.eneco.nl/leiding-over-noord/projectgegevens/warmte-gemaakt-uit/.
[Accessed May 9, 2017].

[17] H. Lund, S. Werner, R. Wiltshire, S. Svendsen, J. E. Thorsen, F. Hvelplund, and B. V.

Mathiesen, “4th Generation District Heating (4GDH): Integrating smart thermal grids
into future sustainable energy systems,” Energy, vol. 68, pp. 1–11, Apr. 2014.

[18] G. Sandou, S. Font, S. Tebbani, A. Hiret, and C. Mondon, “Predictive Control of a Com-
plex District Heating Network,” in IEEE Conference on Decision and Control, vol. 44,
no. 8, Seville, Spain, Dec. 2005, pp. 7372–7377.

[19] H. Runvik, P.-O. Larsson, S. Velut, J. Funkquist, M. Bohlin, A. Nilsson, and S. M. Razavi,
“Production Planning for Distributed District Heating Networks with JModelica. org,” in
Proceedings of the 11th International Modelica Conference, Versailles, France, September
21-23, 2015, no. 118. Linköping University Electronic Press, 2015, pp. 217–223.

[20] S. Trimboli, S. Di Cairano, A. Bemporad, and I. V. Kolmanovsky, “Model predictive

control for automotive time-delay processes: An application to air-to-fuel ratio control,”
IFAC Proceedings Volumes, vol. 42, no. 14, pp. 90–95, 2009.

[21] J. Barreiro-Gomez, G. Obando, C. Ocampo-Martínez, and N. Quijano, “Making Non-

Centralized a Model Predictive Control Scheme by Using Distributed Smith Dynamics,”
IFAC-PapersOnLine, vol. 48, no. 23, pp. 501–506, 2015.

Y.C.C. Putter Master of Science Thesis

 125

[22] A. S. Rene, “Model Predictive Control of District Heating Systems,” mathesis, NTNU,
Jun. 2016.

[23] W. W. Ananduta, “Distributed Energy Management in Smart Thermal Grids with Un-
certain Demands,” mathesis, Delft University of Technology, Aug. 2016.

[24] S. S. Farahani, Z. Lukszo, T. Keviczky, B. De Schutter, and R. H. Murray, “Model

Predictive Control for an Uncertain Smart Thermal Grid,” in Conference on Decision
and Control (CDC), 2015.

[25] V. Rostampour and T. Keviczky, “Robust randomized model predictive control for energy
balance in smart thermal grids,” in Control Conference (ECC), 2016 European. IEEE,
2016, pp. 1201–1208.

[26] H. Larsson, “District Heating Network Models for Production Planning,” mathesis, Lund
University, 2015.

[27] S. Boyd, N. Parikh, E. Chu, B. Peleato, J. Eckstein et al., “Distributed optimization and
Statistical Learning via the Alternating Direction Method of Multipliers,” Foundations
and Trends in Machine learning, vol. 3, no. 1, pp. 1–122, 2011.

[28] K. Kok, “The PowerMatcher: Smart Coordination for the Smart Electricity Grid,”
phdthesis, Technical University of Denmark, 2013.

[29] M. Carrión and J. M. Arroyo, “A Computationally Efficient Mixed-Integer Linear For-

mulation for the Thermal Unit Commitment Problem,” IEEE Transactions on Power
Systems, vol. 21, no. 3, pp. 1371–1378, 2006.

[30] B. d. Schutter and M. Heemels, Lecture Notes: Modelling and Control of Hybrid Systems.
Delft: TU Delft, Aug. 2012, ch. 2, pp. 29–34.

[31] R. Takapoui, N. Moehle, S. Boyd, and A. Bemporad, “A Simple Effective Heuristic for
Embedded Mixed-Integer Quadratic Programming,” in American Control Conference
(ACC). Taylor & Francis, Jul. 2016, pp. 5619–5625.

[32] V. V. Naik and A. Bemporad, “Embedded Mixed-Integer Quadratic Optimization using

Accelerated Dual Gradient Projection,” in 20th IFAC World Congress, vol. 50, no. 1,
2017, pp. 10 723–10 728.

[33] E. H. A. Van Vliet, “Flexibility in Heat Demand at the TU Delft Campus Smart Thermal
Grid with Phase Change Materials,” mathesis, Delft University of Technology, Delft, Feb.
2013.

[34] S. Boyd, L. Xiao, A. Mutapcic, and J. Mattingley, Notes on Decomposition Methods.

Stanford University, May 2015.

[35] B. Kaas, “Model Predictive Control Architecture Comparison for a Power Scheduling
Electric Vehicle Aggregator,” mathesis, Delft University of Technology, Delft, Jun. 2017.

[36] Y. Wang, L. Wu, and J. Li, “A Fully Distributed Asynchronous Approach for Multi-Area
Coordinated Network-Constrained Unit Commitment,” Optimization and Engineering,
pp. 1–34, 2018.

Master of Science Thesis Y.C.C. Putter

126 Bibliography

[37] Biopark Terneuzen, “Folder Biopark 2012,” 2012, [Online]. Available: http://www.
bioparkterneuzen.com/nl/downloads.htm. [Accessed May 05, 2017].

[38] U. Jordan and K. Vajen, “Realistic Domestic Hot-Water Profiles in Different Time
Scales,” Report for IEA-SHC Task, vol. 26, Jan. 2001.

[39] L. Saarinen, “Modelling and Control of a District Heating System,” mathesis, Uppsala
Universitet, Upssala, 2008.

[40] K. Kato, M. Sakawa, K. Ishimaru, S. Ushiro, and T. Shibano, “Heat Load Prediction
through Recurrent Neural Network in District Heating and Cooling Systems,” in 2008
IEEE International Conference on Systems, Man and Cybernetics (SMC 2008). IEEE,
2008, pp. 1401–1406.

[41] The Engineering Toolbox, “Thermal Conductivity of Common Materials

and Gases,” 2003, [Online]. Available: https://www.engineeringtoolbox.com/
thermal-conductivity-d_429.html. [Accessed May 20, 2017]. [Online]. Available:
http://www.engineeringtoolbox.com/thermal-conductivity-d_429.html

[42] M. J. Moran and H. N. Shapiro, Fundamentals of Thermodynamics: sixth edition. John

Wiley & Sons, 2010, ch. 3, pp. 70–113.

[43] R. S. Subramanian, “Heat transfer in Flow through Conduits,” Clarkson Univer-

sity, 2015, [Online]. Available: http://web2.clarkson.edu/projects/subramanian/ch330/
notes/Heat%20Transfer%20in%20Flow%20Through%20Conduits.pdf. [Accessed June. 7,
2017].

[44] A. Mills and P. Education, Basic Heat and Mass Transfer: Pearson New International
Edition, ser. Always Learning. Pearson Education, Limited, 2014. [Online]. Available:
https://books.google.nl/books?id=GogNngEACAAJ

[45] F. M. White, Fluid Mechanics, 7th ed. McGraw Hill, 2011.

[46] K. Ceylan and G. Kelbaliyev, “The Roughness Effects on Friction and Heat Transfer in
the Fully Developed Turbulent Flow in Pipes,” Applied Thermal Engineering, vol. 23,
no. 5, pp. 557–570, 2003.

[47] O. E. Turgut, M. Asker, and M. T. Coban, “A Review of Non Iterative Friction Fac-
tor Correlations for the Calculation of Pressure Drop in Pipes,” Bitlis Eren University
Journal of Science and Technology, vol. 4, no. 1, 2014.

[48] J. Kiijarvi, “Darcy Friction Factor Formulae in Turbulent Pipe Flow,” Lunowa Fluid
Mechanics Paper, 2011.

[49] A. Yan, J. Zhao, Q. An, Y. Zhao, H. Li, and Y. J. Huang, “Hydraulic Performance of a
New District Heating Systems with Distributed Variable Speed Pumps,” Applied energy,
vol. 112, pp. 876–885, 2013.

[50] B. v. d. Heijde, A. Aertgeerts, and L. Helsen, “Modelling Steady-State Thermal Be-

haviour of Double Thermal Network Pipes,” International Journal of Thermal Sciences,
vol. 117, pp. 316–327, 2017.

Y.C.C. Putter Master of Science Thesis

 127

[51] M. Pirouti, A. Bagdanavicius, J. Wu, and J. Ekanayake, “Optimisation of Supply Tem-

perature and Mass Flow Rate for a District Heating Network,” in Proceedings of ECOS
2012 - The 25th International Conference on Efficiency, Cost, Optimization, Simulation
and Environmental Impact of Energy Systems, 2012, pp. 26–29.

[52] S. Grosswindhager, A. Voigt, and M. Kozek, “Efficient Physical Modelling of District

Heating Networks,” Proceedings of the IASTED International Conference on Modelling,
Simulation and Optimatization, Aug. 2011.

[53] M. L. R. v. d. Baar, “Dynamic Modelling and Exergy Analysis of a Smart Building

Heating System,” mathesis, Delft University of Technology, 2016.

[54] R. Scattolini, “Architectures for Distributed and Hierarchical Model Predictive Control:
a review,” Journal of Process Control, vol. 19, no. 5, pp. 723–731, 2009.

[55] R. R. Negenborn, B. De Schutter, and H. Hellendoorn, “Multi-Agent Model Predic-

tive Control of Transportation Networks: Serial versus parallel schemes,” in Proceedings
of the 12th IFAC Symposium on Information Control Problems in Manufacturing (IN-
COMâĂŹ2006) the 12th IFAC Symposium on Information Control Problems in Manu-
facturing (INCOMâĂŹ2006), Saint-Etienne, France, May 2006, pp. 339–344.

[56] O. Miksik, “Tutorial on Decomposition Methods,” Feb. 2014, [Online]. Available: https://
www.robots.ox.ac.uk/~vgg/rg/slides/tutorial_dual_decomposition.pdf. [Accessed Jan.
13, 2018].

Master of Science Thesis Y.C.C. Putter

128 Bibliography

Y.C.C. Putter Master of Science Thesis

 Glossary

List of Acronyms

IPOPT interior point optimizer

ADMM alternating direction method of multipliers

MPC model predictive control

CMPC centralized model predictive control

DMPC decentralized or distributed model predictive control

HMPC hierarchical model predictive control

PWA piece-wise affine

LP linear programming

NLP non-linear programming

QP quadratic programming

MIQP mixed-integer quadratic programming

NP nondeterministic, polynomial time

MCS Monitoring and Control Services

IIS irreducible inconsistent subsystem

LEA Low Energy Architecture

OOP object-oriented programming

EFI energy flexibility interface

P2P peer-2-peer

NDA non-disclosure agreement

Master of Science Thesis Y.C.C. Putter

130 Glossary

DSM demand-side management

CHP combined heat and power

PATCH Ports Adapting To Change

MUP Multi Utility Providing

RES renewable energy sources

MAS multi-agent system

NMDA Niet-Meer-Dan-Anders

3GDH 3rd Generation District Heating

4GDH 4th Generation District Heating

HM HeatMatcher

PM PowerMatcher

DER Distributed Energy Resources

LMP locational marginal pricing

CAPEX capital expenditures

OPEX operational expenditures

ATES aquifer thermal energy storage

Y.C.C. Putter Master of Science Thesis