Parallel Tempering for Constrained Many Criteria
Optimization in Dynamic Virtual Power Plants
Jörg Bremer
Department of Computing Science
University of Oldenburg, Germany
Email: joerg.bremer@uni-oldenburg.de
Abstract—Following the long-term goal of substituting conven-
tional power generation, market oriented approaches will lead to
interaction, competition but also collaboration between different
units. Together with the expected huge number of actors, this
in turn will lead to a need for self-organized and distributed
control structures. Virtual power plants are an established idea
for organizing distributed generation. A frequently arising task
is solving the scheduling problem that assigns an operation
schedule to each energy resource taking into account a bunch of
objectives like accurate resemblance of the desired load profile,
robustness of the schedule, costs, maximizing remaining flexibility
for subsequent planning periods, and more. Nevertheless, also
such dynamic approaches exhibit sub-problems demanding for
centralized solutions for ahead of time scheduling of active power.
In this paper we develop a hybrid approach combining the
advantages of parallel tempering with a constraint handling
technique based on a support vector decoder for systematically
generating solutions; thus ensuring feasible overall solutions. We
demonstrate the applicability with a set of simulation results
comprising many objective scheduling for different groups of
energy resources.
I. I NTRODUCTION
The increasing pervasion of the distribution grid with
renewable energy resources imposes fluctuating and hardly
predictable feed-in and demands new management strategies.
On the other hand, combined with controllable, shiftable loads
and electrical storages, these energy units set up a new flexibil-
ity potential that may be used to full capacity when harnessing
ICT-based control. In order to enable a transition of the current
central market and network structure of todays electricity grid
to a decentralized smart grid, an efficient management of
numerous distributed energy resources (DER) will become
more and more indispensable. Integrating a continuously rising
number of renewable resources means controlling individually
configured and rather small devices and coping with growing
stochastic feed-in effects.
In European countries, especially in Germany where cur-
rently a financial security of guaranteed feed-in prices is given,
the share of DER is rapidly rising. Following the goal defined
by the European Commission [1], a concept is needed to
integrate them into electricity markets for both: active power
provision and ancillary services [2], [3] to reduce subsidy
dependence. For the rest of the paper we will stick with active
power provision. A well known concept for aggregating DER
to a controllable entity are virtual power plants (VPP). Apart
from controlling distributed electricity generation, e.g. com-
bined heat and power (μCHP), photovoltaic or wind power,
978-1-4799-4546-7/14/$31.00 ©2014 IEEE
Michael Sonnenschein
Department of Computing Science
University of Oldenburg, Germany
Email: michael.sonnenschein@uni-oldenburg.de
controllable consuming processes like shiftable loads, heat
pumps or air conditioning might also be included for planning
active power schedules. Battery storages are discussed to
complement such groups of DER. We consider in general
producers that are supposed to pool together with likewise
distributed electricity consumers and prosumers (like batteries)
in order to jointly gain more degrees of freedom in choosing
load schedules. In this way, they become a single controllable
entity with sufficient market power.
Virtual power plants are a well-known instrument for
aggregating and controlling DER [4]. Integration into current
market structures recently let to VPP systems that frequently
re-configure themselves for a market and product specific
alignment [5]. VPP concepts for several purposes (commercial
as well as technical) have been developed. A usual use case
commonly emerging within VPP control is the need for
scheduling the participating DER. Independently of the specific
objective at hand, a schedule for each DER has to be found
such that the schedule that finally is assigned to a DER is
operable without violating any technical constraint. Ensuring
the feasibility of a VPP scheduling solution is a crucial task
[6]. For this paper we go with the example of scheduling for
active power planning in day-ahead scenarios (not necessarily
24 hours but for some given future time horizon).
For large scale problems, distributed (usually agent based)
approaches are currently discussed not least due to further
advantages like ensured privacy issues. Some recent imple-
mentations are [7], [8], [9]. Distributed organization and self-
organized control is also especially a characteristic of dynamic
virtual power plants (DVPP) [5]. Nevertheless, there are still
cases where a centralized algorithm applies to the scene: units
that internally are composed of several sub-units controlled
by a single agent like a household or a hotel may serve as
an example. Even in distributed controlled VPP such entities
would make-up a sub-problem with a central controller in
charge of orchestrating this sub-group.
Here as well as in classical VPP approaches an efficient and
robust central scheduling method is needed for several power
planning tasks. For these cases, we propose a hybrid and easily
adaptable solution that takes into account the individuality of
different DER and constraints and thus ensures the feasibility
of the scheduling result.
The rest of the paper is organized as follows. We start with
briefly introducing the concept of a dynamic virtual power
plant and the active power planning problem arising therein.
We continue with a description of our solution for the cases
where a centralized approach is necessary. The solution com-
prises the adaption of a support vector decoder for constraint
handling and its integration into a Simulated Annealing step
as well as a Parallel Tempering framework that controls the
optimization process. We finally present simulation results that
evaluate our approach.
II. R ELATED WORK
The operational management of an energy system com-
prises complex tasks ranging from technical monitoring as-
pects to organizational measures and business management
coupled within an energy management system. Traditionally,
energy management is implemented as centralized control.
However, given the increasing share of DER as well as flexible
loads in the distribution grid today, the evolution of the
classical, rather static (from an architectural point of view)
power system to a dynamic, continuously reconfiguring system
of individual decision makers (e. g. as described in [10]), it is
unlikely for such centralized control schemes to be able to cope
with the rapidly growing problem size. Thus, the seminal work
of Wu et al. [11] identified the need for decentralized control.
The International Energy Agency (IEA) proposed the con-
cept of a virtual utility (which was originally introduced in the
late nineties) as one means to address the transition to decen-
tralized control by a flexible and market-driven collaboration
of independent entities [4], [12]. As a consequence, the concept
of virtual power plants (VPP) with different operational targets
(commercial as well as technical) has been derived and studied
extensively [13], [14]. A list of realizations can for example be
found in [15], [16]. Usually, such VPP focus on the long-term
aggregation of generators (and sometimes battery storages)
only and are mostly still operated in a centralized manner.
Recently, decentralized control schemes have become a re-
search topic. For instance, [17], [18] survey the use of agent-
based control methods for power engineering applications.
Exemplary applications can be found in [19], [8], [20].
In order to address the integration of the current market
situation, [5] introduces the concept of a dynamic virtual power
plant (DVPP). In this approach VPPs gather dynamically to-
gether with respect to concrete electricity products at an energy
market and will diverge right after delivery. Additionally, fully
distributed control algorithms are used that – in conjunction
with the dynamic composition of DER – do no longer allow
for a static, a priori formulation of the optimization problem
that has to be solved by the VPP for scheduling.
Each DER first and foremost has to serve the purpose it
has been built for. But, usually this purpose may be achieved
in different alternative ways. For example, it is the (intended)
purpose of μCHP to deliver enough heat for the varying heat
demand in a household at every moment in time. Nevertheless,
if heat usage can be decoupled from heat production by use
of a thermal buffer store, different production profiles may be
used for generating the heat. This leads, in turn, to different
respective electric load profiles that may be offered as alterna-
tives to a VPP controller. The set of all schedules that a DER
may operate without violating any technical constraint (or soft
constraint like comfort) is the sub-search-space with respect
to this specific DER from which a scheduling algorithm may
choose solution candidates. Geometrically seen, this set forms
a sub-space F ⊆ Rd in the space of all possible schedules.
In [6] a model has been proposed to derive a description for
this sub-space of feasible solutions that abstracts from any
DER model and its specific constraint formulations. These
surrogate models for the search spaces of different DER may
be automatically combined to a dynamic optimization model
by serving as a means that guides an arbitrary algorithm where
to look for feasible solutions. Due to the abstract formulations
all DER may be treated the same by the algorithm. A detailed
description of the approach may be found in [21], [22]. So
far it has been used with distributed optimization in single
objective cases, for example in [7].
We will here extend this approach by integrating it into
a many objective Parallel Tempering (PT) algorithm. The
centralized approach that we use here is appropriable for a
specialized sub-problem of DVPP: It is likely to have entities
within a DVPP that internally consist of several units that are
connected to the containing DVPP by a single agent. An exam-
ple may be a hotel internally consisting of several DER. The
DVPP control will assign an operation schedule to the hotel
and it will be in the responsibility of the energy management
of the hotel to partition this schedule into schedules for the
different electrical units; what would then be a VPP control
task with central control on a lower hierarchy level. In this
sense, the solution we propose for this problem is also suitable
for classical VPP that wants to ensure feasibility.
III. T HE ACTIVE POWER PLANNING PROBLEM
We start with some preliminary definitions. First, let U
be the set of DER units in the VPP and ZU be the set of
operational states of unit U . We regard the schedule of an
energy unit as a vector p = (p1 , . . . , pd ) ∈ Rd of mean power
generated (or consumed) during the ith time interval; not as a
time series. The starting time and the width of a time interval
(today usually 15 minutes) are defined separately. For the used
support vector decoder it is advantageous to use schedules
with scaled power values [6]. Scaling is done according to
respective minimum (pm in) and maximum (pm ax) nominal
active power output:
x[0,1] = (x0 , x1 , . . . , xd ) ∈ [0, 1]d ,
pi − pmin (1)
with xi = .
pmax − pmin
Individual schedules do have individual impacts on dif-
ferent optimization objectives. Thus, we have to extend the
schedule definition in order to describe schedules with respect
to these objectives. In general, with the term indicator we
denote a parameter that characterizes a schedule with respect to
the progress to some goal that we want to achieve (objective)
and that thus can be used by the optimizer to evaluate candi-
date solutions for comparison. Usually, the term performance
indicator is used in industry [23] with a broader meaning.
For simplicity we will use the term indicator for the rest of
the paper and denote with indicator function I an instruction
to assign a real value to an indicator and not just a binary
membership relation as sometimes in other use cases. For
different objectives different indicators are used. Let K be the
(U )
set of indicators. Let p ∈ Rd be a schedule and x ∈ F[0,1] be
its scaled variant. Then,
(K)
I (x, z0 ) : R × ZU → R,
d
x → I (K) (x, z0 )
is a mapping that unambiguously assigns a value for indicator
K ∈ K to schedule x based on the assumption that x is
operated starting from the operational state z0 . Then
(K)
I[0,1] (x, z0 ) : Rd × ZU → [0, 1],
I (K) (x, z0 )
x → (K)
maxx∈Rd ,z∈ZU {I (x, z)}
maps to the scaled indicator K[0,1] ∈ [0, 1]. This will be helpful
for the integration into scaled schedules in the model.
With this preliminaries, we can now define the scheduling
problem. Let ζ = (p0 , p1 , . . . , pd ) ∈ Rd be an electricity
product that defines for subsequent time intervals the mean
active power that a group of units U is supposed to generate
(or consume if applicable). Thus, ζ defines a schedule that a
given set of DER has to operate jointly. We want to find a
schedule for each individual unit U ∈ U.
Let pi be the schedule that is operated by unit Ui . Let ζ be
the wanted schedule and K the set of describing performance
indicators. Let F (U ) be the feasible region of unit Ui that
contains the feasible schedules that might be operated by the
unit. Let I (K,Ui ) be a unit specific mapping from the electrical
schedule to the value of indicator K. Then, the active power
planning problem is defined as
 n  After mapping the data to a high dimensional feature space
 by means of an appropriate kernel, the smallest enclosing ball
δ pi , ζ → min (4)
i=1
 ball to data space, it forms a set of contours (not necessarily
(K,Ui )
∧ I (pi , z0Ui ) → optKi ∈ {min, max} ∀K ∈ K
U ∈U
such that
pi ∈ F (Ui ) ∀Ui ∈ U.
Here δ is an (in general) arbitrary distance measure for
evaluating the difference between the aggregated schedule of
the group and the desired target schedule. W.l.o.g., in this
contribution we use the Euclidean distance  · 2 .
IV. T HE A LGORITHM
A. Support Vector Decoder
We start with a description of the constraint handling
technique that we adopted for our approach. In [21] a so called
support vector decoder has been introduced. Basically, a de-
coder is a constraint handling technique that gives an algorithm
hints on where to look for feasible solutions. It imposes a
relationship between a decoder solution and a feasible solution
and gives instructions on how to construct a feasible solution
[24]. For example, [25] proposed a homomorphous mapping
between an n-dimensional hyper cube and the feasible region
in order to transform the problem into an topological equivalent
one that is easier to handle. In order to be able to derive such
a decoder mapping automatically from any given energy unit
model, [21] developed an approach based on a support vector
model [6]. We will briefly describe this method.
Ψ̂x
Ψ̃x
(2) x
x∗
RS
(3)
Rd H()
Fig. 1. General support vector model and decoder scheme for solution repair
and constraint handling.
The basic idea is to start with a set of feasible example
schedules derived from the simulation model of an energy
unit and use this sample as a stencil for the region (the sub-
space in the space of all schedules) that contains the feasible
schedules. The schedule sample is then used as a training set
for a support vector based machine learning approach [26] that
derives a geometrical description of the sub-space that contains
the given data (in our case: the feasible schedules). Given a set
of data samples, the inherent structure of the scope of action
of a unit where the data resides in can be derived as follows:
in this feature space is determined. When mapping back this
connected) enclosing the given data sample.
(5) At this point, the set of alternatively feasible schedules
of a unit is represented as pre-image of a high-dimensional
(6) ball S. Figure 1 shows the situation. This representation has
some advantageous properties. Although the pre-image might
be some arbitrary shaped non-continuous blob in Rd , the high-
dimensional representation is still a ball and thus geometrically
easier to handle (right hand side of figure 1). The relation is as
follows: If a schedule is feasible, i.e. can be operated by the
unit without violating any technical constraint, it lies inside
the feasible region (grey area on the left hand side in figure
1). Thus, the schedule is inside the pre-image (that represents
the feasible region) of the ball and thus its image in the high-
dimensional representation lies inside the ball. An infeasible
schedule (e. g. x in Fig. 1) lies outside the feasible region and
thus its image Ψ̂x lies outside the ball. But we know some
relations: the center of the ball, the distance of the image from
the center and the radius of the ball. Hence, we can move the
image of an infeasible schedule along the difference vector
towards the center until it touches the ball. Finally, we calculate
the pre-image of the moved image Ψ̃x and get a schedule at the
boundary of the feasible region: a repaired schedule x∗ that
is now feasible. We do not need a mathematical description
of the original feasible region or of the constraints to do this.
More sophisticated variants of transformation are e. g. given in
[22]. For a detailed description of the support vector decoder
approach we refer to [22], [27].
 Formally, we have a mapping (the decoder γ)
γ : [0, 1] → F[0,1] ⊆ [0, 1]
d d
x → γ(x)
that transforms any given (maybe in-feasible) schedule into a
feasible one. Thus, we are able to transform the scheduling
problem given into an unconstrained formulation. But first,
performance indicators for evaluation with respect to different
objectives have to be integrated for the many-objective case.
In the support vector model, indicators can be integrated
as follows: data vectors containing the mean power levels
for the respective time intervals are extended by one element
per indicator. Thus they become mixed feature vectors. This
approach has been proposed for environmental performance
indicators [28], but in general, arbitrary indicators can be added
in this way as long as a functional relationship exists between
the power part of the vector and the value of the indicator.
In this way, we can build a modified sample X  with
schedules that integrate our model of performance indicators
and construct
(K ) (K )
x = (x1 , . . . , xd , I[0,1]1 (x), . . . , I[0,1]m (x)),
as vectors x ∈ [0, 1]d+m with the first d elements denoting
scaled real power and m trailing elements denoting scaled
indicator values. This sample is fed into exactly the same
support vector training process to build the model. Mapping γ
for the decoder is derived in exactly the same way as described
above. The decoder mapping γ then likewise maps feature
vectors x ∈ X 
[0, 1] d+m
→ F[0,1] × [I[0,1] ] m
 K1 Km
γ(x ) → (x, I[0,1] (x), . . . , I[0,1] (x)).
If x = (x1 , . . . , xd , k1 , . . . , km ) ∈ [0, 1]d+m is given with
arbitrary values then γ(x ) is a vector that contains a feasible
active power schedule in the first d elements as well as m
elements evaluating this schedule correctly (slight inaccuracies
are possible) with regard to the secondary optimization objec-
tives in scaled form. When using a decoder that automatically
transposes any arbitrary solution from the constraint-free for-
mulation to a corresponding feasible solution, the scheduling
problem Eq. (4) simplifies to
 d  the one that we scrutinize here, this problem is even more
 prevalent [35]. Several techniques have been proposed to
δ si ◦ γi (xi ), ζ → min,
i=1
where γi denotes the decoder of unit i that produces feasible
schedules x ∈ [0, 1]d+m and si scales these schedules
entrywise to correct power values resulting in schedules that
are operable by that unit. Please note that in cases where
power or performance indicator do not range between zero
and a positive maximum value (consumers, batteries, etc.),
the un-scaling procedure is slightly different. Minimizing the
difference on an element that denotes an indicator that is to be
maximized is correct if the target value is set to the maximum
value (sum over all units).
Using a decoder fairly eases the implementation of a
solver because no complex constraints have to be considered.
On the other hand, such a decoder may introduce additional
complexity into the optimization problem by the transforma-
tion. For this reason, we scrutinized the fitness landscapes
of both problems (untransformed and transformed) to gain
(7)
insight into the problem structure with means from standard
fitness landscape analysis [29]. Indeed, our findings indicate
a growing complexity by an increased ruggedness with a
growing number of local minima. But, this situation can be
easily countered by using a heuristic that copes well with
rugged non-linear problems like Simulated Annealing or its
variants.
B. Simulated Annealing Step
Simulated Annealing [30] is an established Markov Chain
Monte Carlo Methods (MCMC) for non-linear optimization. It
mimics a physical cooling process. In general, MCMC meth-
ods are an effective tool for statistical sampling applied to op-
timization problems [31]. The basic idea is a Markov Process
that samples a target probability distribution π(x) = z1 e−E(x)
with z as a problem specific normalization parameter and E
measuring the error of the optimization objective. Originally,
the method has been mainly applied to physical problems
finding a minimum energy state and thus E is sometimes still
(8) written Hamiltonian H, e.g. in [32]. We will use the term E. In
this process a new state σt+1 is generated from σt by drawing
from a proposal transition distribution Q(σt+1 |σt ) [33], [34].
The new state is accepted with probability
 
π(σt+1 Q(σt+1 |σt ))
A(σt → σt+1 ) = min 1, . (11)
π(σt)Q(σt |σt+1 )
The proposal distribution Q is a free parameter and must
be adjusted to the individual problem at hand. Starting from a
random initial state σ0 , the process needs a while to reach
(9)
equilibrium and independence from σ0 . After this burn-in
phase the samples represent the target distribution π.
In systems with deep local minima the process can be
trapped without escape in reasonable time. This waiting time
dilemma [35] is due to a stringent requirement for equilibrium.
To escape, the process must generate subsequent states with
higher energy and the probability for such a move declines
roughly exponentially with the energy differences that has to be
overcome. Thus, the expected waiting time for such an escape
grows also exponentially. For high-dimensional problems like
(10) overcome the problem of getting trapped, e.g. [36], [35], [37];
one is the concept of Simulated Annealing (SA).
SA introduces a variable temperature T into the target
distribution: π(x) = z1 e−E(x)/T . The effect is that the Markov
Chain may escape local minima easier at a higher temperature.
The general idea of Simulated Annealing is to interpret the
fitness landscape of an optimization problem as a thermody-
namic system with the objective function E(x) denoting the
error interpreted as the energy level of a proposed solution
x. Initially, the system is at a high temperature. During the
Markov process, the system is gradually cooled down to the
ground state with the global energy minimum.
We use a Parallel Tempering (PT) approach as a parallel
execution of several Simulated Annealing steps. We start our
description with the inner SA step.
 Figure 2 shows the basic flow within our SA. By mimicking Xij ← xi ∼ U (0, 1)d , 1 ≤ i ≤ n
a cooling process, temporarily worse solutions are allowed – Mij ← γi (Xi ), 1 ≤ i ≤ n
depending on temperature and difference in solution quality – ϑ ← ϑstart
in order to escape local minima. In our approach, a solution while ϑ < ϑmin do
is described by two matrices Xij and Mij denoting for each choose random k; 1 ≤ k ≤ n
energy unit i and for each time interval j of the schedule a x∗ ← X k
scaled active power value in [0, 1] and in the many objective mutate(x∗ )
case indicator values that describe the schedule with respect to M ∗ ← M ; Mk∗ ← γk (x∗ )
E(M ∗ )−E(M )
different objectives (cf. Eq. (8)). In this sense, each row within if e− T > r ∼ U (0, 1) then
the matrix is the schedule (feature vector) for one of the units. M ← M ∗ ; X k ← x∗
X contains schedules from the unconstrained search space end if
(hypercube [0, 1]d not further constrained by technical issues T ← cooling(T )
from the units operations). X is initialized with random values. end while
M concurrently holds the respective feasible values generated
by the support vector decoder: Mi = γi (Xi ). Thus, M always Fig. 2. Basic scheme for the Simulated Annealing step (with integrated
represents a feasible (scaled) solution to the problem. support vector decoder) during the Parallel Tempering process.

For the many objective case we define the optimization

target ζ as follows: xi at level i is accepted (according to the Metropolis-Hastings
ζ = (p1 , . . . , pd , ν1 , . . . , νm ), (12) criterion) with probability
where νi denotes the additional objectives that are to be
achieved apart from resembling the electrical target schedule
 with Δi E = E(xt+1 ) − E(xti ).
0 if additional objective i is to be minimized
νi = (13)
n else (scaled case)
and use respective feature vectors for training the decoders of
the units. In each iteration of the SA exactly one schedule x
from X is randomly chosen and mutated. Modification is done
at a randomly chosen element xk by adding a random value
p ∼ N (0, 1):
⎧ between neighboring temperature levels should take place. In
⎨xk + p − 1 if xk + p > 1
xk ← xk + p + 1 if xk + p < 0 (14)
⎩
xk + p else.
Additionally, it can be useful especially for high-dimensional
schedules to allow mutations at more than one element at a
time. Only this mutated schedule has to be mapped by the
respective decoder in order to keep M consistent with X.
Finally, M is a feasible, mutated solution that can be
evaluated by Eq. (10). The Metropolis criterion Eq. (15) then
decides whether this new solution is kept or discarded. A major
advantage of this approach is that always a feasible solution
exists. The Markov chain may evolve in [0, 1](d+m)·n without
taking care of technical constraints of the individual energy
units. The decoder guarantees (apart from minor inaccuracies
that might easily be corrected [22]) the feasibility of the
solution.
C. Parallel Tempering
The general scheme for Parallel Tempering as proposed by
[38], [39] is as follows: A system of n replicas of a MCMC
system at different temperature levels Ti is established and
in parallel executed. The Markov chain at each temperature
level is realized with two types of transition: the Metropolis
transition as described above and an additional exchange
transition that exchanges solutions from two neighbouring
replicas i and i + 1.
The system evolves as follows: at each temperature level
Ti a Markov chain samples E(xi ). The new proposal solution
 −Δi E 
A(xti → xt+1
i ) = min 1, e Ti
, (15)
i
At each iteration a replica transition between temperature
levels i and i + 1 may take place with probaility (cf. [40])
 
−
(E(xi+1 ) E(x ) E(x ) E(x
− T i + T i + T i+1
A(xi ↔ xi+1 ) = min 1, e Ti i+1 i i+1 .
(16)
An important question is, how often such solution exchange
[40] a global temperature has been introduced into Eq. (16) to
be able to adjust the rate. Schug and Wenzel [41] empirically
determined an exchange ratio of 0.5% to 2% for good results.
We adapted the latter one. Another important question is
the best scheme for cooling the temperatures. We used a
proportional cooling scheme: Tt+1 = Tend +(Tt −Tend )·c, cf.
e. g. [31], that concurrently cools down all temperature levels
to an final temperature Tend with a configurable rate c.
V. S IMULATION R ESULTS
A. Setup
Evaluation was done by simulation. This work is part of the
Smart Nord research cluster (http://smartnord.de) and within
this framework simulations comprising large numbers of dif-
ferent types of DER are conducted for the DVPP case. The
list of scrutinized types incorporates for example co-generation
devices with thermal buffer storage and a simulated residential
thermal energy demand, controllable cooling devices, heat
pumps, battery storages and shiftable loads like e.g. dishwash-
ers and many more. Nevertheless, for the results presented here
we will focus on clusters of μCHP units grouped together with
battery storages for a better comparability.
Each simulation of a μCHP unit also comprises the simu-
lation of individual thermal demand by modeling a detached
house with its several heat losses (heater is supposed to keep
the indoor temperature on a constant level) and randomized hot
water drawings (assumed to be predictable) for gaining more
diversity among the devices. In order to have more degrees of
 35,0 target schedule
32,5 aggregated schedule
power / kW

30,0
27,5
25,0
22,5

0,02
error / kW

0,01

0,00

-0,01

-0,02
35
indv. power / kW

30
25
20
15
10 Fig. 4. Distributions of the final state of charge configurations for a group
5 of 20 μCHP units as a second optimization objective. Depicted are the
0
0 8 16 24 32 40 48 56 64 72 80 88
distribution for the whole group in case of disregarding this objective as well as
time interval two distributions for the two objective case (one group wanting a discharged
and one wanting a charged store). The problem comprises schedules for 8
hours with a 15 minute resolution.
Fig. 3. A single objective example for a group of 15 μCHP and schedules
for a whole day.

freedom for controlling the generation of active power, we used

the model of an modulating μCHP-plant with the following
specification (technical constraints):
• Minimum electrical/ thermal power: 1.3/ 4 kW (or
zero if OFF),
• Maximum electrical/ thermal power: 4.7/ 12.5 kW,
• After turning on, the device has to stay on until it
reaches operating temperature
• After shutting down, the device has to stay off for at
least 2 hours
The relationship between electrical (active) power and thermal
power was modeled after engine test benches from [42]. In
order to gain enough degrees of freedom for varying active
power, each μCHP is equipped with an 800  thermal buffer
store. As a further operating constraint for our optimization
problem, buffer temperature has to stay within a given range.
All batteries have been been modeled after [43]. Unless
otherwise stated, all simulations have been repeated 200 times.
For the PT method we used 5 temperature levels initialized
with a temperature of 2, 4, 8, 16, and 32. The target temper-
ature was set to 0.0001 and the cooling rate was adjusted so
as to result in approximately 250.000 iterations.
B. Results
Figure 3 shows a typical result for the single objective
of only resembling the wanted target schedule as close as
possible. In the following, we are more interested in the many
objective case. As a first test case, we divided a group of μCHP
with attached thermal buffer stores into two groups: one group
wanting to have their buffers charged after operating the target
schedule (because they were awaiting a high thermal demand)
and another group wanting their buffers rather discharged;
say, because they want to turn the units on for an afterwards
delivery of another product. Figure 4 shows an exemplary
result for a group of 20 units. After integrating this additional
Fig. 5. Distributions of the final state of charge configurations for a group
of 100 μCHP units as a second optimization objective. Although this time
schedules for merely 4 hours have been considered a better separation of the
two classes has been achieved due to the larger number of units in comparison
to the case in figure 4.
objective as described above, the distribution of the final states
of charge (SOC) of the buffer stores is split up with respect to
the wanted state (compared with the single objective case that
ignores this objective. Figure 5 shows an example for a group
of 100 units with an (as expected) higher effect.
Of course, achieving such additional objectives has an
effect on the primary objective of resembling the target sched-
ule as close as possible. Table 6 compares the improvements
achieved for the state of charge objective with the degradation
in quality of the primary objective. The larger the group the
more potential can be harnessed for achieving the second
objective. The induced additional error (measured in kW h per
time interval as in Fig. 3) of the whole group stays acceptably
small (compared with the mean product contribution of 0.62
kW h per unit and time interval). For longer schedules the
additional error for the primary objective is smaller, obviously
because the scheduling algorithm has more degrees of freedom
to achieve the same second objective.
An important question is how fast the algorithm converges.
 TABLE I. C OMPARISON OF OBJECTIVE IMPROVEMENTS FOR TWO
OBJECTIVES ( RESEMBLANCE OF THE GIVEN TARGET SCHEDULE AND
considered apart from the primary objective, the efficiency
ACHIEVEMENT OF AN DESIRED INDIVIDUAL FINAL STATE OF CHARGE ) of the modulating μCHP when operated at different degrees
FOR 4 AND 12 HOUR SCHEDULES WITH 15 MINUTE RESOLUTION . A of capacity utilization (with different CHP equipped with dif-
SMALLER VALUE FOR THE FINAL SOC DENOTES A SMALLER ERROR . ferent non-linear characteristics), energy losses resulting from
dim. d # units SOC improvement / % add. ΔE / kWh·t intermediate storage in the thermal buffer store (the battery has
5 -4.9363 ± 2.5835 0.0116 ± 0.0016
been considered lossless for the rather short time frame) and
16 25 -13.7708 ± 3.0391 0.1112 ± 0.0517
remaining flexibility for succeeding planning periods. For the
100 -17.0471 ± 1.5426 0.6561 ± 0.0951 latter it has been considered advantageous to achieve a medium
5 -4.6883 ± 5.4528 0.0068 ± 0.0022 charged thermal buffer in order to conserve degrees of freedom
48 25 -12.4881 ± 2.0393 0.0076 ± 0.0031 in both directions for afterwards scheduling operations. As
100 -20.4150 ± 1.5248 0.1527 ± 0.0337 indicator for the latter, an error function measuring the absolute
deviation from 50% SOC has been used. Figure 7 shows an
4,50
4,45 Parallel Tempering
3,93000

3,92975
Parallel Tempering
exemplary result for a group of 5 μCHP grouped with one
4,40
4,35
Simulated Annealing
3,92950
Simulated Annealing
battery for 96 time intervals of 15 minutes.
4,30 3,92925

4,25 3,92900
Error

Error

4,20
3,92875
4,15
4,10
4,05
3,92850

3,92825
VI. C ONCLUSION
4,00
3,92800
3,95
3,90
3,85
3,92775

3,92750
We have presented a hybrid approach for scheduling the
0 2.500 5.000 7.500 10.000
Iteration
12.500 15.000 17.500 20.000 75.000 100.000
Iteration
125.000 150.000
active power production of distributed energy resources in
(a) (b) dynamically organized virtual power plants. Our approach
70
takes into account individual operating constraints and thus
52,65
69
68
Parallel Tempering
Simulated Annealing 52,60
Parallel Tempering
Simulated Annealing always produces feasible solutions that are operable by the
67
66
52,55

52,50
different units. The use of parallel tempering has shown
65
good performance when it comes to centralized optimization.
Error

Error

52,45
64
63
62
52,40

61
60
59
58
0 2.500 5.000 7.500 10.000 12.500
Iteration
(c)
Fig. 6. Average convergence behavior of PT compared with simple SA in
a two objectives case. The figures in the top row show a rather simple case
with 5 μCHP and 16 time intervals, the bottom row shows an example with
100 μCHP and 32-dimensional schedules. The figures on the left 6(a) and
6(c) show the faster convergence of PT at the first iterations.The figures on
the right 6(b) and 6(d) depict the respectively reached final residual error.
We compared the Parallel Tempering approach with a simple
Simulated Annealing. Figure 6 shows some results showing
that PT achieves a faster convergence as well as better results.
These Experiments have been done with target schedules that
are not guaranteed to be operable by the VPP and with the
splitting of the buffer SOC as a second objective.
Finally, we studied an experiment with four objectives
for groups of μCHP and battery storage. In this case we
52,35
Nevertheless, using a support vector decoder for modeling
52,30
individual feasible regions as an abstraction layer for arbitrarily
52,25
15.000 17.500 20.000
52,20
80.000 90.000 100.000 110.000 120.000 130.000 140.000 150.000
modeled, different types of energy resources, easily allows for
Iteration
dynamically fitting arbitrary units into a common optimization
(d) model. Thus, we are able to automatically derive the con-
strained optimization model for dynamically formed coalition
on the fly. So, the approach is suitable also in situations
where setting up a static and reusable model in advance is not
possible due to dynamic changes in the VPP composition. In
this way, our approach easily fulfills the usual requirement for
a self controlled system of autonomously acting units. Parallel
Tempering extensions to distributed execution are also already
known [40].
Currently, we are studying the extension of the approach
to scenarios with complex power for handling active and
reactive power for ancillary services (e.g. for voltage stability)
at the same time in order to integrate even more types of
objectives into optimization. Integrating different weightings
of the objectives will be another topic.
ACKNOWLEDGMENT

50
The Lower Saxony research network ’Smart Nord’ ac-
40
knowledges the support of the Lower Saxony Ministry of
Science and Culture through the ’Niedersächsisches Vorab’
30
grant programme (grant ZN 2764/ZN 2896).
change / %

20

10
R EFERENCES
0

-10 [1] European Parliament & Council, “Directive 2009/28/ec of 23 april

-20
efficiancy
Fig. 7. Example result with four objectives (maximize efficiency of the CHP,
maximize remaining flexibility, minimize energy losses due to intermediate
storage, minimize difference in terms of active power deviation between
aggregated and target schedule) for a group of only 5 μCHP and a battery.
2009 on the promotion of the use of energy from renewable sources
and amending and subsequently repealing directives 2001/77/ec and
flexibility losses error 2003/30/ec.”
[2] O. Abarrategui, J. Marti, and A. Gonzalez, “Constructing the active
european power grid,” in Proceedings of WCPEE09, Cairo, 2009.
[3] A. Nieße, S. Lehnhoff, M. Tröschel, M. Uslar, C. Wissing, H.-J. Ap-
pelrath, and M. Sonnenschein, “Market–based self–organized provision
of active power and ancillary services.” IEEE, 06 2012.
 [4] S. Awerbuch and A. M. Preston, Eds., The Virtual Utility: Accounting,
Technology & Competitive Aspects of the Emerging Industry, ser. Topics
in Regulatory Economics and Policy. Kluwer Academic Publishers,
1997, vol. 26.
[5] A. Nieße, S. Beer, J. Bremer, C. Hinrichs, O. Lünsdorf, and M. Son-
nenschein, “Conjoint dynamic aggrgation and scheduling for dynamic
virtual power plants,” in Federated Conference on Computer Science
and Information Systems - FedCSIS 2014, Warsaw, Poland, M. Ganzha,
L. A. Maciaszek, and M. Paprzycki, Eds., 9 2014.
[6] J. Bremer, B. Rapp, and M. Sonnenschein, “Encoding distributed
search spaces for virtual power plants,” in IEEE Symposium Series on
Computational Intelligence 2011 (SSCI 2011), Paris, France, 4 2011.
[7] C. Hinrichs, J. Bremer, and M. Sonnenschein, “Distributed Hybrid
Constraint Handling in Large Scale Virtual Power Plants,” in
IEEE PES Conference on Innovative Smart Grid Technologies
Europe (ISGT Europe 2013). IEEE Power & Energy Society,
2013. [Online]. Available: http://www-ui.informatik.uni-oldenburg.de/
download/Publikationen/HBS13.pdf
[8] S. D. Ramchurn, P. Vytelingum, A. Rogers, and N. R. Jennings, “Agent-
based homeostatic control for green energy in the smart grid,” ACM
Trans. Intell. Syst. Technol., vol. 2, no. 4, pp. 35:1–35:28, Jul. 2011.
[9] R. Kamphuis, C. Warmer, M. Hommelberg, and K. Kok, “Massive co-
ordination of dispersed generation using powermatcher based software
agents,” in 19th International Conference on Electricity Distribution,
05 2007.
[10] M. D. Ilić, “From hierarchical to open access electric power systems,”
Proceedings of the IEEE, vol. 95, no. 5, pp. 1060–1084, 2007.
[11] F. Wu, K. Moslehi, and A. Bose, “Power system control centers: Past,
present, and future,” Proceedings of the IEEE, vol. 93, no. 11, pp.
1890–1908, 2005.
[12] International Energy Agency, Distributed Generation in Liberalised
Electricity Markets. OECD Publishing, 2002.
[13] S. Lukovic, I. Kaitovic, M. Mura, and U. Bondi, “Virtual power plant as
a bridge between distributed energy resources and smart grid,” Hawaii
International Conference on System Sciences, vol. 0, pp. 1–8, 2010.
[14] M. Tröschel and H.-J. Appelrath, “Towards reactive scheduling for
large-scale virtual power plants.” in MATES, ser. Lecture Notes in
Computer Science, L. Braubach, W. van der Hoek, P. Petta, and
A. Pokahr, Eds., vol. 5774. Springer, Sep. 2009, pp. 141–152.
[15] D. Coll-Mayor, R. Picos, and E. Garciá-Moreno, “State of the art of the
virtual utility: the smart distributed generation network,” International
Journal of Energy Research, vol. 28, no. 1, pp. 65–80, 2004.
[16] Ł. B. Nikonowicz and J. Milewski, “Virtual power plants –
general review: structure, application and optimization.” Journal of
Power Technologies, vol. 92, no. 3, 2012. [Online]. Available:
http://papers.itc.pw.edu.pl/index.php/JPT/article/view/284/492
[17] S. McArthur, E. Davidson, V. Catterson, A. Dimeas, N. Hatziargyriou,
F. Ponci, and T. Funabashi, “Multi-agent systems for power engineering
applications – Part I: Concepts, approaches, and technical challenges,”
IEEE Transactions on Power Systems, vol. 22, no. 4, pp. 1743–1752,
2007.
[18] S. D. Ramchurn, P. Vytelingum, A. Rogers, and N. R. Jennings,
“Putting the ’smarts’ into the smart grid: A grand challenge for artificial
intelligence,” Commun. ACM, vol. 55, no. 4, pp. 86–97, Apr. 2012.
[Online]. Available: http://doi.acm.org/10.1145/2133806.2133825
[19] R. R. Negenborn, Z. Lukszo, and H. Hellendoorn, Eds., Intelligent In-
frastructures, ser. Intelligent Systems, Control and Automation: Science
and Engineering. Springer, 2010, vol. 42.
[20] G. Anders, F. Siefert, J.-P. Steghöfer, H. Seebach, F. Nafz, and W. Reif,
“Structuring and Controlling Distributed Power Sources by Autonomous
Virtual Power Plants,” in IEEE Power and Energy Student Summit
(PESS 2010). IEEE Power & Energy Society, 2010.
[21] J. Bremer and M. Sonnenschein, “A distributed greedy algorithm
for constraint-based scheduling of energy resources,” in FedCSIS,
M. Ganzha, L. A. Maciaszek, and M. Paprzycki, Eds., 2012, pp. 1285–
1292.
[22] ——, “Constraint-handling for optimization with support vector sur-
rogate models – a novel decoder approach,” in ICAART 2013 –
Proceedings of the 5th International Conference on Agents and Artificial
Intelligence, J. Filipe and A. Fred, Eds., vol. 2. Barcelona, Spain:
SciTePress, 2013, pp. 91–105.
[23] C. Fitz-Gibbon, Performance Indicators, ser. BERA Dialogues
Series. Multilingual Matters Limited, 1990. [Online]. Available:
http://books.google.de/books?id=uxK0MUHeiI4C
[24] C. A. Coello Coello, “Theoretical and numerical constraint-handling
techniques used with evolutionary algorithms: a survey of the state of
the art,” Computer Methods in Applied Mechanics and Engineering,
vol. 191, no. 11-12, pp. 1245–1287, Jan. 2002.
[25] S. Koziel and Z. Michalewicz, “Evolutionary algorithms, homomor-
phous mappings, and constrained parameter optimization,” Evol. Com-
put., vol. 7, pp. 19–44, 03 1999.
[26] D. M. J. Tax and R. P. W. Duin, “Support vector data description,”
Mach. Learn., vol. 54, no. 1, pp. 45–66, 2004.
[27] J. Bremer and M. Sonnenschein, “Model-based integration of con-
strained search spaces into distributed planning of active power pro-
vision.” Comput. Sci. Inf. Syst., vol. 10, no. 4, pp. 1823–1854, 2013.
[28] J. Bremer, “Ontology based description of der’s learned environmental
performance indicators,” in Proceedings of the 1st International Con-
ference on Smart Grids and Green IT Systems – SmartGreens 2012,
B. Donnellan, J. P. Lopes, J. Martins, and J. Filipe, Eds. Porto,
Portugal: SciTePress, 04 2012, pp. 107–112.
[29] V. K. Vassilev, T. C. Fogarty, and J. F. Miller, “Information character-
istics and the structure of landscapes,” Evol. Comput., vol. 8, no. 1, pp.
31–60, Mar. 2000.
[30] S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi, “Optimization by
simulated annealing,” Science, vol. 220, no. 4598, pp. 671–680, 1983.
[Online]. Available: http://www.jstor.org/stable/1690046
[31] Y. Li, V. A. Protopopescu, N. Arnold, X. Zhang, and A. Gorin,
“Hybrid parallel tempering and simulated annealing method,” Applied
Mathematics and Computation, vol. 212, no. 1, pp. 216–228, 2009.
[32] A. Müller, J. J. Schneider, and E. Schömer, “Packing a multidisperse
system of hard disks in a circular environment,” Phys. Rev.
E, vol. 79, p. 021102, Feb 2009. [Online]. Available: http:
//link.aps.org/doi/10.1103/PhysRevE.79.021102
[33] N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and
E. Teller, “Equation of state calculations by fast computing machines,”
The Journal of Chemical Physics, vol. 21, no. 6, pp. 1087–1092, 1953.
[Online]. Available: http://link.aip.org/link/?JCP/21/1087/1
[34] W. K. Hastings, “Monte carlo sampling methods using markov chains
and their applications,” Biometrika, vol. 57, no. 1, pp. 97–109,
1970. [Online]. Available: http://biomet.oxfordjournals.org/cgi/content/
abstract/57/1/97
[35] W. H. Wong and F. Liang, “Dynamic weighting in Monte Carlo
and optimization,” Applied Mathematics. Proceedings of the National
Academic of Science, vol. 94, pp. 14 220–14 224, Dec. 1997.
[36] E. Marinari and G. Parisi, “Simulated tempering: a new Monte Carlo
scheme,” Europhys. Lett., vol. 19, no. 6, 1992.
[37] S. Brown and T. Head-Gordon, “Cool walking: A new markov chain
monte carlo sampling method.” Journal of Computational Chemistry,
vol. 24, no. 1, pp. 68–76, 2003.
[38] E. Marinari and G. Parisi, “Simulated tempering: a new Monte Carlo
scheme,” Europhys. Lett., vol. 19, no. 6, 1992.
[39] C. J. Geyer and E. A. Thompson, “Annealing Markov Chain Monte
Carlo with Applications to Ancestral Inference,” Journal of the Ameri-
can Statistical Association, vol. 90, no. 431, pp. 909–920, Sep. 1995.
[40] Y. Li, M. Mascagni, and A. Gorin, “A decentralized parallel implemen-
tation for parallel tempering algorithm.” Parallel Computing, vol. 35,
no. 5, pp. 269–283, 2009.
[41] A. Schug and W. Wenzel, “All-atom folding of the trp-cage protein
with an adpative parallel tempering method,” EPL (Europhysics Letters),
vol. 67, no. 2, p. 307, 2004.
[42] B. Thomas, Mini-Blockheizkraftwerke: Grundlagen, Gerätetechnik, Be-
triebsdaten. Vogel Buchverlag, 2007.
[43] J.-H. Psola, W.-R. Canders, and M. Henke, “Modeling of a redox flow
battery storage for grid applications,” in PCIM Asia, Shanghai, China.
Stuttgart: Mesago PCIM GmbH, 2013.