Proceedings of the 20th World Congress

Proceedings
The of
of the
International
Proceedings 20th
20th World
Federation
the Worldof Congress
Automatic Control
Congress
Proceedings
The
Toulouse, of the
The International
France,
International 20th9-14,
July World
Federation
Federation of Congress
of Automatic
2017
Automatic Control
Control
The International
Available online at www.sciencedirect.com
Toulouse,
Toulouse, France,Federation
France, July
July 9-14, of Automatic Control
9-14, 2017
2017
Toulouse, France, July 9-14, 2017
ScienceDirect
IFAC PapersOnLine 50-1 (2017) 9211–9217
Network
Network Design
Design in
in the
the Presence
Presence of
of a
a Link
Link
Network
Jammer: Design
a in theGame
Zero-Sum Presence of a Link
Formulation
Jammer: a Zero-Sum Game Formulation
Jammer: a Zero-Sum
∗,∗∗
Game Formulation
∗∗
Ioannis Kordonis ∗,∗∗ George P. Papavassilopoulos ∗∗
Ioannis
Ioannis Kordonis
Kordonis ∗,∗∗ George George P. P. Papavassilopoulos
Papavassilopoulos ∗∗
∗
Ioannis Kordonis ∗,∗∗ George P. Papavassilopoulos ∗∗
∗ University of Southern California, Viterbi School of Engineering,
∗ University
∗ University
of
of Southern
3740 McClintock Southern California,
California,
Avenue, Viterbi
ViterbiCA
Los Angeles, School
School
90089, of
of Engineering,
Engineering,
(e-mail:
University
3740 of Southern California, Viterbi School of Engineering,
3740 McClintock Avenue,
Avenue, Los
McClintockjkordonis1920@yahoo.com) Los Angeles,
Angeles, CA CA 90089,
90089, (e-mail:
(e-mail:
∗∗ 3740 McClintockjkordonis1920@yahoo.com)
Avenue, Los Angeles, CA 90089, (e-mail:
jkordonis1920@yahoo.com)
∗∗ National Technical University of
jkordonis1920@yahoo.com) Athens, School of Electrical and
∗∗ National Technical
Computer
∗∗ Technical University
National Engineering, University
9 Iroon of
of Athens,
Athens, School
Polytechniou School
str., of
of Electrical
Electrical
157-80, Athens,and
and
National Engineering,
Computer Technical University 9 Iroon of Athens, School
Polytechniou str., of Electrical
157-80, Athens,and
Computer Engineering, (e-mail: 9yorgos@netmode.ntua.gr)
Iroon Polytechniou str., 157-80, Athens,
Computer Engineering, (e-mail: 9 Iroon Polytechniou str., 157-80, Athens,
(e-mail: yorgos@netmode.ntua.gr)
yorgos@netmode.ntua.gr)
(e-mail: yorgos@netmode.ntua.gr)
Abstract: This paper considers the problem of designing a Network such that a set of dynamic
Abstract:
Abstract:
rules converges This
This paper
as fastconsiders
paper considers
as possible the
the problem
problem
to the Nash of
of designing
designing aa Network
equilibrium in a such
Network classthat
such that aa set
set of
of repeated of dynamic
dynamic
games,
Abstract:
rules convergesThis paper
as considers the problem of designing a Network such that a set of dynamic
rules
despite converges
the attemptas fast
fast
of a as
as possible
possible
jammer to to
to the
slow the
down Nash
Nashthe equilibrium
equilibrium
convergence in
in
by aacutting
class
class aof repeated
ofcertain
repeated numbergames,
games, of
rules
despite converges
the as fast as possible to the Nash equilibrium in a class of repeated games,
despite
edges. the attempt
attemptwe
Particularly of a
a jammer
jammera to
of consider to
classslow
slowof down
down
quadraticthe
the convergence
convergence
games, by
by cutting
motivated cutting
by aa certain
the certain
demand number
number
response of
of
despite
edges. the attemptwe
Particularly of consider
a jammer to slow down the convergence by cutting a certain number of
edges.
problem Particularly
in electricity consider a
we markets. class
aFor
class of
of quadratic
a given networkgames,
quadratic games, motivated
a set by
motivated
structure, by the
the demand
of dynamic demand response
response
rules, based
edges.
problem Particularly
in we consider a class of quadratic games, motivated by the demand response
problem
on approximatein electricity
electricity
gradientmarkets.
decentFor
markets. For a
a given
given network
is described. network structure,
structure, aaspeed
The convergence set
set of dynamic
dynamiconrules,
ofdepends rules, based
based
the graph
problem
on in electricity
approximate gradientmarkets.
decent For
is a given network structure, aspeed set ofdependsdynamiconrules, based
on approximate
through a matrix gradient
which in decent
turn is described.
dependsdescribed.
on the The
The
graph convergence
convergence
Laplacian. speed
The dependsdesign
network on thethe graph
graph
problem
on approximate
through a matrix gradient
which in decent
turn is described.
depends on the The
graph convergence
Laplacian. speed
The dependsdesign
network on the graph
problem
through
is formulateda matrix which
as a zero sum in game
turn depends
between on the graph
a network Laplacian.
designer aiming The network the
to improve design problem
convergence
through
is a matrix
formulated as which in game
turn depends on the graph Laplacian. The network design problem
is formulated
speed and a as a
a zero
jammer zero
who sum
sum game
tries to between
between aa network
deteriorate network
it. Simple designer
designer
heuristics aiming
aiming
for theto improve
improve the
to designer the
and convergence
convergence
the jammer
is formulated
speed and a as a zero
jammer who sum game
tries to between a network
deteriorate it. Simple designer
heuristics aiming to designer
improve the convergence
speed
problems andare a jammer
proposed whoand triesa to deteriorate
numerical it. Simple
example is heuristics for the designer and the jammer
presented. for the and the jammer
speed
problems and a jammer who tries to deteriorate it. Simple heuristics for the designer and the jammer
problems are are proposed
proposed and and a a numerical
numerical exampleexample is is presented.
presented.
© 2017, IFAC
problems are (International
proposed and a numerical
Federation exampleControl)
of Automatic is presented.
Hosting by Elsevier Ltd. All rights reserved.
Keywords: Network Design, Structural Optimization, Game Theory, Jamming, Zero Sum
Keywords:
Keywords:
Game, Network
Network
Dynamic RulesDesign,
Design, Structural Optimization, Game Theory,
Structural Optimization, Game Theory, Jamming,
Jamming, Zero Zero SumSum
Keywords:
Game, Network
Dynamic RulesDesign, Structural Optimization, Game Theory, Jamming, Zero Sum
Game, Dynamic Rules
Game, Dynamic Rules
1. INTRODUCTION namics are designed such that the overall system converges
1. INTRODUCTION
1. INTRODUCTION namics
namics
as closeare are designed
as designed
possible to such
such that
thethat the overall
the
average overall systemdespite
system
consensus, converges
convergesthe
1. INTRODUCTION namics
as close are
as designed
possible such
to thethat the overall
average systemdespite
consensus, convergesthe
Often in distributed systems and games, the sub-systems as close as
attempt of possible
an opponent to the to average
deteriorate consensus,
the despite the
performance of
Often
Often in distributed
in distributed systems
systems and
and games,rules
games, the sub-systems
the sub-systems as close as
attempt of anpossible
an opponent to the to average
deteriorate consensus, despite the
the performance
performance of
or
Oftenplayers follow
in distributed some simple
systems dynamic
and games,rules in order
the sub-systems to attempt
the system. of opponent to deteriorate the of
or
or players
players followactions,
follow some
some simple
simple dynamic
dynamic rules in in order
order to the
to attempt
the system.
system. of an opponent to deteriorate the performance of
determine
or their
players their
followactions,
some simple exchanging
dynamic information
rules in order through to the
determine
determine their actions, exchanging information through In system.
this work we focus on quadratic aggregative games. In
a communication
determine graph. exchanging
their actions, The performance
exchanging
information
informationof thethrough
overall In this work we
through focus on one
quadratic aggregative games. In
a communication
asystem
communication
depends on graph.
graph. The performance
The performance
the structure of that of of the overall
the overall
network. For In In
this this work
class of we
games, focus eachon quadratic
of the aggregative
participantsgames. interactsIn
a communication graph. The performance of the overall this this work
class of we focus
games, eachon one
quadratic
of the aggregative
participants games.
interactsIn
system
system
example depends
depends
in on the
on
consensus thedynamics
structurethe
structure of convergence
of that network.
that network.speed For this
For this class
with class of games,
the aggregate each
actions one
of theof the participants
rest participants interacts
of the playersinteracts
(Jensen
system depends on thedynamics
structurethe of convergence
that network.speed with
For (2010)). the of games,
aggregate each
actions one
of theof the
rest of the players (Jensen
example
example
depends on in the
in consensus
consensus
Laplacian dynamics
of the the convergence graph.
communication speed with with
(2010)).
the There
aggregate
the There
aggregate are
actions applications
are several of the rest ofof
actions applications
several of the rest ofof
the
the
players
large
players
large
(Jensen
aggregative
(Jensen
aggregative
example
depends in the
on consensus
Laplacian dynamics
of the the convergence graph.
communication speed (2010)).games, There
such as are
the several
charging applications
of electric of large
vehicles aggregative
(Ma et al.
depends
This work onconsiders
the Laplacian
a simple of quadratic
the communication
aggregative graph.
game (2010)).
games, There
such are
as the
the several
charging applications
of andelectric of large
vehicles aggregative
(Ma et al.
al.
depends
This work onconsiders
the Laplacian
a simple of quadratic
the communication
aggregative graph.
game games,
(2010), such
Parise as et charging
al. (2014)) of electric
the vehicles
demand (Ma et
response
This work considers
and focuses a simpleofquadratic
on the problem designingaggregative
the communica- game games, (2010), such
Parise as the
et charging
al. (2014)) of andelectric
the vehicles
demand (Ma et al.
response
This
and work
focuses considers
onorder a
the problemsimple
problem quadratic
of designing
designing aggregative
the performance game
communica- (2010), (2010),
in the smart Parise et
grid al.
(Zhu (2014))
and Başar and the
(2011),demand response
Bagagiolo and
and
tion focuses
graph inon the to optimize of the system the communica- in Parisegrid
the smart
smart et (Zhual. (2014))
and Başar and (2011),
Başar the demand response
Bagagiolo and
and
tion focuses
graph inonorder
the problem
order to of designing
optimize the systemthe performance
communica- in Bausothe (2014)). grid
In (Zhu and
this class (2011),the
of examples, Bagagiolo
aggregativeand
tion
(i.e. graph
the in
convergence to optimize
speed to the the
Nash system performance
equilibrium) and in
Bausothe smart
(2014)). grid (Zhu
In thisthis and
classtheBaşar
of energy (2011),
examples, Bagagiolo
thewhich
aggregativeand
tion
(i.e. graph
the in order to
convergence optimize
speed to the
the the
Nash system performance
equilibrium) and Bauso action of (2014)). In
the players class
affects of examples, pricethe aggregative
in turn
(i.e. the convergence
its robustness speed
with respect to link Nash
jamming.equilibrium)
Specifically,and Bauso
action (2014)).
of the In this
players classthe
affects of energy
examples, pricethewhich
aggregative
in turn
turn
(i.e.
its the convergence
robustness with speed
respect to
to the
link Nash
jamming.equilibrium)
Specifically,and action
affects of
the the players
individual affects
costs. the energy price which in
its robustness
we robustness
assume that with respect
therespect to
communication link jamming. Specifically,
network Specifically,
is designed action affects of
the the players affects
individual costs. the energy price which in turn
its
we assumeand that with
thethere to
communication link jamming.
network is wants
designed affects the individual costs.
we assume
centrally that the
that communication
exists a jammer network who is designed to affects
The usetheofindividual
several of costs. dynamic rules for the participants
we assumeand
centrally that thethere
that communication
exists network is wants
designed
centrally
deteriorate and the that there exists
performance of aa jammer
the jammer
system, who
who
by wants to
cutting toa The The
of use aggregative
use
large of several
of several of ofgames
dynamic
dynamic rules
was rulesstudiedfor in
for theKizilkale
the participants
participantsand
centrally
deteriorate and that
thesmall there
performance exists a
ofedges. jammer
the system,
system, who wants
by cutting
cutting to The use aggregative
of several ofgames dynamic rules for intheKizilkale
participants
deteriorate
certain rather the performance
number of
of the This by
problem is aa of
for- of large
large
Caines aggregative
(2013), Grammaticogames was
waset studied
studied
al. (2016), in Kizilkale
Parise etand
and
al.
deteriorate
certain rather the performance
small number of the
of edges. system,
edges.the This by
problem cutting
is for- a of large (2013),
for- Caines
Caines aggregativeGrammaticogames was studied
et Paccagnan
al. (2016), in Kizilkale
Parise etand
al.
certain
mulated rather
as a small
zero sum number
game of
between This problem
network is
designer (2015), (2013),
Paccagnan Grammatico
et al. (2016a), et al. (2016), Parise
et al. et
(2016b),al.
certain rather
mulated as small number of edges.the This problem is for- Caines (2013), Grammatico et Paccagnan
al. (2016), etParise et al.
mulated
and the as aa zero
jammer. zero sum
sum game
This game between
work between
continues theournetwork
network
previous designer
designer
work (2015),(2015), Paccagnan
Koshal Paccagnan
et al. (2012). et In
et al. many
al. (2016a),
(2016a), Paccagnan
of these works,et al. dynamic
al.
the (2016b),
(2016b),
mulated
and the as a zero sum
jammer. This game
work between
continues theournetwork
previous designer
work (2015),
Koshal Paccagnan
et aal.
al.network
(2012). In et al.
In (2016a),
many of Paccagnan
oftothese
these works,theet al. (2016b),
thenecessary
dynamic
and the jammer.
in Kordonis This work continues
and Papavassilopoulos our previous work Koshal
(2016). rules use et (2012). many
structure works,
transmit the dynamic
and the jammer.
in Kordonis
Kordonis This work continues
and Papavassilopoulos
Papavassilopoulos our previous work Koshal
(2016). rules useet aal.network
(2012). In many ofto
structure these works,the
transmit thenecessary
dynamic
in and (2016). rules use
information. a network
In the structure
current work to transmit
we consider thea necessary
variant of
in
The Kordonis
graph design and Papavassilopoulos
for the fast convergence (2016).of the consensus rules use a network
information. In the structure
current work to wetransmit
consider thea necessary
variant of
The
The graph
graphand
dynamics design
design for
the for the fast
the fast
related convergence
convergence
problems optimizing of the
of the theconsensus
consensus information.
the
network information. dynamic In
rule the current
presented inwork
Koshal we consider
et al. a
(2012).variant
It turnsof
The graphand design for the fast convergence of the consensus the dynamic In the
rule currentinwork
presented Koshal we consider
etand a variant
al. (2012).
(2012). Iton
turnsof
dynamics
dynamics
coherence, and
the the
the related
related
Markov problems
problems
chain fastest optimizing
optimizing
mixing the
the
problem network
network
and the
out dynamic
that the rule overallpresented
dynamics in Koshal
is linearet al. dependIt turns
the
dynamics and the
out dynamic
that the rule
the overall
overallpresented
dynamics in Koshal
is linear et
linear and al. (2012).
andresults
dependare It turns
on also
the
coherence,
coherence, thethe
the
the minimization
relatedchain
Markov
Markov
of the
problems
chain fastest
fastest
effective
optimizing
mixing problem
mixing
resistance
the network
problem
of a graph and out
and out
that
communication
that the overall graph. dynamics
dynamics
is
Some convergence
is linear andresults
depend
dependare
on the
on also
the
coherence,
the the Markov
minimization of thechain fastestresistance
effective mixing problem of a graph communication
and communication
proved. graph.
graph. Some convergence
Some convergence results are also
the
were minimization
studied in of the effective
literature using resistance
several of a graph
techniques and communication
proved. graph. Some convergence results are also
the minimization
were studied of the effective
in(Xiao
literature using(2004),resistance
several of a graph
techniques and proved.
were
severalstudied
criteriain literature
and Boydusing several Daitechniques
and Mesbahi and
were
several studied
criteria in(Xiao
literature
and using(2004),
Boyd severalDai techniques
and Mesbahiand proved.Then the network design problem is formulated as a zero
Then the network
network design problem is formulated
formulated as aa zero
zero
several
(2011), criteria
Charalampidis(Xiao and BoydXiao
(2014), (2004),
et Dai
al. and Mesbahi
(2007), Hassan- Then
sum game the between design the networkproblem is
designer and a jammer. as We
several
(2011), criteria
Charalampidis(Xiao and Boyd
(2014), (2004),
Xiao et Dai
al. and
(2007), Mesbahi
Hassan- Then
sum the
game network
between design
the problem
network is
designer formulated
and a as
jammer. a zero
We
(2011),
Moghaddam Charalampidis
and (2014),
Jovanović Xiao
(2015), et al.
Boyd (2007),
et al. Hassan-
(2004), sum
considergame a between
criterion the network
describing designer
the and
discounted a jammer.
cumulative We
(2011), Charalampidis
Moghaddam and (2014),(2015),
Jovanović Xiao etBoyd al. (2007),
et al.al. Hassan-
(2004), considersum
considergame between
criterionthe network designer and a jammer. We
Moghaddam
Ghosh et al. and
Moghaddam and
Jovanović
(2008)). In El (2015),
Jovanović ChamieBoyd
(2015), Boyd
et
and Başar (2004),
(2016),
et al. (2016),
(2004), considerquadraticaaa distance
criterion
criterion
describing
describing
from the Nash
describing
the discounted
the
the
discounted
equilibrium.
discounted
cumulative
cumulative
At first,
cumulative
Ghosh
Ghosh
Liu andet et al. (2008)).
al. (2008)).
Başar In El Chamie
In El Chamie
(2014), Gharesifard andandand
BaşarBaşar
Başar
(2012) (2016), quadratic
and quadratic distance
distance from
from the Nash equilibrium.
equilibrium. At first,
first,
Ghosh the jammer’s problem is the Nash
considered and an approximate At
Liu
Liu andetBaşar
and
Khanafer
al. (2008)).
Başar
et al. (2014),the
(2014),
(2012)
In El Chamie
Gharesifard
Gharesifard
communication andand
and Başar
Başar
Başar
graph (2012)
(2012)
or
(2016),
the and
and
dy- quadratic
the
the jammer’s
jammer’s
distance
problem
problem
from is the
is
Nash equilibrium.
considered
considered and an
and
At first,
an approximate
approximate
Liu and Başar
Khanafer et al. (2014),the
al. (2012)
(2012) Gharesifard
communication and Başargraph(2012) the and
or the dy- the jammer’s problem is considered and an approximate
Khanafer et the communication graph or dy-
Khanafer et al. (2012) the communication graph or the dy-
Copyright
2405-8963 ©© 2017
2017, IFAC 9619Hosting by Elsevier Ltd. All rights reserved.
IFAC (International Federation of Automatic Control)
Copyright
Copyright © 2017
2017 IFAC
© under IFAC 9619
9619Control.
Peer review
Copyright © 2017 IFAC responsibility of International Federation of Automatic
9619
10.1016/j.ifacol.2017.08.1278
Proceedings of the 20th IFAC World Congress
9212
Toulouse, France, July 9-14, 2017 Ioannis Kordonis et al. / IFAC PapersOnLine 50-1 (2017) 9211–9217

algorithm is proposed. Particularly, using a set of relaxed The players exchange information through a network G =
variables, the effect of removing an edge is approximated (V, E) in which each player corresponds to a node. At
by the partial derivative of the cost with respect to the every time step k, each player is informed for the estimated
corresponding adjacency matrix variable. We then study values x̂i (k) of her neighbors. Throughout this section we
the problem of the network designer. In order to do so, assume that the graph G is connected.
we compute the approximate cost sensitivity with respect
to edge removals and then add edges which reduce this We then describe a set of dynamic rules for the participants
sensitivity the most. of the game. At each time step, the action of each player is
updated in order to reduce her cost, using an approximate
gradient decent rule:
1.1 Notation
xi (k + 1) = xi (k) − α(∂Ji /∂xi )aprox , (4)
aprox
For a matrix A ∈ Rm×n , Aij denotes the ij-th element. where (∂Ji /∂xi ) is an approximation of of ∂Ji /∂xi .
The standard basis vectors are denoted by ei , i.e. ei is a The approximation of ∂Ji /∂xi is derived substituting the
column vector having zeros in all its entries except entry i, last term of:
which has the value 1. A column vector consisting of units N
∂Ji 1
is denoted by 1, i.e. 1 = [1 1 . . . 1]T . = (2 + )xi − 2θi + 1 + xj /N, (5)
∂xi N j=1
For any pair of matrices A ∈ Rm×n and B ∈ Rq×r , the
Kronecker product A ⊗ B is defined as: by x̂i . Thus, the dynamic rule for player i is given by:
  xi (k + 1) = (1 − ᾱ)xi (k) − αx̂i (k) + α(2θi − 1) (6)
A11 B . . . A1n B
 ..  . xi (0) = θi ,
A ⊗ B =  ... ..
. .  (1)
where ᾱ = α(2 + 1/N ).
Am1 B . . . Amn B
The vectorization of a matrix A = [A1 |A2 | . . . |Am ], where At every time step, the players update their estimates
Aj is the j-th column of A, is denoted by vec(A) and according to:

is given by [AT1 AT2 . . . ATm ]T . The identity (ex. Horn and x̂i (k + 1) = (1 − di δ)x̂i (k) + δ x̂j (k)+
Johnson): j∈Ni
T + (−ᾱxi (k) − αx̂i (k) + α(2θi − 1))
vec(AXB) = (B ⊗ A)vec(X), (2)
will be used. (7)
x̂i (0) = θi ,
We denote by G = (V, E) an undirected graph, where
V = (v1 , . . . , vN ) is the set of vertices and E the set of where Ni is the set of neighbors of player i and di is the
edges. The adjacency matrix is denoted by A, i.e. Aij = 1 degree (the number of neighbors) of the node of player
if there exists an edge between vertices i and j and 0 i. The first two terms (7) correspond to the consensus
otherwise. dynamics and the last to the fact that xi is actually
changing.
We denote by di the degree of the node i, i.e. the number Remark 1. The dynamic rule proposed, closely parallels
of edges adjacent to vertex i. The Laplacian of the graph the rule described in Koshal et al. (2012). The basic
is given by L = ∆ − A where ∆ = diag(d1 , . . . , dN ). differences are that Koshal et al. (2012) considers a time
varying stochastic pairwise information exchange and a
2. GAME DESCRIPTION AND DYNAMIC RULE Stochastic Approximation type decreasing step-size (αk →
0) is used. Thus, the convergence rate may be slow. On the
Let us describe a class of quadratic games, motivated other hand the current work analyzes a simpler problem,
by the demand response problem. The set of players is in which there is a synchronous communication with all
i = 1, . . . , N and the cost of each player is given by: the neighbours. Hence, a constant step size may be used
 
N and the dynamic rule converges exponentially to the Nash

Ji = (xi − θi )2 + 1 + xj /N  xi , (3) equilibrium.
j=1
The dynamics can be written in compact form as:
where N is the number of energy consumers, xi is the x̃(k + 1) = P x̃(k) + BΘ (8)
amount of energy consumed by player i, θi is the desired  
N Θ
consumption of player i and 1 + j=1 xj /N is the energy x̃(0) =
Θ
cost per unit (a more general price function of the form
N where x = [x1 . . . xN ]T , x̂ = [x̂1 . . . x̂N ]T , x̃ = [xT x̂T ]T ,
c1 +c2 j=1 xj /N could be used, but for simplicity reasons
we use c1 = c2 = 1). The first term of (3) represent Θ = [2θ1 − 1 . . . 2θN − 1]T , (9)
the disutility of player i for having a level of energy and
consumption different from the desired and the second   
term is the total amount of money that she pays. αI (1 − ᾱ)I −αI
B= , P = . (10)
αI −ᾱI (1 − α)I − δL
The game is played repeatedly over time and each player
holds an estimation of the mean value of the actions of the The dynamics of the overall system depends on the matrix
players. Let us denote by x̂i (k) the estimation of player i P , which in turn depends on the Laplacian matrix. Some
for the mean value of the action of the players. properties of matrix P are shown in the following lemma.

9620
Proceedings of the 20th IFAC World Congress

Toulouse, France, July 9-14, 2017 Ioannis Kordonis et al. / IFAC PapersOnLine 50-1 (2017) 9211–9217 9213

Lemma 1. Assuming that G is connected, the dynamics such that the dynamic rules of the players converge as
(8) have the following properties: quickly as possible to the Nash equilibrium, despite the
fact that an opponent may destroy a number of links. The
(i) The matrix P has an eigenvalue 1 with left eigenvec- link cuts may be interpreted either as the actions of an
tor [1T − 1T ] and a unique right eigenvector actual opponent or as link failures. In the latter case the
 
1 problem considered corresponds to the worst case analysis.
d=
−(2 + 1/N )1
We assume that the planner does not know the types Θ
(ii) The subspace: of the players. Furthermore, the same topology is designed
C = {x̃ : [1T − 1T ]x̃ = 0}, (11) for many repetitions of the game. Thus, a stochastic model
for the types of the players is used. Based on the stochastic
is invariant under (8)
description a discounted quadratic criterion is introduced
(iii) If (8) converges to a fixed point:
 N and the network design problem is described as a zero sum
N x game.
x̃ = N ,
x̂
3.1 A Discounted Quadratic Criterion
then xN is the unique Nash equilibrium of the game
and x̂N = (1T xN /N )1. We assume that the type of each player is given by:
Proof : For a similar result see Kordonis and Papavas- θ i = µi + w i , (17)
silopoulos (2016).  where µi is the mean of player i’s type and wi a zero
mean random vector. Denote by θ the vector of types.
The Nash equilibrium of the game is given by: We further assume that the vector of means µ and the
 
N N 1 T covariance matrix Σ = E[wwT ] of θ are known to the
x = I− 11 Θ = D1 Θ (12)
2N + 1 3N + 1 planner.
and the vector x̂N by: The network design problem has two objectives. At first
1 the matrix P̃ should should have spectral radius less
x̂N = 11T Θ = D2 Θ. (13) than or equal to 1, such that the players actions remain
3N + 1
Denote by D the matrix [D1T D2T ]T . bounded. The second objective is captured by a discounted
quadratic criterion, quantifying the speed of convergence:
Lemma 1 shows that both span{d} and C are P -invariant. ∞ N

 
Furthermore, R2N = C + span{d}. The dynamics (8) d
J =E ρ k i i,N 2
(x (k) − x ) . (18)
evolve in C. Thus, an equivalent description of (8) in C k=0 i=1
could be obtained using a matrix P̃ such that P̃ x̃ = P x̃ Remark 2. The jammer may cut a certain number of edges
if x̃ ∈ C and P̃ d = 0. The following lemma shows this resulting to a disconnected network. In this case we may
possibility. not expect convergence to the Nash equilibrium and we
Lemma 2. Consider the matrix P̃ = P (I + M ) where M will have a steady state error. An undiscounted infinite
is given by: horizon cost would lead to an infinite cost. The reason we
  choose a discounted criterion is to take into account both
1 −11T 11T the transient and the steady state error. An alternative
M= 1 1 . (14)
3N + 1 (2 + )11T −(2 + )11T representation is that the dynamic rule will stop after a
N N random number of steps (see for example Boukas et al.
(i) It holds, P̃ x̃ = P x̃ if x̃ ∈ C and P̃ d = 0. (1990)).
(ii) Dynamics (8) has the same trajectories with:
The criterion (18) can be written as:
x̃(k + 1) = P̃ x̃(k) + BΘ, (15)  ∞  

d T k k T k
under the initial conditions described. J = E y (0) ρ (P̃ ) QP̃ y(0) , (19)
k=0
Proof : Immediate  where: 

I 0
We may observe that the matrix P̃ has the same eigen- Q= . (20)
0 0
structure with P except for the eigenpair (1, d) which in
P̃ becomes (0, d). √ ∞
If ρP̃ is stable then the matrix X = k=0 ρk (P̃ T )k QP̃ k
The network will be designed such that the matrix P̃ is is the unique solution of the Lyapunov equation:
stable. Under this assumption, the dynamics (8) converges ρP̃ T X P̃ − X + Q = 0, (21)
to the vector x̃N corresponding to the Nash equilibrium. which can be expressed in terms of the Kronecker product
The distance form equilibrium y(k) = x̃(k) − x̃N evolves as:  
according to: fQ (P̃ , vec(X)) = I − ρP̃ T ⊗ P̃ T vec(X) − vec(Q) = 0
y(k + 1) = P̃ y(k). (16)
(22)
3. THE NETWORK DESIGN PROBLEM The cost can be written as:
2N 
 2N
 
The network transmitting the information is designed Jd = Xij E y i (0)y j (0) = vec(S)T vec(X), (23)
centrally and the aim of the planner is to design a network i=1 j=1

9621
Proceedings of the 20th IFAC World Congress
9214
Toulouse, France, July 9-14, 2017 Ioannis Kordonis et al. / IFAC PapersOnLine 50-1 (2017) 9211–9217

where S = E[y(0)y(0)T ]. In order to compute S recall edges is proposed. In order to do so, the binary variables
that: Aij are temporarily assumed to have continuous values
    
I I in [0, 1] (relaxation) and the derivative of the cost with
y(0) = θ − D(2θ − 1) = − 2D θ + D1. (24) respect to Aij is considered. We assume that Aij with
I I
j < i, corresponding to existing edges are the free variables
T and that Aji = Aij . For a given matrix A the effect of
Denoting by D̄ the matrix [I I] − 2D we get:
removing link (i, j) to the cost is approximated by the
S = D̄(µµT + Σ)D̄T + D̄µ1T DT + D1µT D̄T + D11T DT derivative ∂J d (P̃ (A), X(P̃ (A)))/∂Ai,j .
(25)
This partial derivative can be computed using the chain
3.2 The Network Design Problem rule and implicit function theorem:
 −1
∂J d ∂fQ ∂fQ ∂vec(P̃ )
Let us then state the network design problem. Assume that = −vec(S)T
∂Ai,j ∂vec(X) ∂vec(P̃ ) ∂Aij
there is a graph G0 = (V, E0 ) describing the existing links (27)
among the N players. Then, the designer should choose the
place where a set of additional links Enew = {l1 , . . . , lmax } The first two terms satisfy:
will be introduced to the graph in order to minimize J d ,  −1
T ∂fQ
knowing that possibly a set of at most m edges will be (vec(S)) =
removed. The jammer, knowing the set of edges E = E0 ∪ ∂vec(X)
 −1 T
Enew , chooses a subset of Ej ⊂ E ∩ Evul to cut, with
|Ej | = m in order to maximize Jd . The set Evul represents = I − ρP̃ ⊗ P̃ vec(S) = vec(Y )T (28)
all the links which the jammer has the ability to cut. The
where Y satisfies the Lyapunov equation:
designer’s problem is:
ρ(P̃ T )T Y (P̃ T ) − Y = S, (29)
minimize max {vec(S)T vec(X)}
Enew Ej ⊂E∪Evul ,|Ej |≤m T
or fS (P̃ , vec(Y )) = 0.
subject to P̃ = P̃ (A) For the third term it holds,
 
I − ρP̃ T ⊗ P̃ T vec(X) = vec(Q) ∂fQ  
= −ρ ej  eTi ⊗ P̃ T + P̃ T ⊗ ej  eTi vec(X). (30)
P̃ : has spectral radius less than or equal to 1 ∂ P̃i j 
∂fQ
(26) Denote by f¯i j  (P̃ , X) the value of . Furthermore, it
∂ P̃i j 
where P̃ = P̃ (A) is given by P̃ = P (I + M ), (10) and (14). holds:
 
Remark 3. The solution of (26) represents the security ∂ P̃ ∂P 0  0
level (ex. Basar and Olsder (1999)) of the network designer = =  .
∂Aij ∂Aij 0 δ ei eTj + ej eTi − ej eTj − ei eTi
in the zero sum game where the network designer has cost
(31)
J d and the jammer −J d .
∂ P̃ ∂P
The equality, = holds true due to the fact that
It should be pointed out that if (26) is to be considered ∂Aij ∂Aij
∂P
as a zero sum game and a zero sum solution is sought, ∂Aij M = 0.
then several issues may arise. First, it can actually be seen
equivalently as a matrix a game for which we seek a pure Thus, the last two terms of (27) satisfy:
strategy solution, which may not exist. Then we could seek ∂fQ ∂vec(P̃ ) 
for a mixed solution via the known Linear Programming = δ f¯i+N,j+N (P̃ , X) +
∂vec(P̃ ) ∂Aij
formulation. The dimensions are quite formidable and thus 
we can resort to other algorithms such as Simulated An- +f¯j+N,i+N (P̃ , X) − f¯i+N,i+N (P̃ , X) + f¯j+N,j+N (P̃ , X)
nealing and its variants. Clearly such random search meth- (32)
ods can be employed also for the security level formulation
we chose to consider here since even for this formulation A simple technique for the jammer is described in Algo-
the dimensions are quite large. We will pursue this line in rithm 1
future work.
Algorithm 1
The optimization problem (26) is a nonlinear mixed integer 1 : Get the designed graph G = (V, E).
programming problem (Wolsey and Nemhauser (2014)). 2 : Set lcnt ← 0 and Ej ← ∅.
Thus, in general it could be difficult to solve. In the
3 : Compute the matrices P̃ and X.
following simple heuristics for the designer’s and jammer’s
4 : For every edge (i, j) ∈ E compute ∂J d /∂Aij and choose
problems are suggested.
(i , j  ) which minimizes ∂J d /∂Aij .
5 : Set E ← E \ (i , j  ) and lcnt ← lcnt + 1
4. HEURISTICS FOR THE OPTIMIZATION
5 : If lcnt < m go to Step 3. Else halt.
PROBLEM (26)

4.1 The Jammer’s Problem 4.2 The designer’s problem

Let us start with the jammer’s problem. A simple algo- We then propose a simple heuristic for the designer’s
rithm based on the approximate influence of removing problem. The technique is based on the idea to reduce

9622
Proceedings of the 20th IFAC World Congress

Toulouse, France, July 9-14, 2017 Ioannis Kordonis et al. / IFAC PapersOnLine 50-1 (2017) 9211–9217 9215

the sensitivity of the designed graph with respect to link Algorithm 2

removals. Particularly, the designer adds lmax − m edges
1 : Get the designed graph G = (V, E0 ).
to reduce the cost J d and the rest m of the edges to reduce
2 : Set lcnt ← 0 and Enew ← ∅.
the sensitivity of the designed graph in edge removal.
3 : Compute the matrices P̃ and X.
The sensitivity of a given graph may be approximated by 4 : For every edge (i, j) ∈ E0 ∪ Enew compute ∂J d /∂Aij
max(i,j)∈E {∂J d /∂Aij }. Fix the edge (i , j  ) which attains and choose (i , j  ) which minimizes ∂J d /∂Aij .
the maximum. The dependence of the sensitivity on the 5 If lcnt > lmax − m go to Step 7.
addition of a new edge (in , jn ) may be approximated by 6 : Set E ← E ∪ (i , j  ) and lcnt ← lcnt + 1. Go to Step 9.
∂ [∂J d /∂Ai j  ] 7 : For every edge (in , jn ) ∈ E0 ∪ Enew compute
∂Ai j .
n n ∂ [∂J d /∂Ai j  ]
∂Ain jn and choose (inew , j new ) which minimizes the
It holds:
  partial derivative.
∂ ∂J d /∂Ai j  ∂vec(Y )T ∂fQ ∂vec(P̃ ) 8 : Set E ← E ∪ (inew , j new ) and lcnt ← lcnt + 1.
=− +
∂Ain ,jn ∂Ain jn ∂vec(P̃ ) ∂Ai j  9 : If lcnt < m go to Step 3. Else halt.
 
T ∂ ∂fQ ∂vec(P̃ )
+vec(Y ) (33)
∂Ain jn ∂vec(P̃ ) ∂Ai j 

Let us first compute the first term of (33). The vector

Y satisfies the Lyapunov equation (29). If P̃ is stable,
implicit function theorem implies that there exists locally
a function gS (P̃ T ), such that fS (P̃ T , gS (P̃ T )) = 0 and Fig. 1. The graph of Example 1
that:
 −1
∂vec(Y ) ∂fS ∂fS ∂vec(P̃ T )
= . (34)
∂Ain jn ∂vec(Y ) ∂vec(P̃ T ) ∂Ain jn
Furthermore,
∂fS  
= −ρ ej  eTi ⊗ P̃ + P̃ ⊗ ej  eTi vec(Y ), (35)
∂ P˜T j  i
∂vec(P̃ T )
and ∂Ain jn was already computed in (31).
Let us then compute the second term of (33). Using (32),
∂ f¯i j  (P̃ ,X)
it remains to compute ∂Ain jn . It holds:
Fig. 2. The cost J d for the graph in Example 1. The
∂ f¯i j  (P̃ , X)  ∂ f¯i j  ∂ P̃ij horizontal axis is the index of the link removed by
= (36)
∂Ain jn i,j
∂ P̃ij ∂Ain jn the jammer and the vertical axis is the scaled cost.
where That is, each player has a type correlated only with his
∂ f¯i j    immediate neighbors. Figure 2 illustrates the value of the
= −ρ ej  eTi ⊗ ej eTi + ej eTi ⊗ ej  eTi vec(X)+ criterion J d when the edge among nodes (i, i + 1) has been
∂ P̃ij
  removed, for various values of i and p.
+ρ2 ej  eTi ⊗ P̃ T + P̃ T ⊗ ej  eTi (I − ρP̃ T ⊗ P̃ T )−1 ·
  We may observe that the effects of an edge removal depend
· ej eTi ⊗ P̃ T + P̃ T ⊗ ej eTi vec(X) essentially on the stochastic characteristics of the players’
types.
(37) Example 2. In this example, the initial graph is shown in
∂ [∂J d /∂Ai j  ]
Figure 3. The means and covariance matrix of the players’
Combining (32)-(37) ∂Ain jn is computed. types is as in Example 1 with p = 0.1. We assume that
both the network designer can design 1 additional link
A simple technique for the network designer is described and the jammer could cut 1 link. Applying Algorithm 1,
in Algorithm 2. the jammer’s choice would be link (3, 7). Using Algorithm
2, the network designer will add the edge (2, 7).
5. NUMERICAL EXAMPLES
The heuristics examined do not necessarily give the opti-
Example 1. In this example we study how the designer’s mal solution and in the future we plan to use or develop
cost varies when an edge is removed. We consider a very alternative techniques for the problem (26).
simple linear graph with 20 nodes (Figure 1). The mean
of the type of each player µi is 0.5. The matrix covariance 6. CONCLUSION
of the types is given by:
  We considered the problem of designing an information
0.5 p 0 . . . 0
  exchange network such that a set gradient based rules
Σ =  p 0.5 p . . . 0 . (38) to converge to the Nash equilibrium as fast as possible,
.. .. .. . . ..
. . . . . despite the presence of a link jammer. The underlying

9623
Proceedings of the 20th IFAC World Congress
9216
Toulouse, France, July 9-14, 2017 Ioannis Kordonis et al. / IFAC PapersOnLine 50-1 (2017) 9211–9217
Fig. 3. The graph of Example 2.
game is motivated by the demand response problem. We
first characterized the convergence to the Nash equilibrium
using the eigenstructure of a matrix depending on the
graph Laplacian and then stated a discounted infinite
horizon criterion capturing both the transient and steady
state errors. The network design problem was formulated
as an integer valued zero sum game among the designer
and the jammer, for which a security level solution, i.e.
minimax was sought. Some simple heuristic rules for both
players were proposed.
There are several possible directions for future research.
A very interesting one is to study alternative methods to
solve the optimization problems. Due to the high dimen-
sionality, good candidates would be simulated annealing
and other organized random searches. Another direction is
to generalize the underlying game structure to games in-
volving players with different influence, multidimensional
or constrained actions and non-quadratic cost functions.
REFERENCES
Bagagiolo, F. and Bauso, D. (2014). Mean-field games and
dynamic demand management in power grids. Dynamic
Games and Applications, 4(2), 155–176.
Basar, T. and Olsder, G.J. (1999). Dynamic noncoopera-
tive game theory, volume 23. Siam.
Boukas, E.K., Haurie, A., and Michel, P. (1990). An
optimal control problem with a random stopping time.
Journal of optimization theory and applications, 64(3),
471–480.
Boyd, S., Diaconis, P., and Xiao, L. (2004). Fastest mixing
markov chain on a graph. SIAM review, 46(4), 667–689.
Charalampidis, A.C. (2014). Graph topology optimization
for fast consensus based on the influence of new links to
a quadratic criterion. In Decision and Control (CDC),
2014 IEEE 53rd Annual Conference on, 1452–1457.
IEEE.
Dai, R. and Mesbahi, M. (2011). Optimal topology design
for dynamic networks. In Decision and Control and
European Control Conference (CDC-ECC), 2011 50th
IEEE Conference on, 1280–1285. IEEE.
El Chamie, M. and Başar, T. (2016). A zero-sum game
between the network designer and an adversary in
9624
consensus protocols. In Advances in Dynamic and
Evolutionary Games, 117–137. Springer.
Gharesifard, B. and Başar, T. (2012). Resilience in
consensus dynamics via competitive interconnections.
IFAC Proceedings Volumes, 45(26), 234–239.
Ghosh, A., Boyd, S., and Saberi, A. (2008). Minimizing
effective resistance of a graph. SIAM review, 50(1), 37–
66.
Grammatico, S., Parise, F., Colombino, M., and Lygeros,
J. (2016). Decentralized convergence to nash equilibria
in constrained deterministic mean field control. IEEE
Transactions on Automatic Control, PP(99), 1–1. doi:
10.1109/TAC.2015.2513368.
Hassan-Moghaddam, S. and Jovanović, M.R. (2015). An
interior-point method for topology identification and
optimal design of noisy networks. arXiv preprint
arXiv:1506.03437.
Horn, R.A. and Johnson, C.R. (????). Topics in matrix
analysis, 1991. Cambridge University Presss, Cam-
bridge, 37, 39.
Jensen, M.K. (2010). Aggregative games and best-reply
potentials. Economic theory, 43(1), 45–66.
Khanafer, A., Touri, B., and Başar, T. (2012). Consensus
in the presence of an adversary. IFAC Proceedings
Volumes, 45(26), 276–281.
Kizilkale, A.C. and Caines, P.E. (2013). Mean field
stochastic adaptive control. Automatic Control, IEEE
Transactions on, 58(4), 905–920.
Kordonis, I. and Papavassilopoulos, G. (2016). Network
design for fast convergence to the nash equilibrium in
a class of repeated games. In 2016 24th Mediterranean
Conference on Control and Automation (MED), 1026–
1032. IEEE.
Koshal, J., Nedic, A., and Shanbhag, U.V. (2012). A gossip
algorithm for aggregative games on graphs. In Decision
and Control (CDC), 2012 IEEE 51st Annual Conference
on, 4840–4845. IEEE.
Liu, J. and Başar, T. (2014). Toward optimal network
topology design for fast and secure distributed compu-
tation. In International Conference on Decision and
Game Theory for Security, 234–245. Springer.
Ma, Z., Callaway, D., and Hiskens, I. (2010). Decentralized
charging control for large populations of plug-in electric
vehicles. In Decision and Control (CDC), 2010 49th
IEEE Conference on, 206–212. IEEE.
Paccagnan, D., Gentile, B., Parise, F., Kamgarpour, M.,
and Lygeros, J. (2016a). Distributed computation of
generalized Nash equilibria in quadratic aggregative
games with affine coupling constraints. In IEEE Con-
ference on Decision and Control.
Paccagnan, D., Kamgarpour, M., and Lygeros, J. (2016b).
On aggregative and mean field games with applications
to electricity markets. In European Control Conference,
1–6.
Parise, F., Colombino, M., Grammatico, S., and Lygeros,
J. (2014). Mean field constrained charging policy for
large populations of plug-in electric vehicles. In Decision
and Control (CDC), 2014 IEEE 53rd Annual Confer-
ence on, 5101–5106. IEEE.
Parise, F., Grammatico, S., Gentile, B., and Lygeros, J.
(2015). Network aggregative games and distributed
mean field control via consensus theory. arXiv preprint
arXiv:1506.07719.
Proceedings of the 20th IFAC World Congress

Toulouse, France, July 9-14, 2017 Ioannis Kordonis et al. / IFAC PapersOnLine 50-1 (2017) 9211–9217 9217

Wolsey, L.A. and Nemhauser, G.L. (2014). Integer and

combinatorial optimization. John Wiley & Sons.
Xiao, L. and Boyd, S. (2004). Fast linear iterations for
distributed averaging. Systems & Control Letters, 53(1),
65–78.
Xiao, L., Boyd, S., and Kim, S.J. (2007). Distributed aver-
age consensus with least-mean-square deviation. Jour-
nal of Parallel and Distributed Computing, 67(1), 33–46.
Zhu, Q. and Başar, T. (2011). A multi-resolution large
population game framework for smart grid demand
response management. In Network Games, Control
and Optimization (NetGCooP), 2011 5th International
Conference on, 1–8. IEEE.

9625