Static & Dynamic Game Theory:

Foundations & Applications

Series Editor
Tamer Başar, University of Illinois, Urbana-Champaign, IL, USA

Editorial Advisory Board

Daron Acemoglu, MIT, Cambridge, MA, USA
Pierre Bernhard, INRIA, Sophia-Antipolis, France
Maurizio Falcone, Università degli Studi di Roma “La Sapienza”, Rome, Italy
Alexander Kurzhanski, University of California, Berkeley, CA, USA
Ariel Rubinstein, Tel Aviv University, Ramat Aviv, Israel; New York University,
New York, NY, USA
William H. Sandholm, University of Wisconsin, Madison, WI, USA
Yoav Shoham, Stanford University, Stanford, CA, USA
Georges Zaccour, GERAD, HEC Montréal, Canada

For further volumes:

www.springer.com/series/10200
Lacra Pavel

Game Theory for

Control of Optical
Networks
Lacra Pavel
Department of Electrical and
Computer Engineering
University of Toronto
Toronto, ON
Canada

ISBN 978-0-8176-8321-4 ISBN 978-0-8176-8322-1 (eBook)

DOI 10.1007/978-0-8176-8322-1
Springer New York Heidelberg Dordrecht London

Library of Congress Control Number: 2012938417

Mathematics Subject Classification: 91-02, 93-02, 94-02

© Springer Science+Business Media New York 2012

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of
the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology
now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection
with reviews or scholarly analysis or material supplied specifically for the purpose of being entered
and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of
this publication or parts thereof is permitted only under the provisions of the Copyright Law of the
Publisher’s location, in its current version, and permission for use must always be obtained from Springer.
Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations
are liable to prosecution under the respective Copyright Law.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
While the advice and information in this book are believed to be true and accurate at the date of pub-
lication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any
errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect
to the material contained herein.

Printed on acid-free paper

Springer is part of Springer Science+Business Media (www.birkhauser-science.com)

Preface

Game theory has recently been enjoying increased popularity in the research com-
munity, as it provides a new perspective on optimization, networking, and dis-
tributed control problems. It incorporates paradigms such as Nash equilibrium and
incentive compatibility, it can help quantify individual preferences of decision-
makers, and it has an inherently distributed nature. Consequently, game theoretic
models have been applied to a variety of problem domains ranging from economics
to computer science, networking, and security. Game theoretic models provide not
only a basis for analysis but also for design of network protocols and decentralized
control schemes. This makes it attractive as a theoretical framework for the design
of networks and in particular communication networks. Applications range from
power control in wireless communications to congestion control for the Internet, as
well as sensor or ad-hoc networks.
As the backbone of the Internet, optical networks provide huge bandwidth and in-
terconnect countries and continents. Unlike conventional networks, which are well-
established and even standardized, the optical networks field is much younger, and
in fact still evolving. Designing and setting up networking applications within the
optical context is inherently more difficult than in conventional wired or wireless
networks, in part due to the more complex physical-layer effects, and in part due to
the lack of automatic methodologies developed for optical networks. But in these
respects, the optical networking field has seen a tremendous growth in recent years.
From a game theory and control perspective, there are a multitude of problems to
be tackled in the optical networks area and the field is still in its infancy. Aside from
the fact that the area is much younger, research work requires an unusual blend of
interdisciplinary expertise. In this spirit, the present monograph draws the author’s
research background in control theory, practical industrial experience in optical net-
works, and more than five years of research in game theory and control in optical
communications.
The book is focused on mathematical models, methodologies, and game theory
for optical networks from a control perspective. The general setup is that, given an
optical communication network and some performance measures to be optimized
among many networks units/players/channels, one must design an algorithm that

v
vi Preface

achieves as good a performance as possible for each channel. The algorithm should
be decentralized and provably convergent, and should use localized and decentral-
ized feedback. By regarding channels in the network as players in a game this multi-
objective optimization problem fits within the setup of game theory.
The first theoretical question of interest is how to formulate meaningful and
tractable game theoretical problems that can be used as a basis for developing such
algorithms, taking into account the various mathematical characteristics arising nat-
urally in optical networks. These mathematical characteristics are imposed by phys-
ical constraints and by network specific features and topologies. They translate into
constraints on the game formulation (separable versus coupled action space), con-
straints on player’s interaction (localized versus global interaction, i.e., one shot
game versus stage or partitioned game), and constraints on players’ actions (global
constraints versus propagated/modified constraints).
Thus, one of the characteristics of optical networks is that game theoretic for-
mulations and results cannot be transferred directly from other application domains
(e.g., wireless networks or congestion control). Due to inherent physical complex-
ities, the optical area uncovers new theoretical problems, such as games with cou-
pled utilities and coupled constraints. Moreover, in the networking setup these con-
straints are modified across the network links and a framework for dealing with
network games has to be developed. This makes game theory in optical networks
an excellent starting point that could open new research problems and that could be
generalized to other classes of games in networks.
This monograph has a two-fold aim. Its first goal is to provide researchers in
the control and game theoretic community with background on the rich problems
and the initial results in this area. There is a broad scope for fundamental control
and game theoretical research; the hope is that the book will provide background
material such that non-specialists in optical networks can approach these research
problems within the optical networking domain. The book’s second goal is to pro-
vide researchers in the networking and optical community with game theoretical
methodologies that could be used to solve optical networking problems.
The following topics are covered. In the first part of the book, two chapters
present non-cooperative game theory background and some new mathematical re-
sults for Nash equilibria computation in games with coupled constraints. In the sec-
ond part, background and mathematical models for optical networks are presented,
followed by game theory formulations developed for various topologies. The basic
game considered is a power control game in the class of games with continuous
action spaces, coupled utilities in normal form. The book in fact introduces in a sys-
tematic, gradual way the different types of game theoretical problems: first games
with no coupled constraints in normal form, then games with coupled constraints;
all-to-all interaction versus localized player interaction, which leads to games in
ladder-nested form; multi-link (single-sink) and finally mesh topologies and how to
deal with games in such scenarios, while building on the simpler cases and results.
The third part considers issues such as robustness and time-delay effects, as well
as other types of game problems in optical networks, including routing and path
coloring games.
Preface vii

Much of the work in game theory applications has been within the framework of
networking or computer science fields. Essentially, this book uses a mathematical
approach developed from a control theoretical perspective. This is different from
a networking approach, which is typically application-specific and focused on par-
ticular protocols, and unlike algorithmic-only approaches as typically used in com-
puter science. Game theoretical problems are mathematically formulated herein in
a systemic manner, with analytic conditions derived for existence and uniqueness of
Nash equilibrium. Based on these, iterative algorithms with provable convergence
properties to Nash equilibria are developed. The control theoretic approach to games
as taken here allows for the treatment of important issues such as stability and time-
delay effects in a dynamic system context. The abstract mathematical models and
results could be applied to other application domains.
I wish to acknowledge and express my thanks to the many people who influ-
enced and shaped my work over the past several years and thus indirectly helped me
in writing this book: Tamer Başar, Tansu Alpcan, Peter Caines, Roland Malhamé,
and Eitan Altman from the game theoretical and control community; and Stewart
Aitchison, Li Qian, and Dan Kilper from the optical communications community.
I am indebted to my graduate students, Yan Pan, Nem Stefanovic, and Quanyan
Zhu, with whom many of the research work was done and without whom this book
would not have been possible.
Toronto Lacra Pavel
Contents

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Game Theory in Networks . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Optical Networks . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Scope of This Monograph . . . . . . . . . . . . . . . . . . . . . . 5

Part I Game Theory Essentials

2 Basics of Game Theory . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.2 Game Formulations . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.3 Games in Extensive Form . . . . . . . . . . . . . . . . . . . . . . 13
2.4 Games in Normal Form . . . . . . . . . . . . . . . . . . . . . . . 16
2.5 Game Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.5.1 Strategy Space: Matrix vs. Continuous Games . . . . . . . 17
2.5.2 Mixed vs. Pure Strategy Games . . . . . . . . . . . . . . . 18
2.5.3 Competitive Versus Cooperative . . . . . . . . . . . . . . . 18
2.5.4 Repetition . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.5.5 Knowledge Information . . . . . . . . . . . . . . . . . . . 21
2.6 Solution Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.6.1 Minimax Solution . . . . . . . . . . . . . . . . . . . . . . 21
2.6.2 Best Response . . . . . . . . . . . . . . . . . . . . . . . . 22
2.6.3 Nash Equilibrium Solution . . . . . . . . . . . . . . . . . 23
2.6.4 Pareto Optimality . . . . . . . . . . . . . . . . . . . . . . 24
2.7 The Rationality Assumption . . . . . . . . . . . . . . . . . . . . . 25
2.8 Learning in Classical Games . . . . . . . . . . . . . . . . . . . . 25
2.9 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3 Matrix Games . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.2 Bimatrix Games . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.2.1 Mixed Strategies . . . . . . . . . . . . . . . . . . . . . . . 30
3.3 m-Player Games . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

ix
x Contents

3.3.1 Pure and Mixed Strategies . . . . . . . . . . . . . . . . . . 32

3.3.2 Mixed-Strategy Cost Functions . . . . . . . . . . . . . . . 33
3.4 Dominance and Best Replies . . . . . . . . . . . . . . . . . . . . 35
3.4.1 Best-Response Correspondence . . . . . . . . . . . . . . . 36
3.5 Nash Equilibria Theorem . . . . . . . . . . . . . . . . . . . . . . 36
3.6 Nash Equilibria Refinements . . . . . . . . . . . . . . . . . . . . 41
3.6.1 Perfect Equilibrium . . . . . . . . . . . . . . . . . . . . . 41
3.6.2 Proper Equilibrium . . . . . . . . . . . . . . . . . . . . . 42
3.6.3 Strategically Stable Equilibrium . . . . . . . . . . . . . . . 43
3.7 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
4 Games with Continuous Action Spaces . . . . . . . . . . . . . . . . . 45
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
4.2 Game Formulations . . . . . . . . . . . . . . . . . . . . . . . . . 45
4.3 Extension to Mixed Strategies . . . . . . . . . . . . . . . . . . . . 46
4.4 Nash Equilibria and Reaction Curves . . . . . . . . . . . . . . . . 47
4.5 Existence and Uniqueness Results . . . . . . . . . . . . . . . . . . 49
4.6 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
5 Computational Results for Games with Coupled Constraints . . . . 55
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
5.2 Nash Equilibria and Relaxation via an Augmented Optimization . . 57
5.3 Lagrangian Extension in a Game Setup . . . . . . . . . . . . . . . 60
5.4 Duality Extension . . . . . . . . . . . . . . . . . . . . . . . . . . 62
5.5 Hierarchical Decomposition in a Game Setup . . . . . . . . . . . . 64
5.6 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

Part II Game Theory in Optical Networks

6 Optical Networks: Background and Modeling . . . . . . . . . . . . . 71
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
6.2 Transmission Basics . . . . . . . . . . . . . . . . . . . . . . . . . 74
6.3 Topologies and Setup . . . . . . . . . . . . . . . . . . . . . . . . 75
6.4 Power Control and OSNR Model . . . . . . . . . . . . . . . . . . 78
6.4.1 Point-to-Point Link Model . . . . . . . . . . . . . . . . . . 78
6.4.2 Network Model . . . . . . . . . . . . . . . . . . . . . . . 86
6.4.3 Link Capacity Constraint . . . . . . . . . . . . . . . . . . 93
6.5 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
7 Games in Point-to-Point Topologies . . . . . . . . . . . . . . . . . . . 97
7.1 Game Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . 97
7.2 Games Without Coupled Constraints . . . . . . . . . . . . . . . . 98
7.2.1 Utility and Nash Equilibrium Solution . . . . . . . . . . . 98
7.2.2 Iterative Algorithm . . . . . . . . . . . . . . . . . . . . . 104
7.3 Games with Coupled Constraints: Indirect Penalty Approach . . . 106
7.3.1 Nash Equilibrium Solution . . . . . . . . . . . . . . . . . 107
7.3.2 Iterative Algorithm and Convergence Analysis . . . . . . . 110
Contents xi

7.4 Games with Coupled Constraints: Lagrangian Pricing Approach . . 114

7.4.1 Lagrangian Pricing and Duality Extension . . . . . . . . . 114
7.4.2 Iterative Algorithm . . . . . . . . . . . . . . . . . . . . . 119
7.5 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
8 Games in Network Topologies . . . . . . . . . . . . . . . . . . . . . . 121
8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
8.2 Games in γ -Link Topologies . . . . . . . . . . . . . . . . . . . . 122
8.2.1 Partitioned Game Formulation . . . . . . . . . . . . . . . . 122
8.3 Games in Single-Sink Topologies . . . . . . . . . . . . . . . . . . 128
8.3.1 Convexity Analysis (Coupled Constraints) . . . . . . . . . 129
8.3.2 Hierarchical Decomposition . . . . . . . . . . . . . . . . . 130
8.4 Games with Coupled Constraints: Mesh Topologies . . . . . . . . 134
8.4.1 Partitioned Game Formulation . . . . . . . . . . . . . . . . 137
8.4.2 Hierarchical Decomposition . . . . . . . . . . . . . . . . . 138
8.4.3 Primal–Dual (Channel-Link) Iterative Algorithm . . . . . . 140
8.5 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
9 Nash Equilibria Efficiency and Numerical Studies . . . . . . . . . . 143
9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
9.2 A System Optimization Formulation . . . . . . . . . . . . . . . . 144
9.3 Distributed Algorithm . . . . . . . . . . . . . . . . . . . . . . . . 148
9.4 Numerical Study of Efficiency in Nash Game . . . . . . . . . . . . 151
9.5 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
10 Simulations and Experimental Studies . . . . . . . . . . . . . . . . . 161
10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
10.2 Point-to-Point Link Topologies . . . . . . . . . . . . . . . . . . . 162
10.2.1 Simulations Results . . . . . . . . . . . . . . . . . . . . . 163
10.2.2 Experimental Results . . . . . . . . . . . . . . . . . . . . 165
10.3 Multi-link and Mesh Topologies . . . . . . . . . . . . . . . . . . . 168
10.3.1 Multi-link Topologies . . . . . . . . . . . . . . . . . . . . 171
10.3.2 Quasi-ring Topologies . . . . . . . . . . . . . . . . . . . . 173
10.3.3 Mesh Topologies . . . . . . . . . . . . . . . . . . . . . . . 177
10.3.4 Experimental Results . . . . . . . . . . . . . . . . . . . . 180

Part III Robustness, Delay Effects, and Other Problems

11 Robustness and Delay Effects on Network Games . . . . . . . . . . . 189
11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
11.2 System Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
11.3 Delay Effects in Network Games Without Constraints . . . . . . . 192
11.4 Delay Effects in Network Games with Constraints . . . . . . . . . 197
11.4.1 Delayed Primal–Dual Algorithm . . . . . . . . . . . . . . 197
11.4.2 Time-Scale Stability Analysis: Point-to-Point Topologies . 201
11.5 Robustness and Delay Effects Combined . . . . . . . . . . . . . . 204
11.5.1 OSNR Model with Time Delays and Uncertainties . . . . . 204
xii Contents

11.5.2 Single-Link Analysis . . . . . . . . . . . . . . . . . . . . 206

11.6 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
12 Games for Routing and Path Coloring . . . . . . . . . . . . . . . . . 211
12.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211
12.2 Game-Theoretic Model . . . . . . . . . . . . . . . . . . . . . . . 213
12.3 Cost Functions and PoA . . . . . . . . . . . . . . . . . . . . . . . 217
12.4 Solution to Online S-RPC as Nash Equilibria . . . . . . . . . . . . 220
12.5 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
13 Summary and Conclusions . . . . . . . . . . . . . . . . . . . . . . . 225
Appendix A Supplementary Material . . . . . . . . . . . . . . . . . . . 229
A.1 Notations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
A.2 Standard Optimization Review . . . . . . . . . . . . . . . . . . . 230
A.3 Diagonally Dominant and M-matrices . . . . . . . . . . . . . . . . 233
A.4 Maximum Theorem . . . . . . . . . . . . . . . . . . . . . . . . . 234
A.5 Fixed-Point Theorems . . . . . . . . . . . . . . . . . . . . . . . . 235
A.6 Projection Theorem . . . . . . . . . . . . . . . . . . . . . . . . . 236
A.7 Lipschitz Continuity . . . . . . . . . . . . . . . . . . . . . . . . . 236
A.8 Variational Inequalities . . . . . . . . . . . . . . . . . . . . . . . 237
A.9 Singular Perturbation Theory . . . . . . . . . . . . . . . . . . . . 237
A.10 Time-Delay Stability Results . . . . . . . . . . . . . . . . . . . . 240
A.11 Time-Delay Stability Theory . . . . . . . . . . . . . . . . . . . . 240
A.11.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . 240
A.11.2 Razumikhin Stability Theory . . . . . . . . . . . . . . . . 243
Appendix B List of Notations . . . . . . . . . . . . . . . . . . . . . . . . 247
B.1 Notations and Symbols . . . . . . . . . . . . . . . . . . . . . . . 247
B.2 List of Symbols . . . . . . . . . . . . . . . . . . . . . . . . . . . 248
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
Subject Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259
List of Acronyms

ABC Automatic power control

APC Automatic power control
ASE Amplified spontaneous emission noise
BER Bit-error rate
CW Continuous wave
DSL Digital subscriber line
GA Gradient algorithm
LHS Left-hand side
LS Light source
NE Nash equilibrium
OA Optical amplifier
OADM Optical add/drop multiplexer
OLT Optical line terminal
ONTS Optical network test system
OSA Optical spectrum analyzer
OSC Optical service channel
OSNR Optical signal-to-noise ratio
OXC Optical cross-connect
PUA Parallel update algorithm
RHS Right-hand side
Rx Receiver
SIR Signal to interference ratio
Tx Transmitter
VOA Variable optical attenuator
VOL Virtual optical link
WDM Wavelength division multiplexing

xiii
Chapter 1
Introduction

Abstract This chapter provides an introduction and overview of the monograph,

which is aimed at understanding how control algorithms can be designed for opti-
cal networks from a game-theoretic perspective. The first section gives a review of
work in game theory for networks, followed by a brief introduction to the area of
optical networks. The last section presents the scope of the monograph, followed by
a chapter by chapter description of the monograph.

1.1 Game Theory in Networks

Our dependence on communication networks to distribute information is growing
at an exponential rate. Different communication networks have evolved to provide
a variety of services to end users. These communication networks include radio
networks, satellite networks, and optical networks to name a few. In the past decade,
game theory and system control have been utilized to optimize the performance of
these networks, where heuristics were previously used [68].
Game theory, an area initiated more than fifty years ago [96, 159], has been of
interest for researchers working in a broad range of areas from economics [16],
computer science [73, 97, 129], social studies, and engineering and communication
networks [8, 10, 13, 14, 41, 64, 78, 79, 83, 118, 133, 160, 163]. The recent popularity
it has been enjoying in engineering has to do with the fact that game theory brings
new perspectives to optimization and control of distributed systems and networks.
It incorporates paradigms such as Nash equilibrium and incentive compatibility,
which can help quantifying individual preferences of decision-making agents. In
fact game theory provides a rigorous mathematical framework for modeling actions
of individual selfish or cooperating agents/players and interactions among players.
Furthermore, it has an inherently distributed nature and provides a foundation for
developing distributed algorithms for dynamic resource allocation. Moreover, recent
interest has been on extending the standard game setup in various ways, some having
to do with computation of equilibria [36, 40, 120] others being concerned with the
inherent static nature of a traditional game setup, how to extend it to a dynamic
process by which the equilibria is to be achieved [84, 137] or to address equilibria
efficiency [1, 60, 63, 123].
Game-theoretic applications in communication networks can involve either co-
operative or noncooperative games, static or repeated games, finite or continuous

L. Pavel, Game Theory for Control of Optical Networks, 1

Static & Dynamic Game Theory: Foundations & Applications,
DOI 10.1007/978-0-8176-8322-1_1, © Springer Science+Business Media New York 2012
2 1 Introduction

strategy games [14]. Applications include power control problems in different mul-
tiuser environments [8, 41, 61, 72, 118, 133], routing [12, 99] or congestion con-
trol [4, 11, 13, 21, 76, 129, 139, 160], extending the system-based optimization
approaches [68, 80–82, 87, 153]. The many “players” interact within the network
and make (sequential) decisions, i.e., play a game. For example, in a noncooper-
ative (Nash) game framework the natural players/agents can be the Internet ser-
vice providers (ISP) or domain operators [79, 83], the routers [12, 21, 99, 162], or
even the users themselves in an access network application with dedicated wave-
lengths [112]. As another example in wired networks, there could be two sets of
players: telecom firms/ISP and end users. Both sets of players have different objec-
tives and non-negligible interaction across players exists. In wireless networks there
could be wireless LANs where users/players communicate with a fixed access point,
or wireless ad-hoc networks where users/players communicate with each other in
the absence of any fixed infrastructure support.
In a noncooperative (Nash) game [20, 96, 128] each player pursues the maxi-
mization of its own utility, or equivalently the minimization of its own cost func-
tion, in response to the actions of all other players. The stable outcomes of the
interactions of noncooperative selfish agents correspond to Nash equilibria. On the
other hand, in a cooperative game framework the natural players/agents can be the
network nodes, routers or switches (as software agents), or users [9]. These play-
ers/agents cooperate to redistribute the network resources (bandwidth, wavelength
capacity, power).
Why game theory? Consider the case of a multiple access network problem; most
optimization-based approaches find the optimal multiple access control (MAC) and
routing parameters that optimize network throughput, lifetime, delay etc. and as-
sume all nodes in the network use these parameters. But there is no reason to believe
that nodes will adhere to the actions that optimize network performance. Cheaters
may deviate in order to increase their payoffs which in turn affects other users. In
effect any scenario where there is some strategic interaction among self-interested
players is best modeled via a game-theoretic model. Game theory helps to capture
this interaction, the effect of actions of rational players on the performance of the
network. Although the selfish behavior of players causes system performance loss
in a Nash game [1, 38, 64, 73, 122], it has been shown in [64, 134] that proper
selection of network pricing mechanism can help preventing the degradation of net-
work system performance. In the context of evolution, a game captures the essential
features where strategic interactions occur. The basic idea is that actions which are
more “fit”, given the current distribution of actions, tend over time to displace less
fit actions [47, 49, 50].

1.2 Optical Networks

Optical networks represent a typical example of complex communication networks
and the Internet relies on them for its backbone infrastructure. This book is aimed
at understanding how such control algorithms can be designed for optical networks
from a game-theoretic perspective.
1.2 Optical Networks 3

Fig. 1.1 Network deployment example

Unlike conventional networks which are well-established and even standardized,

the optical networks area is much younger, in fact is still an evolving area. Design-
ing and setting up networking applications within the optical context is inherently
more difficult than in conventional wired or wireless networks in part due to the
more complex physical-layer effects, in part due to the lack of automatic methodolo-
gies developed for optical networks. The combination of (a) need for dynamism—
rapid provisioning of links; or allowing large but fairly brief bursts of data to travel
through the network from source to destination, (b) need for transparency—the de-
sire not to convert from optical to electrical to optical at switches, but to have con-
tinuous optical flow of signals, means that one needs to create a dynamic network
using optical components—which are themselves continuing to evolve with new
capabilities. This involves reconfigurable lightpaths, a dynamic optical layer with
dynamic virtual topologies and optimized, adaptive transport layer (see Fig. 1.1).
One of the aims of this book is to address this issue.
In optical networks information transmission is realized by transmitting several
intensity-modulated wavelengths of light over the same optical fiber. After many
years of delay, communication via the optical fiber is now being accepted in a wide
range of military, aerospace, commercial, and security applications. About 85 per-
cent of all global communications traverse optical fibers, including cable and tele-
phone signals.
Optical networks are comprised of a set of fiber optic links that connect nodes
which facilitate adding, dropping, and rerouting of signal channels in a dynami-
cal manner. Laser-based transmitters emit continuous wave (CW) light with wave-
lengths around 1550 nm (optical frequencies of around 190 THz). This continu-
ous wave light is modulated in intensity by the information data and represents the
transmitted optical signal. Each channel is associated with a single wavelength (fre-
quency) in the beam of light. Signals (channels) are multiplexed into a single beam
of light using wavelength division multiplexing (WDM) before being transmitted
through the network. Signals are demultiplexed at their destination points (Rx)
4 1 Introduction

Fig. 1.2 Network layers

to be processed outside the optical domain. Optical line terminals (OLT) are net-
work elements that interface with the optical network clients (SONET, IP, ATM)
and are comprised of transponders (Tx/Rx pairs), as well as optical multiplex-
ers/demultiplexers. Individual powers of the channels can be adjusted at the signal
sources (Tx) as well as at some intermediary nodes. Optical amplifiers (OAs) are
used to amplify the channel powers every 80 to 100 kilometers in an optical link.
Recent years have witnessed the evolution from point-to-point WDM fiber links to
switched optical networks (also referred to as wavelength-routed optical networks).
Channels can be rerouted dynamically through different links within the network
using optical cross-connects (OXCs) or optical add/drop multiplexers (OADMs).
See the List of acronyms for commonly used ones. By using direct photonic links
and optical-domain connectivity unnecessary and expensive optical to electrical to
optical (o/e/o) conversion is avoided.
In static optical networks, lightpaths are statically provisioned up front. The op-
erational expense of reconfiguring the network is quite high and such reconfigura-
tion is time-consuming in the sense that any change in the virtual topology, i.e., the
topology needed for a set of transmission requests, needs manual reconfiguration
and additional equipments. Evolution of emerging tunable network element (such
as reconfigurable OADMs, OXCs, and tunable lasers) make it possible to realize
reconfigurable optical networks [131, 152, 156, 166]. Specific for these is that net-
work reconfiguration can be quickly performed under software control for arbitrary
virtual topologies (control plane layer, see Fig. 1.2). As important is the fact that
the length of fiber links and the number of optical amplifiers deployed along a link
can be changing. Different channels may travel on different fiber links and arbitrary
virtual topologies can be formed over the transport layer (see Fig. 1.2).
Then the interesting question is how to deploy reconfigurable optical networks in
which quick reconfiguration is realized under software control and at the same time,
the network stability and channel transmission performance are maintained. Typical
optical network topologies are mesh, consisting of interconnection of multiple links,
which complicates network performance analysis and network control. This book is
aimed at understanding how such control algorithms can be designed for optical
networks from a game-theoretic perspective.
1.3 Scope of This Monograph 5

1.3 Scope of This Monograph

From a game theory and control perspective, there are a multitude of problems to be
tackled in optical networks and the field is still in its infancy. An important problem
is how to control the channel transmission performance in optical networks while
incorporating physical-layer impairments. More precisely, transmitted signals de-
grade in quality due to physical-layer impairments (attenuation, noise, dispersion,
nonlinear effects). Channel transmission performance at the physical layer (trans-
port layer) is characterized through the bit-error rate (BER), which is defined as
the average probability of incorrect bit identification. Performance degradation is
associated with an increase in BER, which in turn depends on optical signal-to-
noise ratio (OSNR), dispersion, and various nonlinear effects [2]. While a channel
traverses a number of optical elements (e.g., OAs and OXCs) through the network
(see Fig. 1.3), it may degrade in quality due to these impairments. Moreover, signals
of new established channels affect the quality of existing channels on their shared
fiber links. An optical network is considered as a dynamic multiuser environment, if
the signal of a channel is regarded as interfering for all others on the same fiber link.
Power control is a key issue in designing an interference-limited multiuser commu-
nication network system. A major portion of this monograph will be focused on this
first type of problem by using game-theoretic approaches. Another part of the book
will discuss game-theoretic approaches for other challenging problems such as rout-
ing and wavelength-assignment in optical networks [99, 125, 165] in the context of
reconfigurable optical networks.
For simplicity, we use “optical networks” to generally refer “reconfigurable
switched optical networks”. We also use “channel (input) power” to refer to
“channel (input) signal power”. Our approach is primarily based on game theory.
Throughout the book the focus is on analysis of dynamic systems arising from game
formulations with non-separable player utilities and with coupled constraints as well
as propagated (modified) constraints, in the class of continuous strategy games. It
turns out this is the case for a class of games in optical networks, where the utility
of each player is affected by decisions of other players, and action sets are no longer
orthogonal (Chaps. 7 and 8).
The monograph is organized as follows. Chapter 2 provides an overview of basic
concepts in game theory, mostly focused on noncooperative (Nash) game theory.
Chapter 3 focuses on matrix games, while Chap. 4 on games with continuous action
sets and cost functions. Chapter 5 presents some relatively new theoretical results for
Nash games with coupled constraints, i.e., coupled action sets. Chapter 6 provides
an overview of basic background on transmission in optical networks and on gen-
eral topologies to be studied. The concepts of optical signal-to-noise ratio (OSNR)
and coupled power constraints in an optical link are introduced, and the various
models are presented. In Chap. 7 the basic formulation of a game framework to-
wards maximizing channel OSNR is presented. We restrict our attention to single
point-to-point optical links, as the simplest network topology. A Nash game played
among channels is set up towards maximizing OSNR, in this first instance without
coupled power constraints. Thereafter, two approaches are considered in order to
6 1 Introduction

Fig. 1.3 Block diagram of an optical WDM network system

incorporate the coupled power constraint: an indirect approach (penalty-based) and

a direct one, based on the theoretical results in Chap. 5. In network configurations,
coupled power constraints are propagated along fiber links and these functions be-
come intricate from the end-to-end point of view. Approaches on how to deal with
this case of games in more complicated network topologies are studied in Chap. 8.
Various cases of increasing complexity are considered: multi-link adjustable topolo-
gies, single-sink topologies and finally mesh topologies. Chapter 9 addresses some
efficiency issues of such a Nash game. Simulations and some practical experimental
implementations are presented in Chap. 10. Network control schemes and game-
theoretic inspired algorithms need to consider the effects of propagation delay. This
is because this propagation delay may not just impact the performance of the algo-
rithms, but it may destabilize the network as a whole. Chapter 11 addresses the
impact of time-delay in optical network control algorithms derived from game-
1.3 Scope of This Monograph 7

theoretic formulations. Stability properties of game and pricing dynamics are in-
vestigated by using Lyapunov theory and a singular perturbation approach, under
the assumption of time-scale separation and considering time-delay effects. Other
game formulations for wavelength-routing and wavelength-assignment (routing and
path-coloring) fall into the class of finite strategy games and are typically treated
as extension of integer-linear programming (ILP) heuristics. These are covered in
Chap. 12.
Techniques developed herein and game-theoretic formulations as exemplified for
optical networks could be well extended to other cases of more general classes of
games with propagated constraints. There is a lot of scope for fundamental control
and game-theoretic research. The hope is that this book will provide background
material such that non-specialists in optical networks can approach such research
problems within the optical networking domain. Its second goal is to provide re-
searchers in the networking and optical community with game-theoretic method-
ologies that could be used to solve optical networking problems.
 Part I
Game Theory Essentials
Chapter 2
Basics of Game Theory

Abstract This chapter provides a brief overview of basic concepts in game theory.
These include game formulations and classifications, games in extensive vs. in nor-
mal form, games with continuous action (strategy) sets vs. finite strategy sets, mixed
vs. pure strategies, and games with uncoupled (orthogonal) vs. coupled action sets.
The next section reviews basic solution concepts, among them Nash equilibria being
of most relevance. The chapter is concluded with some remarks on the rationality
assumption and learning in classical games. The following chapters will introduce
these concepts formally.

2.1 Introduction
Game theory is a branch of applied mathematics concerned with the study of situ-
ations involving conflicting interests. Specifically game theory aims to mathemat-
ically capture behavior in strategic situations, in which an individual’s success in
making choices depends on the choices of others. The field was born with the work
of John von Neumann and Oskar Morgenstern [159], although the theory was de-
veloped extensively in the 1950s by many among whom John Nash [95, 96].
In this chapter we shall introduce the game-theoretic notions in simplest terms.
Our goal will be later on to study and formalize mathematically various game prob-
lems, by which we understand problems of conflict with common strategic features.
While initially developed to analyze competitions in which one individual does bet-
ter at another’s expense (zero-sum games), it has been expanded to treat a wide
class of interactions, which are classified according to several criteria, one of these
being cooperative versus noncooperative interactions. Typical classical games are
used to model and predict the outcome of a wide variety of scenarios involving a
finite number of players (or agents) that seek to optimize some individual objective.
Noncooperative game theory studies the strategic interaction among self-interested
players.
Historically, game theory developments were motivated by studies in economics,
but many interesting game theory applications have emerged in fields as diverse as
biology [141], computer science [54], social science and engineering [74]. In engi-
neering, the interest in noncooperative game theory is motivated by the possibility
of designing large scale systems that globally regulate their performance in a dis-

L. Pavel, Game Theory for Control of Optical Networks, 11

Static & Dynamic Game Theory: Foundations & Applications,
DOI 10.1007/978-0-8176-8322-1_2, © Springer Science+Business Media New York 2012
12 2 Basics of Game Theory

tributed, and decentralized manner. Modeling a problem within a game-theoretic

setting is particularly relevant to any practical application consisting of separate
subsystems that compete for the use of some limited resource. Examples of such
applications include most notably congestion control in network traffic (i.e. Inter-
net, or transportation), problems of optimal routing [11, 13, 14], power allocation in
wireless communications and optical networks [118, 133].
The chapter is organized as follows. We present basic concepts and game formu-
lations, then we review some classifications: games in extensive vs. in normal form,
games with continuous (infinite) action (strategy) set vs. finite action (finite strategy
games), mixed vs. pure strategy games, and games with uncoupled (orthogonal) vs.
coupled action sets. We follow with a discussion of basic solution concepts, among
them Nash equilibria being of most relevance herein. We conclude with some re-
marks on the rationality assumption and learning in classical games.

2.2 Game Formulations

Game theory involves multiple decision-makers and sees participators as competi-
tors (players). In each game players have a sequence of personal moves; at each
move, each player has a number of choices from among several possibilities, also
possible is the chance or random move. At the end of the game there is some payoff
to be gained (cost to be paid) by the players which depends on how the game was
played. Noncooperative game theory [96] studies the strategic interaction among
self-interested players. A game is called noncooperative if each player pursues its
own interests which are partly conflicting with others’. It is assumed that each player
acts independently without collaboration or communication with any of the oth-
ers [96].
This is in contrast to a standard optimization where there is only one decision-
maker who aims to minimize an objective function by choosing values of variables
from a constrained set such that the system performance is optimized (see the ap-
pendix for a review of basic results, mainly drawn from [24]).
So far we have mentioned three elements: alternation of moves (individual or ran-
dom (chance)), a possible lack of knowledge and a payoff or cost function. A game
G consists of a set of players (agents) M = {1, . . . , m}, an action set denoted Ωi
(also referred to as a set of strategies Si ) available for each player i and an individ-
ual payoff (utility) Ui or cost function Ji for each player i ∈ M.
In a game, each player individually takes an optimal action which optimizes its
own objective function and each player’s success in making decisions depends on
the decisions of the others. We define a noncooperative game G as an object specified
by the triplet (M, S, Ω, J), where
S = S 1 × S2 × · · · × Sm
is known as the strategy space,
Ω = Ω1 × Ω2 × · · · × Ωm
2.3 Games in Extensive Form 13

is the action space, and J : Ω → R m , defined as

 T
J(u) = J1 (u), . . . , Jm (u) , u∈Ω
is the vector of objective functions associated to each of the m players, or agents
participating in the game. In some cases (see Chap. 12) a graph notation might be
more appropriate than the set M notation. Conventionally J represents a vector of
cost functions to be minimized by the agents. An alternative formulation is to use
a vector of utility functions (payoff functions) U to be maximized by the agents.
Without loss of generality we assume that each agent aims to minimize its cost so
we will use J throughout. Since usually we shall identify each player’s action set
and cost function we shall use the notation G(M, Si , Ωi , Ji ), where the subscript is
associated to each player i, i ∈ M. In some cases we shall identify strategies with
actions (one-shot games) and drop one of the arguments. The properties of the sets
Ωi and the functions Ji (u), i ∈ M depend on the modeling scenario, and hence the
type of game under consideration.
Each player’s success in making decisions depends on the decisions of the oth-
ers. Let Ωi denote the set of actions available to player i, which can be finite or
infinite. This leads to either finite actions set games, also known as matrix games, or
infinite (continuous action set) games. In the latter case each player can choose its
action from a continuum of (possibly vector-valued) alternatives. A strategy can be
regarded as a rule for choosing an action, depending on external conditions. Once
such a condition is observed, the strategy is implemented as an action. In the case of
mixed strategies, this external condition is the result of some randomization process.
Briefly, a mixed strategy for agent i is a probability distribution x i over its action
set Ωi . In some cases actions are pure, or independent of any external conditions,
and the strategy space coincides with the action space. In discussing games in pure
strategies we shall use the term “strategy” and “action” interchangeably to refer to
some u ∈ Ω, and the game G can simply be specified as the pair G(M, Ωi , Ji ).
In the introduction we have already distinguished between cooperative and non-
cooperative games. There are numerous other classifications of games, but only a
few are relevant to our purposes in this monograph. We will briefly review the dis-
tinctions between these types of game and introduce these concepts for possible
forms of a game as well as what we understand by various solution concepts.

2.3 Games in Extensive Form

The extensive form of a game amounts to a translation of all the rules into technical
terms of a formal system designed to describe all games.
Extensive-form games generally involve several acts or stages, and each player
chooses a strategy at each stage. The game’s information structure, i.e., how much
information is revealed to which players concerning the game’s outcomes and their
opponents’ actions in the previous stages, significantly affects the analysis of such
games. Extensive-form games are generally represented using a tree graph. Each
14 2 Basics of Game Theory

node (called a decision node) represents every possible state of play of the game
as it is played [20]. Play begins at a unique initial node, and flows through the tree
along a path determined by the players until a terminal node is reached, where play
ends and cost s are assigned to all players. Each non-terminal node belongs to a
player; that player chooses among the possible moves at that node, each possible
move is an edge leading from that node to another node. Their analysis becomes
difficult with increasing numbers of players and game stages.
A formal definition is as follows.

Definition 2.1 An m-player game G in an extensive form is defined as a graph

theoretic tree of vertices (states) connected by edges (decisions or choices) with
certain properties:
1. G has a specific vertex called the starting point of the game,
2. a function called the cost function which assigns an m-vector (tuple) J1 , . . . , Jm
to each terminal vertex (outcome) of the game G, where Ji denotes the cost of
player i, M = {1, . . . , m},
3. each non-terminal vertex of G is partitioned into m + 1 possible sets,
S0 , S1 , . . . , Sm called the player sets, where S0 stands for the choice of chance
(nature),
4. each vertex of S0 has a probability distribution over the edges leading from it,
5. the vertices of each player Si , i = 1, . . . , m are partitioned into disjoint subsets
known as information sets, Si,j , such that two vertices in the same information
set have the same number of immediate (choices/edges) followers and no vertex
can follow another vertex in the same information set.

As a consequence of (5) a player knows which information set he is in but not

which vertex of the information set.
A player i is said to have perfect information in a game G is each information
set for this player consists of one element. The game G in extensive form is said
to have perfect information if every player has perfect information. A pure strategy
for player i denoted by ui is defined as a function which assigns to each of player’s
i information sets Si,j , one of the edges leading from a representative vertex in
this set Si,j . As before we denote by Ωi the set of all pure strategies of player i,
ui ∈ Ωi and by u = (u1 , . . . , um ) the m-tuple of all players strategies, with u ∈ Ω =
Ω1 × · · ·× Ωm . A game in extensive form is finite if it has a finite number of vertices,
hence each player has only a finite number of strategies. Let us look at a couple of
examples.

Example 2.2 In the game of Matching Pennies (see Fig. 2.1) player 1 chooses
“heads” (H) or “tails” (T), player 2, not knowing this choice, also chooses between
H or T. If the two choose alike (matching) than player 2 wins 1 cent from player 1
(hence +1 for player 2 and −1 for player 1); else player 1 wins 1 cent from player
2 (reverse case). The game tree is shown below with vectors at the terminal vertices
indicating the cost function, while the number near vertices denote the player to
2.3 Games in Extensive Form 15

Fig. 2.1 Game of matching

pennies in extensive form

Fig. 2.2 Game of matching

pennies in extensive form

Fig. 2.3 Game of matching

pennies in extensive form

whom the move corresponds. The dotted (shaded) area indicates moves in the same
information set.

The next two figures show other two zero-sum game examples which differ by the
information available to player 2 at the time of its play (information set), denoted
by the shaded area (dotted). In the first case, Fig. 2.2, the two possible nodes of
player 2 are in the same information set, implying that even though player 1 acts
before player 2 does, player 2 does not have access to it s opponent decision. This
means that at the time of its play, player 2 does not know at which node (vertex)
he is. This is as saying that both players act simultaneously. The extensive form in
the second case Fig. 2.3, admits a different matrix game in normal form. In this
case each node of player 2 is included in a separate information set, i.e., has perfect
information as to which branch of the tree player 1 has chosen.
16 2 Basics of Game Theory

2.4 Games in Normal Form

Games in normal form (strategic form) model scenarios in which two or more play-
ers must make a one-time decision simultaneously. These games are sometimes re-
ferred to a one-shot game, simultaneous move games. The normal form is a more
condensed form of the game, stripped of all features but the choice of each player’s
pure strategies, and it is more convenient to analyze. The fact that all players make
their choice of strategy simultaneously has nothing to do with a temporal constraint,
but rather with a constraint on the information structure particular to this type of
game. The information structure of a game is a specification of how much each
player knows at the time he chooses his strategy. For example, in Stackelberg games
[20], where there are leaders and followers, some players (followers) choose their
strategies only after the strategic choices made by the leaders have already been
revealed.
In order to describe a normal-form game we need to specify players’ strategy
spaces and cost functions. A strategy space for a player is the set of all strategies
available to that player, where a strategy is a complete plan of action for every stage
of the game, regardless of whether that stage actually arises in play. A cost function
of a player is a mapping from the cross-product of players’ strategy spaces to that
player’s set of costs (normally the set of real numbers). We will be mostly concerned
with these type of normal-form games herein.
For any strategy profile u ∈ Ω, where u = (u1 , . . . , um ) ∈ Ω is the m-tuple of
players’ pure strategies and Ω = Ω1 ×· · ·×Ωm is the overall pure strategy space, let
Ji (u) ∈ R denote the associated cost for player i that depends on all players’ strate-
gies.These costs depend on the context: in economics they represent a firm’s profits
or a consumer’s (von Neumann–Morgenstern) utility, while in biology they repre-
sent the fitness (expected number of surviving offspring). All these real numbers
Ji (u), i ∈ M, form the combined pure strategy vector cost function of the game,
J : Ω → Rm . A normal-form game G(M, Ωi , Ji ) is defined by specifying Ωi and
Ji . It is possible to tabulate functions Ji for all possible values of u1 , . . . , um ∈ Ω
either in the form of a relation (easier for continuous or infinite games when Ω is
a continuous set), or, as an m-dimensional array (table) in the case of finite games
(when Ω is finite set). In this latter case and when m = 2 this reduces to a ma-
trix whose size is given by the number of available choices for the two players and
whose elements are pairs of real numbers corresponding to outcomes (costs) for the
two players. Let us look at a few examples for m = 2, where we shall use rows for
player 1 as the columns for player 2. Hence entry (j, k) indicates the outcome of
player 1 using the j pure strategy and player 2 using k strategy.
Example 2.3 (Matching Pennies) Consider the game of Matching Pennies above,
where each player has two strategies “Heads” (H) or “Tail” (T). The normal form of
this game is described by the matrix
player 2
H T
player 1 H (−1, 1) (1, −1)
T (1, −1) (−1, 1)
2.5 Game Features 17

or given as the matrix

 
(−1, 1) (1, −1)
M=
(1, −1) (−1, 1)
Most of the times instead of M we shall use a pair of cost matrices (A, B) to indicate
the outcome for each player separately, matrix A for player 1 and matrix B for
player 2. For the above game this simply means the pair of matrices (A, B) where
   
−1 1 1 −1
A= , B=
1 −1 −1 1

One can transform any game in extensive form into an equivalent game in normal
form, so we shall restrict most of our theoretical development to games in normal
form only.

2.5 Game Features

Depending of various features of the game one could classify them in different cat-
egories. Below we briefly discuss such classification depending on the competitive
nature of the game, the knowledge/information available to the players, and the
number of times the game is repeated.

2.5.1 Strategy Space: Matrix vs. Continuous Games

In a matrix game with m players, each player i has a finite number of discrete
options to choose from, i.e., there are ni possible actions, so that the set of its actions
is simply identified with a set of indices Mi = {1, . . . , ni } corresponding to these
possible actions. Then one considers the action ui ∈ Ωi with the action sets being
defined as Ωi = {e1 , . . . , eni }, and ej being the j th unit vector in Rni . Given the
action u ∈ Ω chosen by all players, player i has a cost matrix Ai ∈ Rn1 ×···×nm ,
∀i ∈ M, that defines his cost by
Ji (u) = [Ai ]u1 ,...,um ∈ R, i∈M
This is easiest seen in the case of a matrix game when we can explicitly write the
cost functions as
J1 (u) = uT1 A u2 , J2 (u) = uT1 B u2 (2.1)
Such as two-player matrix game is the Matching Pennies game in Example 2.3.
A symmetric game is a game where the payoffs for playing a particular strategy
depend only on the other strategies employed, not on who is playing them. If the
identities of the players can be changed without changing the payoff to the strate-
gies, then a game is symmetric and this corresponds to B = AT . Many of the com-
monly studied 2 × 2 games are symmetric. The standard representations of the
18 2 Basics of Game Theory

Chicken game, Prisoner’s Dilemma game, and the Stag Hunt game are all symmet-
ric games [20].
Unlike matrix games, where players have a finite set of actions, in a continuous
game each player can choose its action from a continuum of (possibly vector-valued)
alternatives, that is, Ωi ⊂ Rni . We shall review results for both matrix games and
continuous games in the next two chapters, but most of the games we shall consider
afterwards are continuous games.

2.5.2 Mixed vs. Pure Strategy Games

A strategy can be regarded as a rule for choosing an action, depending on external

conditions. Once such a condition is observed, the strategy is implemented as an
action. In the case of mixed strategies, this external condition is the outcome of some
randomization process. Consider an m-player matrix game and denote by xi,j the
probability that player i will choose action j from ni his available alternatives in Ωi .
Then a mixed strategy xi is defined as the vector composed of the probabilities
associated with available actions, i.e., xi = [xi,j ], j = 1, . . . , ni , xi ∈ Δi , i ∈ M
where
 
ni
Δi := xi ∈ R | xi,j ≥ 0,
ni
xi,j = 1 , ∀i ∈ M
j =1
is a simplex. In some cases actions are pure, or independent of any external condi-
tions, and the strategy space coincides with the action space. In discussing games in
pure strategies we shall use the term “strategy” and “action” interchangeably to refer
to some u ∈ Ω, and the game G can simply be specified as the pair G(M, Ωi , Ji ).
This will be the case considered throughout most of the monograph.

2.5.3 Competitive Versus Cooperative

A cooperative game is one in which there can be cooperation between the players
and/or they have the same cost (also called team games). A noncooperative game
is one where an element of competition exists and among these we can mention
coordination games, constant-sum games, and games of conflicting interests. We
give below a few such examples.

2.5.3.1 Coordination Games

In coordination games, what is good for one player is good for all players. An ex-
ample coordination game in normal form is described by
 
(−3, −3) (0, 0)
M=
(0, 0) (−4, −4)
2.5 Game Features 19

In this game, players try to coordinate their actions. The joint action (j, k) = (2, 2)
is the most desirable (least cost), but the joint action (j, k) = (1, 1) also produces
negative costs to the players. This particular game is called a pure coordination game
since the players always receive the same payoff.
Other coordination games move more toward the domain of games of conflicting
interest. For example, consider the Stag Hunt game: stag hare (we shall come back
to this example)
 
(−4, −4) (0, −1)
M=
(−1, 0) (−1, −1)
In this game, each player can choose to hunt stag (first row or first column) or hare
(second row or second column). In order to catch a stag (the biggest animal, hence
the bigger payoff or lowest cost of −4), both players must choose to hunt the stag.
However, a hunter does not need help to catch a hare, which yields a cost of −1.
Thus, in general, it is best for the hunters to coordinate their efforts to hunt stag, but
there is considerable risk in doing so (if the other player decides to hunt hare). In
this game, the costs (payoffs) are the same for both players when they coordinate
their actions, but their costs are not equal when they do no coordinate their actions.

2.5.3.2 Constant-Sum Games

Constant-sum games are games in which the sum of the players’ payoffs sum to the
same number. These games are games of pure competition of the type “my gain is
your loss”. Zero-sum games are particular example of these games, which in terms
of the two-players cost matrices can be described by B = −A. An example of such
game is the Rock, Paper, and Scissors game with the matrix form
⎡ ⎤
(0, 0) (1, −1) (−1, 1)
M = ⎣ (−1, 1) (0, 0) (1, −1) ⎦
(1, −1) (−1, 1) (0, 0)

2.5.3.3 Games of Conflicting Interests

These fall in between constant-sum games and coordination games and cover a large
class, whereby the players have somewhat opposing interests, but all players can
benefit from making certain compromises. One can say that people (and learning
algorithms) are often tempted to play competitively in these games (both in the real
world and in games), though they can often hurt themselves by doing so. However,
on the other hand, taking an extreme cooperative approach (same actions) can lead
to similarly bad (or worse) payoffs (high costs). One of the most celebrated games
of this type is the Prisoners’ Dilemma game, with the choices of to “Confess” (co-
operate) or “Don’t Confess” (defect) as the actions of two prisoners (players) put in
separate cells. If they both confess each they each receive 3 years in prison. If only
one confesses, he will be freed, used as witness, and the other will be convicted
20 2 Basics of Game Theory

and receive 8 years in prison. If neither confesses they will be convicted of a minor
offense and receive each only 1 year. The normal form (strategic form) of this game
is described by the matrix
 
(−3, −3) (0, −8)
M=
(−8, 0) (−1, −1)
or, as pair of cost matrices (A, B) for the two players,
   
−3 0 −3 −8
A= , B=
−8 −1 0 −1

2.5.4 Repetition

Any of the previously mentioned kinds of game can be played any number of times
between the same players, and the game can be the same at each play or can be
state-dependent.

2.5.4.1 One-Shot Games

In one-shot games, players interact for only a single round (or stage). Thus, in these
situations there is no possible way for players to reciprocate (by inflicting punish-
ment or rewards) thereafter.

2.5.4.2 Repeated Games

In repeated games, players interact with each other for multiple rounds (playing the
same game). In such situations, players have opportunities to adapt to each others’
behaviors (i.e., “learn”) in order to try to become more successful. There can be
finite-horizon repeated games where the same game is repeated a fixed number of
times by the same players, or infinite-horizon games in which the play is repeated
indefinitely.

2.5.4.3 Dynamic Games

The case where the game changes when players interact repeatedly is what can be
called a repeated dynamic game, characterized by a state. These are also called dif-
ferential games. Unlike a repeated game where the agents play the same game every
time, in a dynamic game the state of the game influences the play and the outcome.
Important in this class are the so called stochastic games, which are extensions of
Markov decision processes to the scenario with m multiple players, where prob-
abilistic transitions are modeled. We shall not cover these types of game in this
monograph.
2.6 Solution Concepts 21

2.5.5 Knowledge Information

Depending on the amount of information a player has different plays and outcomes
may be possible. For example, does an player know the costs (or preference order-
ings) of other players? Does the player know its own cost (payoff) matrix? Can he
view the actions and costs of other players? All of these (and other related) questions
are important as they can help determine how the player should learn and act. The-
oretically, the more information an player has about the game, the better he should
be able to do. In short, the information an player has about the game can vary along
the following dimensions: knowledge of the player’s own actions; knowledge of the
player’s own costs; knowledge of the existence of other players; knowledge of the
other players’ actions; knowledge of the other players’ costs and in case learning is
used, knowledge of the other players’ learning algorithms.
In a game with complete information each player has knowledge of the payoffs
and possible strategies of other players. Thus, incomplete information refers to situ-
ations in which the payoffs and strategies of other players are not completely known.
The term perfect information refers to situations in which the actual actions taken by
associates are fully observable. Thus, imperfect information implies that the exact
actions taken by associates are not fully known.

2.6 Solution Concepts

Given a game’s specification G(M, Ωi , Ji ) an important issue is to predict how the

game will be played, i.e., to determine its outcome. These predictions are called
solutions, and describe which strategies will be adopted by players, therefore pre-
dicting the result of the game. A solution concept briefly describes how to use a
certain set of mathematical rules to decide how to play the game. Various solution
concepts have been developed, in trying to indicate/predict how players will behave
when they play a generic game. Herein we only introduce these solution concepts
in short.

2.6.1 Minimax Solution

One of the most basic properties of every game is the minimax solution (or minimax
strategy), also called security strategy. The minimax solution is the strategy that
minimizes a player’s maximum expected loss (cost). There is an alternate set of
terminology we can use (often used in the literature as we mentioned before). Rather
than speak of minimizing our maximum expected loss, we can talk of maximizing
our minimum expected payoff. This is known as the maximin solution. Thus, the
terms minimax and maximin can be used interchangeably. The minimax solution is
an essential concept for zero-sum games.
22 2 Basics of Game Theory

Let us look at the Prisoner’s Dilemma matrix game above. In the prisoner’s
dilemma, both players are faced with the choice of cooperating or defecting. If both
players cooperate, they both receive a relatively low cost (which is −3 in this case).
However, if one of the players cooperates and the other defects, the defector receives
a very low cost (−8 in this case) (called the temptation cost), and the cooperator re-
ceives a relatively high cost (0 in this case). If both players defect, then both receive
a higher cost (which is −1 in this case). So what should you do in this game? Well,
there are a lot of ways to look at it, but if you want to play conservatively, you might
want to invoke the minimax solution concept, which follows from the following rea-
soning. If you play cooperate, the worst you can do is get a cost of 0 (thus, we say
that the security of cooperating is 0). Likewise, if you play defect, the worst you
can do is get a cost of −1 (security of defecting is −1). Alternately, we can form a
mixed strategy over the two actions. However, it just so happens in this game that
no mixed strategy has higher security than defecting, so the minimax strategy in this
game is to defect. This means that the minimax value (which is the maximum cost
one can incur when plays the minimax strategy) is −1.
However, even though the minimax value is the lowest cost you can guarantee
yourself without the cooperation of your associates, you might be able to do much
better on average than the minimax strategy if you can either outsmart your asso-
ciates or get them to cooperate or compromise with you (in a game that is not fully
competitive). So we need other solution concepts as well.

2.6.2 Best Response

Another basic solution concept in multi-player games is to play the strategy that
gives you the lowest cost given your opponents’ strategies. That is exactly what
the notion of the best response suggests. Suppose that you are player i, and your
opponents’ play u−i . Then the your best response in terms of pure strategies is u∗i
such that

Ji u−i , u∗i ≤ Ji (u−i , ui ), ∀ui ∈ Ωi
In the case of mixed strategies, assuming your opponents’ play the strategy x−i ,
your best response is the strategy xi∗ such that

Ji x−i , xi∗ ≤ Ji (x−i , xi ), ∀xi ∈ Δi
where Δi is the probability simplex. The best-response idea has had a huge impact
on learning algorithms. If you know what your other players are going to do, why not
get the lowest cost (highest payoff) you can get (i.e., why not play a best response)?
Taking this one step further, you might reason that if you think you know what other
players are going to do, why not play a best response to that belief? While this
obviously is not an unreasonable idea, it has two problems. The first problem is that
your belief may be wrong, which might expose you to terrible risks. Secondly, this
“best-response” approach can be quite unproductive in a repeated game when other
players are also learning/adapting [48].
2.6 Solution Concepts 23

2.6.3 Nash Equilibrium Solution

We now introduce briefly a most celebrated solution concept for a N -player nonco-
operative game G. John Nash’s identification of the Nash equilibrium concept has
had perhaps the single biggest impact on game theory. Simply put, in a Nash equi-
librium, no player has an incentive to unilaterally deviate from its current strategy.
Put another way, if each player plays a best response to the strategies of all other
players, we have a Nash equilibrium.
We will discuss the extent to which this concept is satisfying by looking at a few
examples later on.

Definition 2.4 Given a game G a strategy N -tuple (profile) u∗ = (u∗1 , . . . , u∗N ) is

said to be a Nash equilibrium (or in equilibrium) if and only if
 
Ji u∗1 , . . . , u∗N ≤ Ji u∗1 , . . . , u∗i−1 , ui , u∗(i+1) , . . . , u∗N , ∀ui ∈ Ωi , ∀i ∈ M (2.2)
or, in compact notation,
 
Ji u∗−i , u∗i ≤ Ji u∗−i , ui , ∀ui ∈ Ωi , ∀i ∈ M
where u∗ = (u∗−i , u∗i ) and u∗−i denotes u∗ of all strategies except the ith one.

Thus u∗ is an equilibrium if no player has a positive incentive for unilateral

chance of his strategy, i.e., assuming the others keep their same strategies. In partic-
ular this means that once all choices of pure strategies have been revealed no player
has any cause for regret (hence the point of no regret concept). A similar definition
holds for x mixed strategies (as seen above in the best response).

Example 2.5 Consider the game with normal form

player 2
u2,1 u2,2
player 1 u1,1 (3, 1) (0, 0)
u1,2 (0, 0) (1, 3)
and note that both (u1,1 , u2,1 ) and (u1,2 , u2,2 ) are equilibrium pairs. For matrix
games we shall use the matrix notation and for the above we will say that (3, 1)
and (1, 3) are equilibria.

If we look at another game (a coordination game),

 
(−3, −3) (0, 0)
M=
(0, 0) (−1, −1)
By inspection as in the above, we can conclude that both the joint actions (j, k) =
(1, 1) and (j, k) = (2, 2) are Nash equilibria since in both cases, neither player can
benefit by unilaterally changing its strategy. Note, however, that this illustrates that
not all Nash equilibria are created equally. Some give better costs than others (and
some players might have different preference orderings over Nash equilibrium).
24 2 Basics of Game Theory

While all the Nash equilibria we have identified so far for these two games are
pure strategy Nash equilibrium, they need not be so. In fact, there is also a third
Nash equilibrium in the above coordination game in which both players play mixed-
strategies. The next chapter we shall formally review this extension.
Here are a couple more observations about the Nash equilibrium as a solution
concept:
• In constant-sum games, the minimax solution is a Nash equilibrium of the game.
In fact, it is the unique Nash equilibrium of constant-sum games as long is there
is not more than one minimax solution (which occurs only when two strategies
have the same security level).
• Since a game can have multiple Nash equilibrium, this concept does not tell us
how to play a game (or how we would guess others would play the game). This
poses another question: Given multiple Nash equilibria, which one should (or
will) be played? This leads to considering refinements of Nash equilibria.
Strategic dominance is another solution concept that can be used in many games.
Loosely, an action is strategically dominated if it never produces lower costs (higher
payoffs) and (at least) sometimes gives higher costs (lower payoffs) than some other
action. An action is strategically dominant if it strategically dominates all other ac-
tions. We shall formally define this later on. For example, in the Prisoner’s Dilemma
(PD) game, the action defect strategically dominates cooperate in the one-shot game.
This concept of strategic dominance (or just dominance, as we will sometimes call
it) can be used in some games (called iterative dominance solvable games) to com-
pute a Nash equilibrium.

2.6.4 Pareto Optimality

One of the features of a Nash equilibrium (NE) is that in general it does not cor-
respond to a socially optimal outcome. That is, for a given game it is possible for
all the players to improve their costs (payoffs) by collectively agreeing to choose a
strategy different from the NE. The reason for this is that a posteriori some players
may choose to deviate from such a cooperatively agreed-upon strategy in order to
improve their payoffs further at the group’s expense. A Pareto optimal equilibrium
describes a social optimum in the sense that no individual player can improve his
payoff (or lower his cost) without making at least one other player worse off. Pareto
optimality is not a solution concept, but it can be an important attribute in deter-
mining what solution the players should play (or learn to play). Loosely, a Pareto
optimal (also called Pareto efficient) solution is a solution for which there exists no
other solution that gives every player in the game a higher payoff (lower cost). A PE
solution is formally defined as follows.

Definition 2.6 A solution u∗ is strictly Pareto dominated if there exists a joint ac-
tion u ∈ Ω for which Ji (u) < Ji (u∗ ) for all i, and weakly Pareto dominated if there
exists a joint action u = u∗ ∈ Ω for which Ji (u) ≤ Ji (u∗ ) for all i.
2.7 The Rationality Assumption 25

Definition 2.7 A solution u∗ is weakly Pareto efficient (PE) if it is not strictly Pareto
dominated and strictly Pareto efficient (PE) if it is not weakly Pareto dominated.

Often, a Nash equilibrium (NE) is not Pareto efficient (optimal). Then one speaks
of a loss of efficiency, which is also referred to as the Price of Anarchy. An interest-
ing problem is how to design games with improved Nash efficiency, and pricing or
mechanism design is concerned with such issues.
In addition to these solution concepts other important ones include the Stackel-
berg equilibrium [20], which is relevant in games where the information structure
plays an important role, and correlated equilibria [48, 98], which case is relevant
in games where the randomization used to translate players’ mixed strategies into
actions are correlated.

2.7 The Rationality Assumption

Given the number of available solution concepts, NE refinements and the apparent
arbitrariness with which they may be applied, why would one expect that in an
actual noncooperative game players would choose any particular refined NE? This
question turns out to be a valid objection, namely the perfect rationality of all the
participating agents. In the literature rationality is often discussed, without being
precisely defined. One possible formulation is as follows: a player is rational if it
consistently acts to improve its payoff without the possibility of making mistakes,
has full knowledge of other players’ intentions and the actions available to them,
and has an infinite capacity to calculate a priori all possible refinements to NE(G)
in an attempt to find the “best one”. If a game involves only rational agents, each of
whom believe all other agents to be rational, then theoretical results offer accurate
predictions of the game outcomes.

2.8 Learning in Classical Games

Yet another game classification is related to this rationality assumption or the lack
of it. In this monograph we will be concerned with rational players and one-shot
games. A more realistic modeling scenario involves players that are less than ratio-
nal and a repeated game play. We review here very briefly the conceptual differences
for completeness. The reader is referred to extensive references on this topic such
as [48]. We will use the term bounded rationality to describe players that do not
necessarily have access to full or accurate information about the game, and who
have a limited capacity to perform calculations on the information that is available
to them. Instead of immediately playing a perfect move, boundedly rational players
adapt their strategy based on the outcomes of previous matches [48, 49, 137]. We
can refer to this modeling scenario as a classical game with learning. All solution
concepts studied in classical game theory remain important in games with learning.
26 2 Basics of Game Theory

The important problem to study is not only to classify games for which equilibria
exist and have favorable properties such as uniqueness, but also, in conjunction, to
classify the strategy update laws that yield convergence to these equilibria under
repeated play. In the terminology of [158], a game with learning can be said to have
an “inner game” (i.e. the underlying classical game G) and an “outer game” (i.e. the
dynamics of the strategy update laws).
One of earliest strategy update laws to be studied is fictitious play (FP) [31]. In
FP, each player keeps a running average, known as an empirical frequency of his op-
ponent’s actions, and chooses his next move as a best response to this average. The
term “fictitious play” comes from the consideration of playing under the unrealistic
assumption that the opponent is playing a constant strategy, hence he is a fictitious
player. It has been proved that under a number of different sufficient conditions FP
is guaranteed to converge to one. One of the earlier cases studied was the class of fi-
nite, two-player, zero sum games, for which convergence of FP was proved in [127],
followed by other results [75].
A continuous time version of fictitious play can be derived by considering the
infinitesimal motions of the empirical frequencies [58]. It can be shown that if both
players are updating their strategies so that these are the best response to the other
one, we may write
u̇i = −ui + ri (u−i )
This equation known as the best-response (BR) dynamic, and clearly displays the
very convenient feature that u̇ = 0 if and only if u has reached a fixed point of the
reaction function r(u). By the characterization of NE given above (see (4.5)), we
conclude that the set of equilibria of the best-response dynamic coincides precisely
with NE(G). Most of these studies are done within the framework of evolutionary
game theory [132, 142], while this monograph is concerned only with the setup of
classical games.

2.9 Notes

As a preface to the remaining chapters, this chapter provided a brief overview of

basic game concepts.
Chapter 3
Matrix Games

Abstract This chapter considers normal-form games with finite action sets, hence
matrix games. Two-player bimatrix cases are treated first, followed by m-player ma-
trix games, both introducing pure- and mixed-strategy Nash equilibrium concepts.
The concepts of dominance and best replies are reviewed, as well as Nash equilibria
theorem and Nash equilibria refinements. Results are mostly adapted from (Basar
and Olsder in Dynamic Noncooperative Game Theory. SIAM Series Classics in
Applied Mathematics, 1999; Owen in Game Theory, Academic Press, San Diego,
1995).

3.1 Introduction

In this chapter we consider the class of m-player matrix games, where each player i
has a finite number of discrete options to choose from, i.e., there are ni possible ac-
tions, so that the set of its actions is simply identified with a set of indices {1, . . . , ni }
corresponding to these possible actions. On the other hand, in a continuous game
each player can choose its action from a continuum of (possibly vector-valued) al-
ternatives, that is, Ωi ⊂ Rni . This will be the focus of the next chapter.
We consider first the two-player matrix game case, or bimatrix games, and gen-
eralize then to m-players matrix games. We discuss pure and mixed- strategy game
formulations, review concepts of dominance and best replies, and then prove the
important Nash equilibria theorem, followed by a brief review of Nash equilibria
refinements.

3.2 Bimatrix Games

Consider a two-player matrix game, where player 1 and 2 each have a finite number
n1 and n2 of discrete options or pure strategies to choose from. Then the set of their
actions Ω1 and Ω2 can be simply identified with the set of indices M1 := {1, . . . , n1 }
and M2 := {1, . . . , n2 } corresponding to these possible actions.
Let the action of player 1 be denoted by u1 ∈ Ω1 ; the j th action can be identified
with the index j ∈ M1 . Similarly for player 2, let its action be denoted by u2 ∈ Ω2 ;

L. Pavel, Game Theory for Control of Optical Networks, 27

Static & Dynamic Game Theory: Foundations & Applications,
DOI 10.1007/978-0-8176-8322-1_3, © Springer Science+Business Media New York 2012
28 3 Matrix Games

the kth action can be identified with the index k ∈ M2 . Each of the two players have
cost matrix A and B, respectively. Then we can write for the game outcome denoted
by J1 when strategy pair (j, k) is used
T 
aj k = e1j Ae2k := J1 e1j , e2k

where e1j ∈ R n1 and e2k ∈ R n2 are the j th unit vector in R n1 and the kth unit vector
in R n2 , respectively. Thus player 1’s cost when strategy pair (u1 , u2 ) is set to be the
(j, k) pair is
 T
J1 (u1 , u2 ) = J1 e1j , e2k = e1j Ae2k

Then for player 1 we can write u1 ∈ Ω1 := {e11 , . . . , e1j , . . . , e1n1 } and correspond-
ingly for player 2, u2 ∈ Ω2 := {e21 , . . . , e2k , . . . , e2n2 }, and its cost when (u1 , u2 ) is set
to be the (j, k) pair is
 T
J2 (u1 , u2 ) = J2 e1j , e2k = e1j Be2k

Each of the players aims to minimize its own cost, J1 and J2 , respectively, described
by its own cost matrix A and B, respectively. We call this a (A, B) bimatrix game.
In the case when B = −A the game is called a zero-sum game.
The definition of a Nash equilibrium in pure strategies for a bimatrix game is
given next.

Definition 3.1 (Pure-strategy Nash equilibrium (NE)) For a given (A, B) bimatrix
game let (j ∗ , k ∗ ) be pair of pure strategies chosen by the two players. Then if for
any other pure strategies (j, j ) both

aj ∗ k ∗ ≤ aj k ∗ , ∀j = 1, . . . , n1
bj ∗ k ∗ ≤ bj ∗ k , ∀k = 1, . . . , n2

hold, then (j ∗ , k ∗ ) is a pure-strategy Nash equilibrium (NE) solution for the bima-
trix game, and (aj ∗ k ∗ , bj ∗ k ∗ ) is a NE equilibrium outcome of the game.

In the case when more than a single NE solution exist, we shall denote this set
by NE(G). The game outcome can be different for different NE solutions and the
question of ordering the elements in NE(G) arises. Since we cannot have total or-
dering between pairs of numbers one can resort to partial ordering in order to arrive
at some preferential choice.

Definition 3.2 A pair of pure strategies (j ∗ , k ∗ ) is said to be better than another

pair (j˜, k̃) if
aj ∗ k ∗ ≤ aj˜k̃ , and bj ∗ k ∗ ≤ bj˜k̃
and if at least one of these inequalities is strict.
3.2 Bimatrix Games 29

It can be shown that the set of NE equilibrium points is invariant to positive affine
transformations on the cost functions.

Definition 3.3 Two bimatrix games (A, B) and (Ã, B̃) are said to be strategically
equivalent games if there exist positive constants α1 , α2 > 0 and scalar β1 , β2 such
that

ãj k ∗ = α1 aj k ∗ + β1 , α1 > 0, β1 ∈ R, ∀j
b̃j ∗ k = α2 bj ∗ k + β2 , α2 > 0, β2 ∈ R, ∀k

(operation on columns in A and on rows in B).

Proposition 3.4 All strategically equivalent bimatrix games have the same NE equi-
libria.

We consider below a couple of examples and their pure NE equilibria.

Example 3.5 (Prisoner’s Dilemma) Recall the Prisoner’s Dilemma (PD) game re-
viewed in Chap. 2, where cost matrices (A, B) for the two players are
   
−3 0 −3 −8
A= , B=
−8 −1 0 −1

for the choices of to “Confess” (first strategy) or to “Not Confess” (second strategy)
available for the two prisoners. This is a game in which there are gains from coop-
eration between players; the best outcome is for the players to not confess (second
strategy), but each player has an incentive to be a “free rider”. Whatever one player
does, the other prefers “Confess” so that game has a unique pure Nash equilibrium
(Confess, Confess) or pair (1, 1). Indeed player 1’s first pure strategy (“Confess”)
gives a smaller loss (higher payoff) than its second pure strategy, irrespective of what
strategy player 2 uses. Similarly, player 2’s first pure strategy (defect) gives always
a smaller loss (higher payoff) than its second pure strategy: each entry in first col-
umn of B matrix is less than the corresponding entry in the second column. Hence
individual rationality of minimizing its cost or loss leads to each player to select
the first strategy, hence both “Confess”. The dilemma arises because both players
would have lower loss if they were to select their second strategy (“Not Confess”),
but this would require some coordination.

Unfortunately not every game has a pure equilibrium. For example the game of
Matching Pennies in Example 2.3 in Chap. 2 does not have a (pure) Nash equilib-
rium. This leads to the mixed-strategy extension of a game.
30 3 Matrix Games

3.2.1 Mixed Strategies

A player is said to use a mixed strategy whenever he/she chooses to randomize

over the set of available pure actions. Formally, a mixed strategy is a probability
distribution that assigns to each available action a likelihood of being selected. If
only one action has a positive probability of being selected, the player is said to
use a pure strategy. Thus when an equilibrium cannot be found in pure strategies
the space of strategies can be enlarged, and the players are allowed to base their
decisions on random events, hence mixed strategies.
In the Matching Pennies game if both players choose each of their pure strategies
with probability 12 then each will have an expected cost (loss) of 0 which seems an
acceptable solution. This is the case when the game is played repeatedly and each
player aims to minimize the expected (average) outcome of individual plays.
Because randomization is more complex and cognitively demanding than is the
deterministic selection of a single action, this raises the question of how mixed
strategies should be interpreted. An interpretation is given in [57] by John Harsanyi
who showed that a mixed-strategy equilibrium of a game with perfect information
can be viewed as the limit point of a sequence of pure-strategy equilibria of games
with imperfect information. Specifically, starting from a game with perfect infor-
mation, one can obtain a family of games with imperfect information by allowing
for the possibility that there are small random variations in payoffs and that each
player is not fully informed of the payoff functions of the other players. Harsanyi
showed that the frequency with which the various pure strategies are chosen in these
perturbed games approaches the frequency with which they are chosen in the mixed-
strategy equilibrium of the original game as the magnitude of the perturbation be-
comes vanishingly small. The introduction of mixed strategies was successful as a
new solution concept because von Neuman [159] was able to show (even from 1928)
that for any matrix game the minimax value is equal to the maximin value in mixed
strategies, hence any such game has a solution in mixed strategies (von Neumann’s
Minimax Theorem). Nash extended this to any m-player finite matrix game.
Now for a two-player game N = 2, with pure strategies u = (u1 , u2 ) we de-
note by x and y the mixed strategy x1 of player 1 and x2 player 2, respectively,
and x = (x1 , x2 ) = (x, y). For player 1 a mixed strategy denoted x is a probabil-
ity distribution defined over the set Ω1 := {e11 , . . . , e1j , . . . , e1n1 } of pure strategies,
x : Ω1 → [0, 1]. Let xj denote the probability that player 1 will choose action j
from his n1 available (pure) alternatives in Ω1 , i.e.,
  
Pr χ = e1j = x e1j := xj

where χ denotes random variable of choosing one of the pure strategies in set Ω1 .
Since Ω1 is finite this probability distribution is a vector of probabilities associ-
ated with the pure action. A mixed strategy is identified as the vector composed of
the probabilities associated with available actions, i.e., x = [xj ], j ∈ {1, . . . , n1 },
3.2 Bimatrix Games 31

x ∈ Δ1 , where
  
 n1
Δ1 := x ∈ R n1  xj = 1, xj ≥ 0, ∀j = 1, . . . , n1 (3.1)

j =1

is the unit simplex of (n1 − 1) dimension. Note that pure strategies are just extreme
case of mixed strategies (vertices of the simplex Δ1 ). Similarly, for player 2 we
shall identify his mixed strategy with y = [yk ], k ∈ {1, . . . , n}, y ∈ Δ2 , where yk
denotes the probability that player 2 will choose action k from his n2 available
(pure) alternatives in Ω2 and y ∈ Δ2 ,
  n2 
 
Δ2 := y ∈ Rn2  yk = 1, yk ≥ 0, ∀k = 1, . . . , n2
k=1

Assuming the two players’ strategies to be jointly independent, the proba-

bility of selecting pure-strategy pair (u1 , u2 ) = (e1j , e2k ) ∈ Ω1 × Ω2 is given by
x(u1 ) · y(u2 ) = x(e1j ) · y(e2k ) = xj · yk . In terms of the cost for player 1, the ex-
pected (averaged) cost when a (x, y) mixed-strategy pair is used is given as
     
E J1 (u) = E J1 (u1 , u2 ) = J1 (u1 , u2 )x(u1 )y(u2 )
u1 ∈Ω1 u2 ∈Ω2


n1 
n2

= J1 e1j , e2k xj yk (3.2)
j =1 k=1

and we denote this by J 1 (x, y) or J 1 (x), as the mixed-strategy cost function, when
x ∈ Δ1 , y ∈ Δ2 . Using J1 (e1j , e2k ) = aj k into (3.2) this gives


n1 
n2
J 1 (x) = J 1 (x, y) = aj k xj yk = x T Ay
j =1 k=1

Similarly for player 2


n1 
n2
J 2 (x) = J 2 (x, y) = bj k xj yk = x T By
j =1 k=1

where x, y are n1 , n2 -dimensional vectors.

Remark 3.6 In the case when B = −A, the game is a two-player zero-sum matrix
game
J 1 (x, y) + J 2 (y, x) = 0, ∀x, ∀y
So in this game each player minimizes its cost J 1 and J 2 , but with respect to J
player 1 is the minimizer, while player 2 is regarded as the maximizer of J .
32 3 Matrix Games

Definition 3.7 (Mixed-strategy Nash equilibrium (NE)) For a given (A, B) bimatrix
game let x∗ = (x ∗ , y ∗ ) be pair of mixed strategies chosen by the two players. Then
if for any other mixed strategies (x, y) both

x ∗T Ay ∗ ≤ x T Ay ∗ , ∀x ∈ Δ1
x ∗T By ∗ ≤ x ∗T By, ∀y ∈ Δ2

hold, then x∗ = (x ∗ , y ∗ ) ∈ ΔX is a Nash equilibrium in in mixed strategies for the

bimatrix game.

Computational methods for finding a mixed NE solution strategies for bimatrix

games include the Lemke–Howson algorithm or are based on conversion to a non-
linear programming problem [20].

3.3 m-Player Games

The game formulation can easily be extended from two-player (bimatrix) games
to m-player finite (matrix) games. Let the set of players be denoted by M =
{1, . . . , m}, and assume that ui ∈ Ωi where Ωi is the finite action set.

3.3.1 Pure and Mixed Strategies

Let the strategy of player i, i ∈ M = {1, . . . , m} be denoted by ui (pure strat-

egy) or by xi (mixed strategy). In a matrix case, each player i has a finite num-
ber ni of actions at its disposal. Player i, i ∈ M, uses pure strategy ui by simply
choosing an action j from his ni possible ones in Mi = {1, . . . , ni }, or by ui tak-
ing a value eij from his ni available alternatives in Ωi = {ei1 , . . . , eij , . . . , eini }. Let
u = (u1 , . . . , um ) ∈ Ω denote the pure-strategy (action) profile used by all m play-
ers, where Ω = Ω1 × · · · × Ωm . The cost function is denoted by Ji = Ji (u1 , . . . , um )
corresponding to a cost matrix Ai .
In the case of mixed strategies, the actual action selected is the outcome of some
randomization process. A mixed strategy for player i, i ∈ M, denoted by xi is a
probability distribution on the set Ωi of its pure ui strategies, hence xi (ui ). In this
matrix game case let xi,j denote the probability that player i will choose action
j , ui = eij , from his ni available (pure) alternatives in Ωi . Then a mixed strategy
xi is defined as the vector composed of the probabilities associated with available
actions, i.e., xi (ui ) = [xi,j (eij )], j ∈ Mi , i ∈ M. We sometimes drop the argument
so that xi = [xi,j ], j ∈ Mi , where xi ∈ Δi , and
  ni 

ni 
Δi := xi ∈ R  xi,j = 1, xi,j ≥ 0, ∀j ∈ Mi (3.3)
j =1
3.3 m-Player Games 33

denotes the unit simplex in R ni space, which has dimension ni − 1. The vertices of
Δi are the unit vectors eij . Every mixed strategy xi ∈ Δi is some convex combination
of the unit vectors (pure strategies) eij ∈ Ωi ,


ni
xi = xi,j eij
j =1

and the mixed-strategy simplex Δi is the convex hull of its vertices.

Definition 3.8 For a mixed strategy xi ∈ Δi , we define its support or carrier as the
set of pure strategies that have assigned positive probabilities,

supp(xi ) = {j ∈ Mi | xi,j > 0}

The subset
int(Δi ) = {xi ∈ Δi | xi,j > 0, ∀j ∈ Mi }
is called the interior of Δi . Hence, mixed strategies in the interior xi ∈ int(Δi ) are
called completely mixed or interior, in that they assign positive probabilities to all
player’s pure strategies, hence have full support, i.e., supp(xi ) = Mi .

3.3.2 Mixed-Strategy Cost Functions

A game could be denoted by G(M, Ωi , Ji ) in its pure-strategy representation or

when we refer to its mixed-strategy extension G(M, Δi , J i ). The overall strategy
 
space is Ω = i∈M Ωi indicating a pure-strategy game, or ΔX = i∈M Δi , indi-
cating a mixed-strategy game.
Since the game is non-cooperative, the mixed strategies viewed as probability
distributions are assumed jointly independent, that is the probability of arriving at a
pure-strategy profile (m-tuple) u = (u1 , . . . , um ) ∈ Ω, denoted by x(u) is given as


m
x(u) = xi (ui )
i=1

where x is the m-tuple x = (x1 , . . . , xm ) ∈ ΔX . Thus the probability of using pure-

strategy profile (m-tuple) (k1 , . . . , km ) ∈ M1 × Mm , or u = (e1k1 , . . . , em
km ), is

 m
  m

km =
x e1k1 , . . . , em xi eiki = xi,ki , k i ∈ Mi
i=1 i=1

In terms of pure strategies the cost for player i when the pure-strategy m-tuple u
is used is denoted by Ji (u) ∈ R. If mixed strategies xi , i ∈ M are used according
34 3 Matrix Games

to the distribution x(u) then the cost will be the statistical expectation of Ji with
respect to the probability distribution x(u), namely
  
E Ji (u) = Ji (u)x(u) := J i (x)
u∈Ω

Expanding Ω, u, and x(u) this is written as

  
m
J i (x) = J i (x1 , . . . , xm ) = ··· Ji (u1 , . . . , um ) xl (ul ) (3.4)
u1 ∈Ω1 um ∈Ωm l=1

We often use the following notation: a strategy profile x = (x1 , . . . , xi , . . . , xm ) ∈

ΔX is written as x = (x−i , xi ), where xi ∈ Δi and x−i = (x1 , . . . , xi−1 , xi+1 , . . . , xm )
is the (m − 1)-tuple obtained from x without the ith player component i ∈ M,

x−i ∈ Δ−i where Δ−i = l =i Δl . We denote by (x−i , ui ) the strategy profile where
player i has replaced xi ∈ Δi by its pure strategy ui ∈ Ωi , while all other players
use strategies according to x ∈ ΔX , i.e.,

(x−i , ui ) = (x1 , . . . , xi−1 , ui , xi+1 , . . . , xm )

Using this notation we can, based on (3.4), write

    
m
J i (x−i , ui ) = ··· ··· Ji (u) xl (ul ) (3.5)
u1 ∈Ω1 ui−1 ∈Ωi−1 ui+1 ∈Ωi+1 um ∈Ωm l=1,l =i

We also write (x−i , zi ) ∈ ΔX for the strategy profile in which player i plays mixed
strategy zi ∈ Δi while all other players use strategies according to x ∈ ΔX . This
notation is particularly useful when a single player considers “deviations” zi ∈ Δi
from a given profile x ∈ ΔX .
The definition of an NE equilibrium point in mixed strategies (MSNE) is given
next.

Definition 3.9 (Mixed-strategy NE) Given a non-cooperative m-player finite game

G a mixed-strategy m-tuple x∗ = (x1∗ , . . . , xm
∗ ) ∈ Δ , x ∗ ∈ Δ , i ∈ M is an equilib-
X i i
rium point (or a mixed-strategy Nash equilibrium point if
 
J i x1∗ , . . . , xm
∗
≤ J i x1∗ , . . . , xi−1
∗ ∗
, wi , x(i+1)∗ , . . . , xm , ∀wi ∈ Δi , ∀i ∈ M

or, in compact notation,

 
J i x∗ ≤ J i x∗−i , wi , ∀wi ∈ Δi , ∀i ∈ M (3.6)

where x∗ = (x∗−i , xi∗ ) and x∗−i denotes x∗ of all mixed strategies except the ith one,
i ∈ M.
3.4 Dominance and Best Replies 35

Since there can be more than a single NE we denote this set as NE(G), and x∗ is
an element of this set, x∗ ∈ NE(G). Then, based on Definition 3.9,
   
NE(G) = x∗ ∈ ΔX | J i x∗ ≤ J i x∗−i , wi , ∀i ∈ M, ∀wi ∈ Δi (3.7)

Now, based on the above identification of Ω1 with {e11 , . . . , e1k1 , . . . , e1M1 }, we can
replace u1 ∈ Ω1 by e1k1 for k1 ∈ M1 and we can write

   
m
J i x1∗ , . . . , xm
∗
= ··· Ji (u1 , . . . , um ) ∗
xl,kl
u1 ∈Ω1 um ∈Ωm l=1

  1 
m
∗
= ··· m
Ji ek1 , . . . , ekm xl,kl
k1 ∈M1 km ∈Mm l=1

with the second form explicitly showing the finite matrix form. Using this form and
km ) = Ak1 ,...,km (element in the cost matrix of player i), we
identifying Ji (e1k1 , . . . , em i

see that (3.6) can be re-written as a set of m inequalities, and the ith one, i ∈ M,
∀wi = [xi,j ] ∈ Δi , j ∈ Mi

  
m   
m
∗ ∗
··· Aik1 ,...,km xl,kl
≤ ··· Aik1 ,...,km xl,k x
l i,j
(3.8)
k1 ∈M1 km ∈Mm l=1 k1 ∈M1 km ∈Mm l=1,l =i

We prefer (3.6) as the more compact notation.

3.4 Dominance and Best Replies

One could define partial ordering of a player’s (pure or mixed) strategy set as given
by the consequences on the outcome for that player. We shall define everything
herein in terms of mixed strategies since pure strategies are special cases of these,
so we work on the mixed-strategy simplex Δi of each player i. A strategy weakly
dominates another strategy if it never results in a worse outcome (higher loss) than
the second and sometimes results in lower loss. A strategy is undominated if there is
no strategy that weakly dominates it. A strategy strictly dominates another strategy
if it always results in lower loss.

Definition 3.10 A strategy zi ∈ Δi weakly dominates strategy wi ∈ Δi if ∀x−i ∈

Δ−i the following holds: J i (x−i , zi ) ≤ J i (x−i , wi ), with strict inequality for some
x−i . A strategy wi ∈ Δi is undominated if no such strategy wi exists.

Definition 3.11 A strategy zi ∈ Δi strictly dominates strategy wi ∈ Δi if ∀x−i ∈

Δ−i the following holds: J i (x−i , zi ) < J i (x−i , wi ).
36 3 Matrix Games

One of the basic assumptions in non-cooperative game theory is that players are
rational and do not ever use strictly dominated strategies. This is the reason why
pure dominated pure strategies can be deleted without affecting the game outcome.

3.4.1 Best-Response Correspondence

We review next the concept of best-reply or best-response (BR) correspondence

in mixed strategies. Consider a game G(I, Δi , J i ). Given a mixed-strategy profile
x−i ∈ Δ−i , a mixed-strategy best-reply (-response) correspondence for player i is a
mixed strategy zi ∈ Δi such that no other mixed strategy gives him lower cost (loss)
against x−i . We denote this set by Φi : Δ−i → Δi ,
 
Φi (x−i ) := zi ∈ Δi | J i (x−i , zi ) ≤ J i (x−i , wi ), ∀wi ∈ Δi (3.9)

for any given x−i ∈ Δ−i , also called the optimal response set (or rational reac-
tion set) [20]. The overall mixed-strategy best response of the m-players to strategy
profile x ∈ ΔX is Φ : ΔX → ΔX

Φ(x) = Φ1 (x−1 ), . . . , Φi (x−i ), . . . , Φm (x−m ) ⊂ ΔX

which can be multi-valued, hence a correspondence. Now since every mixed strat-
egy is a convex combination of pure strategies and J i (x−i , zi ) is linear in zi , it
follows that no mixed strategy zi ∈ Δi can give a lower cost against x−i than one
of the pure best replies to x−i of player i. The best-response set Φi (x−i ) is a face
of Δi , i.e., a convex hull of some pure strategies (vertices) in Δi ) and is always
non-empty, closed, and convex.

3.5 Nash Equilibria Theorem

In this section we give a proof for existence of a mixed-strategy Nash equilibria

(NE) for any m-player finite matrix game [95]. Consider an m-player finite game
G(M, Δi , J i ), and let NE(G) ⊂ ΔX denote the set of its NE points, as in (3.7), i.e.,
   
NE(G) = x∗ ∈ ΔX | J i x∗−i , xi∗ ≤ J i x∗−i , wi , ∀wi ∈ Δi , ∀i ∈ M

Based on the best-reply correspondence Φ, (3.9) for any mixed-strategy Nash profile
(equilibrium point) x∗ = (x∗−i , xi∗ ) ∈ NE(G), we can write

xi∗ ∈ Φi x∗−i , ∀i ∈ M

In fact, x∗ ∈ ΔX is an NE solution if xi∗ is a solution to its optimization problem,

given all other players taking equilibrium actions, x∗−i , i.e., xi∗ is player i’s best
3.5 Nash Equilibria Theorem 37

response to all his opponents’ actions, and the same is true for all the players i ∈ M.
Hence
    
NE(G) = x∗ ∈ ΔX | xi∗ ∈ Φi x∗−i , ∀i ∈ M = x∗ ∈ ΔX | x∗ ∈ Φ x∗ (3.10)

which says that in terms of best replies, a mixed-strategy profile x∗ ∈ ΔX is an NE if

it is a best reply to itself, or if it is a fixed point of the mixed-strategy best-reply corre-
spondence Φ. Note that this is not the fixed point of a function but of a multi-valued
(set-valued) mapping. This last interpretation is what leads to one of the proofs for
existence of an NE in any m-player finite game, namely the proof based on Kakutani
fixed-point theorem (and best-response mapping even if not continuous) given later
on.

Definition 3.12 An NE x∗ ∈ ΔX is called a strict NE if each player’s component

xi∗ is the unique best reply to x∗ (singleton) hence if
  
Φ x∗ = x∗

This means that while an NE requires that no unilateral deviation be profitable

(lower cost), a strict NE requires that all such deviations are costly (higher cost).
Thus a strict NE cannot involve any randomization at all, since the there would exist
two pure strategies that give the same lowest cost to some player (over which he
can randomize). As a consequence every strict NE is a pure-strategy profile, i.e., a
vertex in ΔX .
Another property is that an NE strategy xi∗ cannot be strictly dominated, but can
be weakly dominated. As seen from Definition 3.10, there can be another best reply
zi to the NE profile x∗ that is as good as xi∗ against all strategy profiles and that does
better (has lower cost) against some profile.
We now give a proof of Nash equilibrium existence based on Brouwer’s fixed-
point Theorem A.18 (see Appendix A). The proof uses the following preliminary
results.

Lemma 3.13 Let x∗ = (x∗−i , xi∗ ) ∈ ΔX . Then x∗ ∈ NE(G), i.e., it is a mixed-strategy

Nash equilibrium (NE) if and only if for every player i,
 
J i x∗−i , xi∗ ≤ J i x∗−i , eij , ∀j ∈ Mi , ∀i ∈ M (3.11)

Proof (Necessity) If x∗ is an mixed NE point, (3.6) holds for any wi . Since any pure
strategy is a particular case of a mixed strategy, we can take wi = eij which shows
that (3.11) holds.
(Sufficiency) Choose an arbitrary mixed strategy wi ∈ Δi , hence
 
wi = eij αji , for some αji ≥ 0, αji = 1
j ∈Mi j ∈Mi
38 3 Matrix Games

Multiply (3.11) by αji , followed by summing over j ∈ Mi , so that

   ∗ i i
αji J i x∗−i , xi∗ ≤ J i x−i , ej αj
j ∈Mi j ∈Mi

This yields
 i 
J i x∗−i , xi∗ αj ≤ J i x∗−i , wi
j ∈Mi

Since j ∈Mi αji = 1, by (3.9) this implies that

xi∗ ∈ Φi x∗−i

and this holds for ∀i ∈ M hence by (3.10) x∗ is an NE. 

Lemma 3.14 Consider any mixed strategy x = (x−i , xi ) ∈ ΔX , xi ∈ Δi . For every

player i ∈ M there exists a k ∈ Mi such that
 
xi eik = xi,k > 0 and J i (x−i , xi ) ≤ J i x−i , eik

Proof
 We prove  the statement by contradiction. Consider any xi ∈ Δi , xi =
j ∈Mi xi,j ej =
i i
k∈supp(xi ) xi,k ek as a convex combination of pure strategies,

where k∈supp(xi ) xi,k = 1, and supp(xi ) = 0 since for at least one eik we have
xi,k > 0. Assume by contradiction that for all pure strategies k ∈ supp(xi ), i.e., with
xi,k > 0, we have J i (x−i , xi ) > J i (x−i , eik ). Hence for all such strategies,

J i (x)xi,k > J i x−i , eik xi,k

Summing the foregoing over k yields

  
J i (x)xi,k > J i x−i , eik xi,k
k∈supp(xi ) k∈supp(xi )

Then factoring J i (x) on the LHS and using the representation of xi gives

J i (x) > J i (x−i , xi ) = J i (x)

hence a contradiction. 

Theorem 3.15 (Existence of mixed-strategy NE [95]) Every finite (static) m-player

non-cooperative game G has at least one mixed-strategy equilibrium point, called
mixed-strategy Nash equilibrium (NE).

Proof If player i has ni pure strategies, then the set of his mixed strategies can be
represented geometrically by an (ni − 1)-dimensional simplex, Δi . Each Δi is a
convex, closed, and bounded (hence compact) subset of the R ni Euclidean space.
3.5 Nash Equilibria Theorem 39

Hence the overall Δ = Δ1 × · · · × Δm is a convex, closed, and bounded (hence

compact) subset of the R n , where n = m i=1 ni Euclidean space, and any mixed-
strategy m-tuple x = (x1 , . . . , xm ) can be viewed as a point x ∈ ΔX . For any such
x ∈ ΔX and any pure strategy ui = eij ∈ Ωi for player i we define
  
Cji (x) = max J i (x) − J i x−i , eij , 0

and we see that Cji is a function that measures the decrease in the cost for player i
when in the mixed-strategy m-tuple (profile) x, player i replaces strategy xi by its
pure strategy ui = eij . We also see that Cji (x) ≥ 0. Any possible increase in cost is
not indicated since in this case the function vanishes.
Now for each i ∈ M, j ∈ Mi we consider the transformation

xi (eij ) + Cji (x)

ηji := 
1 + j ∈Mi Cji (x)

where xi (eij ) = xi,j . Each of these is non-negative also. Note that, since xi ∈ Δi ,
  
xi eij = xi,j = 1
j ∈Mi j ∈Mi

and summing over all j ∈ Mi (or ui ∈ Ωi ) we have

 xi (eij ) + Cji (x)

 =1
j ∈Mi
1 + j ∈Mi Cji (x)

Hence j ∈Mi ηji = 1 and for fixed x and i this ηji can be taken a probability attached
to the strategy ui,j , and ηi ∈ Δi itself.
Moreover, since J i (x) is polynomial in xi,j , Cji (x) is a continuous function of x.

Since 1 + j ∈Mi Cji (x) = 0, it follows that ηi is itself a continuous function of x.
Thus gathering all in a m-tuple (vector), we see that η = [η1 , . . . , ηm ]T is a contin-
uous vector-valued function of x that we denote by η = T (x), that maps the convex
closed bounded set ΔX into itself. Then by Brouwer’s fixed-point Theorem A.18
(see Appendix A) it follows that there exists a fixed point for T , hence

T (x̃) = x̃

The theorem is proved if we can show that every such fixed point is necessarily a
Nash equilibrium and vice versa that every Nash equilibrium is a fixed point of this
mapping. Since x̃ = (x̃ 1 , . . . , x̃ m ) is a fixed point for T it follows that for all i ∈ M
and for all l ∈ Mi
 x̃ i (eil ) + Cli (x̃)
x̃ i eil :=  (3.12)
1 + j ∈Mi Cji (x̃)
40 3 Matrix Games

For this x̃ ∈ ΔX , by Lemma 3.14 and the definition of Cji it follows that for any
player i there exist a k and a pure strategy eik such that x̃ i (eik ) = x̃i,k > 0
 
x̃ i eik > 0 and J i (x̃) − J i x̃−i , eik ≤ 0

hence

x̃ i eik > 0 and Cki (x̃) = 0
For this particular strategy (3.12) becomes

 x̃ i (ei )
x̃ i eik :=  k i
1 + j ∈Mi Cj (x̃)

so that
 i
x̃ i eik Cj (x̃) = 0
j ∈Mi

which since x̃ i (eik ) > 0 implies j ∈Mi Cji (x̃) = 0. Since all Cji ≥ 0 the foregoing
implies
Cji (x̃) = 0, ∀i ∈ M, ∀j ∈ Mi

and from the definition of Cji this means that for each player i we have

J i (x̃) ≤ J i x̃−i , eij , ∀i ∈ M, ∀j ∈ Mi

By Lemma 3.14, x̃ is a NE equilibrium point and proof is complete. 

An alternative proof based on Kakutani’s fixed-point theorem is the variant that

John Nash used in his description of Nash equilibrium for m-player games, work
that would later earn him a Nobel Prize in Economics, i.e., that every finite game
with mixed strategies for any number of players has a Nash equilibrium. Kakutani’s
fixed-point theorem is a generalization of Brouwer’s fixed-point theorem for set-
valued maps or correspondences that requires it to be hemi-continuous, or that it
has a closed graph. Unlike the previous proof where we constructed a single-valued
function C i , here we can simply use the best-response map (correspondence) Φi
and Φ. We give this proof for completeness.

Proof (based on Kakutani’s fixed-point theorem) Recall that Φi (x−i ) is the set of
tuples of mixed strategies chosen by each player in a game, and Φ(x) is the overall
best response. Since there may be a number of responses which are equally good,
Φi (x−i ) is set-valued rather than single-valued. Then the Nash equilibrium of the
game is defined as a fixed point of Φ.
Note that ΔX is a non-empty, compact, and convex set. The best-reply correspon-
dence (mapping) Φ : ΔX → ΔX is upper hemi-continuous by the continuity of the
payoff function J i , which in itself follows form Berge’s maximum Theorem A.17
3.6 Nash Equilibria Refinements 41

(see Appendix A). For every x ∈ ΔX , the image Φ(x) ⊂ ΔX is a non-empty, closed,
and convex set. Hence, by Kakutani’s fixed-point Theorem A.19, Appendix A, Φ
has at least one fixed point in ΔX , i.e., there exists x∗ ∈ ΔX such that x∗ ∈ Φ(x∗ ). 

3.6 Nash Equilibria Refinements

In this section we review some NE refinements. In order to go beyond certain weak-

ness of the Nash equilibrium when several of these exist, a large range of refinements
have been proposed since about the late 1970. We shall review only a few of these
for normal-form game only.
Let NE(G) ⊂ ΔX denote the set of NE points of a m-player finite game
G(I, Δi , J i ).

3.6.1 Perfect Equilibrium

This refinement is due to Selten [136], best known as the “trembling hand perfec-
tion”. This intuitively means that NEs that are not robust to “trembles” in players’
strategies are discarded.
Consider a game G(M, Δi , J i ) and let μ be an error function such that a number
μi,k ∈ (0, 1) is assigned for each player i ∈ M, and pure strategy k ∈ Mi which
defines the probability
 that this k pure strategy will be played by mistake (trembling
hand), with k ∈ Mi μi,k < 1. Note that such a small probability > 0 is assigned to
all pure strategies.
This means that for each player i ∈ M the error function μ defines a subset of
mixed strategies

Δi (μ) := {xi ∈ Δi | xi,k ≥ μi,k } ⊂ int(Δi )

that player i can use. Define a perturbed game G(μ) = G(M, Δi (μ), J i ), where

ΔX (μ) = i∈M Δi (μ) ⊂ int(ΔX ). By a similar argument based on Kakutani’s
frixed-point theorem, every such perturbed game has an non-empty set NE(G(μ))
of NEs. The smaller all error probabilities are, the larger ΔX (μ) is and as μ → 0
the perturbed game approaches the original game.

Definition 3.16 An NE, x∗ ∈ NE(G) is a perfect equilibrium if, for some sequence
{G(μt )}μt →0 of perturbed games, there exist profiles xt ∈ NE(G(μt )) such that
xt → x ∗ .

Let us call PE(G) the set of perfect equilibria. Note that every interior (com-
pletely mixed) or inner NE is perfect. This is seen because if x∗ ∈ int(ΔX ),
then for sufficiently small μi,k , x∗ ∈ int(ΔX (μ)). If in addition x∗ ∈ NE(G) then
42 3 Matrix Games

x∗ ∈ NE(G(μ)). The definition above requires only that the NE be robust with re-
spect to some trembles (error function), so existence can be established even if there
are no interior NE.

Proposition 3.17 For any finite game, PE(G) = ∅.

Proof For any sequence {G(μt )}μt →0 let xt ∈ NE(G(μt )) for each t. Since {xt },
t = 1, . . . , ∞ is a sequence in a compact set ΔX it has a convergent subsequence,
{xts }, s = 1, . . . , ∞ with limit x∗ ∈ ΔX . For each s, G(μts ) is the associated per-
turbed game. Then since xts ∈ NE(G(μts )) it follows that x∗ ∈ NE(G) by continuity
arguments, and x∗ is perfect since xts → x∗ and xts ∈ NE(G(μts )) for all s. 

We give without proof the following useful result (for a proof see [157]).

Proposition 3.18 Every perfect equilibrium x ∈ PE(G) is undominated, i.e., every

player’s strategy component xi is undominated, i ∈ M. In a two-player game M =
{1, 2} if an NE x ∈ NE(G) is undominated, then it is perfect, i.e., x ∈ PE(G).

As a most stringent condition on the “trembles”, the concept of strictly perfect

equilibrium requires robustness with respect to any low probability tremble.

Definition 3.19 An NE, x∗ ∈ NE(G) is a strictly perfect equilibrium if, for every
sequence {G(μt )}μt →0 of perturbed games, there exist profiles xt ∈ NE(G(μt )) such
that xt → x∗ .

Note that any interior (inner) NE equilibrium x ∈ NE(G) is strictly proper, since
one could take xt = x∗ for all t sufficiently large such that x∗ ∈ NE(G(μt )). In
fact a perfect NE equilibrium that is not strictly perfect is vulnerable against some
sequence of trembles.

3.6.2 Proper Equilibrium

An intermediary refinement is one introduced by Myerson [93] which imposes some

conditions on the “trembles” to which the NE should be robust (not only to some
unqualified trembles). Specifically, an NE equilibrium should be robust with respect
to those trembles that are such that more costly “errors” are less probable. This is
like requesting the perturbed (“trembled”) strategy to be “proper” is some sense,
formalized as follows. Given some
> 0, a strategy profile x ∈ int(ΔX ) is called

-proper if
 
J i x−i , eik > J i x−i , eij =⇒ xi,k ≤
xi,j
3.7 Notes 43

3.6.3 Strategically Stable Equilibrium

The concept of a strictly perfect NE can be extended to robustness of NE sets, called

strategically stable sets.

Definition 3.20 A set of NEs, X ∗ ⊂ NE(G) is strategically stable if it is the smallest

non-empty, closed set such that for every
> 0 there exists some δ > 0 such that
every strategy perturbed game G(μ) = (M, ΔX (μ), J i ) with errors μi,k < δ has
some NE within distance
from the set X ∗ .

3.7 Notes

In this chapter we reviewed basic concepts and results for matrix games, i.e., games
with finite action sets. In the next chapter we consider games with infinite (continu-
ous) action sets.
Chapter 4
Games with Continuous Action Spaces

Abstract This chapter focuses on Nash games with continuous kernel, i.e., games
with continuous action spaces and cost functions. Basic concepts and results are
reviewed focused on game formulations, Nash equilibria, reaction curves, and exis-
tence results. These are mostly adapted from (Basar and Olsder in Dynamic Nonco-
operative Game Theory. SIAM Series Classics in Applied Mathematics, 2nd edn.,
1999). For more in-depth treatment the reader is referred to this and other more
extensive references on the subject.

4.1 Introduction

In this chapter we consider games with infinite action spaces (hence continuous)
and with players’ payoff (cost) functions as continuous in the actions. We mostly
treat pure strategies (actions). We introduce basic NE concepts and existence re-
sults for the case in which no coupling exists between those action spaces, results
mostly adapted from [20, 98]. For more in-depth treatment the reader is referred to
those and other more extensive references on the subject. A review of the associated
optimization results is presented in one of the appendices.
The chapter is organized as follows. We present basic concepts and game formu-
lation for games with continuous action spaces and uncoupled constraints, and then
we present fundamental existence and uniqueness results for the Nash equilibrium.

4.2 Game Formulations

A game consists of a set of players, an action set action set (also referred to as a set
of strategies) available to those players and an individual objective function for each
player. In a game, each player individually takes an optimal action which optimizes
its own objective function and each player’s success in making decisions depends
on the decisions of the others.
Let us consider games with continuous action spaces and pure strategies. Specif-
ically, as the focus of this chapter we consider games with continuous action spaces
and uncoupled constraints. The following chapter will treat such games, but with

L. Pavel, Game Theory for Control of Optical Networks, 45

Static & Dynamic Game Theory: Foundations & Applications,
DOI 10.1007/978-0-8176-8322-1_4, © Springer Science+Business Media New York 2012
46 4 Games with Continuous Action Spaces

coupled constraints. Consider Ωi = [mi , Mi ], where mi > 0 and Mi > mi are

scalars. Each player i ∈ M has an action variable ui ∈ Ωi and such a game is
called a game with orthogonal (uncoupled constraints), as before the overall action
space Ω being defined by the Cartesian product of Ωi

Ω = Ω1 × · · · × Ωm (4.1)

We also let Ω−i := Ω1 × · · · × Ωi−1 × Ωi+1 × · · · × Ωm . It can be seen that Ω

is compact, convex, and has a nonempty interior set. Moreover, the action space Ω
is separable by construction since there is no coupling among Ωi . Any two players
i and j can take their actions independently from separate action sets Ωi and Ωj ,
j = i, respectively. An action vector u ∈ Ω can be written as u = [u1 , . . . , um ]T , or
u = (u−i , ui ) with the vector u−i ∈ Ω−i obtained by deleting the ith element from
u, i.e., u−i = [u1 , . . . , ui−1 , ui+1 , . . . , um ]T .
Given a game’s specification G(I, Ωi , Ji ) an important issue is to predict how the
game will be played, i.e., to determine its outcome. These predictions are called so-
lutions, and describe which strategies will be adopted by players, therefore predict-
ing the result of the game. The outcome of a game is referred to as its equilibrium,
or solution, and it is given by a point u∗ ∈ Ω. This solution corresponds to a set of
strategies (actions) that the players will adopt according to their knowledge of the
game, and of other players’ actions and intentions. We shall discuss next the Nash
equilibrium [96] in the context of such games. Recall that in brief Nash equilibrium
(NE) is a solution concept of a game in which each player is assumed to know the
equilibrium strategies of the other players, and no player has anything to gain by
changing its own strategy unilaterally.
Before we discuss the NE concept we shall briefly remark on the mixed-strategy
extension for game with continuous action spaces.

4.3 Extension to Mixed Strategies

Similarly to the concept of a mixed strategy for matrix games, this concept can be
defined for continuous (or infinite) games by employing the appropriate distribution
function. We only consider for simplicity the N = 2 case. Let the two players’ pure
strategies be denoted u1 ∈ Ω1 , u2 ∈ Ω2 , where this time Ωi are infinite set such as
for example the interval [0, 1].
A mixed strategy for player 1 is a cumulative distribution function σ1 defined
on Ω1 : for every u1 ∈ Ω1 , σ1 (u1 ) is a random process for choosing a number not
greater than u1 . That is,
σ1 (u1 ) = Pr (ξ ≤ u1 )
where Pr denote the probability and ξ the random variable of the process.
(i) σ1 (u1 ) ≥ 0, ∀u1 ∈ Ω1 , (ii) σ1 (0) = 0, σ (1) = 0, (iii) σ1 (u1 ) is non-decreasing,
(iv) σ1 (u1 ) is right-continuous in the open (0, 1). Similarly σ2 (u2 ) is the mixed
4.4 Nash Equilibria and Reaction Curves 47

strategy of player 2 having similar properties. If player 1 chooses pure strategy u1

and player 2 chooses mixed strategy σ2 (u2 ) the expected cost for player 1 is


J 1 (u1 , σ2 ) = E J1 (u1 , σ2 ) = J1 (u1 , u2 ) dσ2 (u2 )
Ω2

where the integral is Lebesgue–Stieljes integral. Similarly if player 2 chooses pure

strategy u2 and player 1 chooses mixed strategy σ1 (u1 ) the expected cost for
player 1 is


J 1 (σ1 , u2 ) = E J1 (σ1 , u2 ) = J1 (u1 , u2 ) dσ1 (u1 )
Ω1

Now if player 1 and 2 use mixed strategies σ1 (u1 ) and σ2 (u2 ), then joint probability
distributions have to be considered, and assuming them independent, the expected
cost for player 1 is
 
J 1 (σ1 , σ2 ) = J (u1 , u2 ) dσ1 (u1 ) dσ2 (u2 )
Ω1 Ω2

and similarly the expected cost for player 2. Then definition for a Nash equilibrium
can be given in the mixed-strategy case using the set of distribution functions in
terms of these expected costs.

4.4 Nash Equilibria and Reaction Curves

Return to the basic case of games with continuous action spaces in the pure-strategy
case, G(I, Ωi , Ji ). Let us define an NE solution for this case. In such a game an
individual cost function Ji : Ω → R is assigned to each player i, Ji (u), and each
player aims to minimize its own cost function Ji (ui , u−i ), in the presence of all
other players.
Then a pure-strategy Nash equilibrium (NE), or simply a Nash equilibrium (NE)
of this noncooperative game is the defined as a point u∗ ∈ Ω that satisfies
 
Ji u∗ ≤ Ji u∗−i , ui , ∀ui ∈ Ωi , ∀i ∈ M

or, alternatively given the actions of all other players, u−i ∈ Ω−i , each player i
independently minimizes its own cost:

min Ji (u−i , ui )
(4.2)
subject to ui ∈ Ωi

The formal definition of a Nash equilibrium (NE) is given next.

48 4 Games with Continuous Action Spaces

Definition 4.1 (Nash equilibrium (NE)) A vector u∗ ∈ Ω is called a Nash equilib-

rium (NE) solution of G(M, Ωi , Ji ) if for all i ∈ M and for every u∗−i ∈ Ω−i ,
 
Ji u∗−i , u∗i ≤ Ji u∗−i , ui , ∀ui ∈ Ωi (4.3)

If in addition u∗ is not on the boundary of the action space Ω, it is called an inner

NE solution.

As we remarked before, at an NE no player can benefit by altering its action uni-

laterally. In other words, u∗ is an NE if given the action u∗−i taken by his opponents,
no player has an incentive to choose a different action on his own, i.e. an NE is a
so-called no-regret strategy.
For a given u−i ∈ Ω−i , player i takes an action ξ ∈ Ωi such that

Ji (u−i , ξ ) ≤ Ji (u−i , ui ), ∀ui ∈ Ωi

Then for a game G(M, Ωi , Ji ), we may define a best response correspondence

 
Ri (u−i ) = ξ ∈ Ωi | Ji (u−i , ξ ) ≤ Ji (u−i , ui ), ∀ui ∈ Ωi (4.4)

which is called the optimal reaction set of player i ∈ M, i.e., the set of all optimal re-
sponses by player i to any fixed u−i . The concepts of best response correspondence
and the NE concept, defined here for pure-strategy game G(M, Ωi , Ji ), correspond
to those reviewed in Chap. 3 for a mixed-strategy matrix game G(M, Δi , J i ).
Note that u∗ ∈ Ω is an NE solution if u∗i is a solution to the optimization problem
(4.2), given all other players taking equilibrium actions, u∗−i , i.e., u∗i is player i’s
best response to all his opponents’ actions, and the same is true for all the players
i ∈ M. Based on this we define next the reaction function of a player.

Definition 4.2 (Reaction function) If the optimal reaction set Ri (u−i ) (4.4) is a
singleton (i.e., a set with exactly one element) for every given u−i ∈ Ω−i , then
Ri (u−i ) is called the reaction function of player i, and is specifically denoted by
ri : Ω−i → Ωi .

We present next properties of reaction functions of G(M, Ωi , Ji ) under a typical

assumption.

Assumption 4.3 The cost function Ji (u−i , ui ) is twice continuously differentiable

in all its arguments and strictly convex in ui for every u−i ∈ Ω−i .

Proposition 4.4 Consider G(M, Ωi , Ji ) under Assumption 4.3. For each i ∈ M, a

unique continuous reaction function ri : Ω−i → Ωi exists.

Proof Under Assumption 4.3, the cost function Ji (u−i , ui ) is strictly convex in ui .
By Proposition A.2, there exists a unique minimum ui , for any given u−i , such that

Ji (u−i , ui ) < Ji (u−i , ξ ), ∀ξ ∈ Ωi , ξ = ui

4.5 Existence and Uniqueness Results 49

By Definition 4.2, this implies that a unique mapping, i.e., a reaction function ri :
Ω−i → Ωi exists for each player i. The continuity can be obtained directly from
Berge’s Maximum Theorem (Theorem A.17). 

4.5 Existence and Uniqueness Results

An important question is whether a game admits a (possible unique) Nash equilib-

rium (NE), and conversely to characterize classes of games for which there exists a
unique NE. When equilibria do exist, they may not be unique and in such cases the
NE is said to be weak as a solution concept.
From the definition of an NE (Definition 4.1) and the definition of the best re-
sponse (reaction function), (4.4), it follows that for each player an NE (if it exists)
must lie on his own reaction curve, i.e. u∗i = ri (u∗−i ). Therefore in graphical terms,
the existence of a Nash equilibrium (NE) is equivalent to requiring that a plot of the
graph of each ri (u−i ) ∈ Ω results in at least one common intersection point. Let us
denote the set of all NE by NE(G). Then the set of all such intersections in Ω forms
NE(G). We are thus led to another important characterization of NE(G) as the set of
fixed points of the vector function r : R m → R m , i.e.

u∗ = r u∗ (4.5)

where r(u) = [r1 (u−1 ), . . . , ri (u−i ), . . . rm (u−m )]T .

Intuitively, this implies that an NE is a strategy profile that constitutes the best
reply to itself. When u∗ is the unique best reply to itself we say that u∗ is a strict
NE. In that case it will be true that
 
Ji u∗ < Ji u∗−i , ui , ∀ui ∈ Ωi , ∀i ∈ M

and the set NE(G) is a singleton.

Provided that reaction functions are well defined and a common intersection
point of the reaction functions exists, the existence of an NE solution can be estab-
lished. This reasoning leads to the following proposition (adapted from Theorem 4.3
in [20]).

Proposition 4.5 Under Assumption 4.3, G(M, Ωi , Ji ) admits an NE solution.

Proof By strict convexity of the cost function and Proposition 4.4, for every given
u−i ∈ Ω−i , there exists a unique reaction function ri of player i ∈ M such that

ui = ri (u−i ) ∈ Ωi , ∀i ∈ M.

The corresponding vector form is u = r(u) ∈ Ω with r(u) = [r1 (u−1 ), . . . ,

rm (u−m )]T . Furthermore, by Proposition 4.4, ri is continuous, and so is r(u). On
50 4 Games with Continuous Action Spaces

the other hand, r(u) maps a compact and convex set Ω into itself. From (Theo-
rem A.18), there exists a u∗ such that u∗ = r(u∗ ). By Definition 4.1, it follows that
u∗ is an NE solution of G(M, Ωi , Ji ). 

As to the property of an NE solution of being inner, the following result is im-

mediate from Definition 4.1 and Proposition A.1.

Proposition 4.6 Under Assumption 4.3, if a vector u∗ is an inner NE solution of

G(M, Ωi , Ji ), then u = u∗ satisfies the following set of necessary conditions:

∂Ji
(u−i , ui ) = 0, ∀i ∈ M (4.6)
∂ui

Now let us define an associated optimization problem from the cost functions Ji ,
i.e.,


m
min Jj (u)
j =1
subject to u∈Ω

Then in general an NE solution is not an optimal solution to this associated optimiza-

tion problem. It is well-known that in most cases Nash equilibria do not optimize
the system performance with the Prisoner’s Dilemma [98] being the best-known
example. Let us take a look at a two-player Nash game for illustration.

Example 4.7 Consider a two-player Nash game with the cost functions

J1 (u2 , u1 ) = 2u21 − 2u1 − u1 u2

1
J2 (u1 , u2 ) = u22 − u2 − u1 u2
2
and where ui ∈ Ωi = [0, 8], i = 1, 2.
The constraint set Ω = Ω1 × Ω2 is convex and compact and has a nonempty
2 2
interior set. From the cost functions, it follows that ∂ J21 = 4 and ∂ J22 = 2. Thus As-
∂u1 ∂u2
sumption 4.3 is satisfied. From Proposition 4.4, the reaction functions ri , i = 1, 2,
exist and are continuous. By Definition 4.2, the reaction function r1 (u2 ) can be
obtained by optimizing J1 (u2 , u1 ) with respect to u1 for every given u2 . It fol-
lows that r1 (u2 ) = 14 (u2 + 2). Similarly, the reaction function r2 (u1 ) is obtained as
r2 (u1 ) = 12 (u1 + 12 ). The reaction curves are shown in Fig. 4.1. By Definition 4.1, an
NE solution lies on both reaction curves. Therefore the intersection point of r1 (u2 )
and r2 (u1 ), (u∗1 , u∗2 ) = ( 14
9 4
, 7 ), is an NE solution. The corresponding optimal cost
values are J1 = − 98 and J2∗ = − 16
∗ 81
49 . Figure 4.1 indicates that this game admits a
unique NE solution.
4.5 Existence and Uniqueness Results 51

Fig. 4.1 Reaction curves,

r1 (u2 ) and r2 (u1 )

Next consider the associated optimization problem

min J0 (u1 , u2 )
subject to 0 ≤ ui ≤ 8, i = 1, 2,

where J0 (u1 , u2 ) = J1 (u2 , u1 ) + J2 (u1 , u2 ) = 2u21 + u22 − 2u1 − 12 u2 − 2u1 u2 . The

constraint set Ω is still convex and compact. The Hessian of J0 is positive definite
for every u ∈ Ω with
 
4 −2
∇ J0 =
2
−2 2
such that the cost function J0 is strictly convex over Ω. From Proposition A.2 the
associated optimization problem admits a unique global minimum. By using Propo-
opt
sition A.3, the optimal solution is uopt = ( 54 , 32 ) and the optimal value is J0 = − 13
8 .
It is clear that u∗ = uopt . The following is a further discussion. In the game frame-
work, given uopt , the cost values are J1 (uopt ) = − 54 , which is less than J1 (u∗ ) and
J2 (uopt ) = − 38 which is less than J2 (u∗ ). It follows that in the presence of selfish-
ness, the NE solution is not necessarily the optimal solution from a social point of
view.

From the individual cost functions Ji (u−i , ui ) let us define now an augmented
system-like two-argument cost function J˜ : Ω × Ω → R as in [20]


m
J˜(u; x) := Ji (u−i , xi ), (4.7)
i=1

where each Ji (u−i , ui ) satisfies Assumption 4.3. We shall call this two-argument
function J˜ the Nash game (NG) cost function of the game. Based on it another
52 4 Games with Continuous Action Spaces

useful characterization of an NE can be derived as follows. We provide an alternate

proof of Proposition 4.5 by using the NG cost function. In addition we show how
necessary conditions for an inner NE solution of G(M, Ωi , Ji ) can be reformulated
with respect to the NG cost function. As we will show in the next chapter this NG
cost function can be used to solve Nash games with coupled constraints.

Definition 4.8 (NE-NG sense) A vector u∗ ∈ Ω satisfying

 
J˜ u∗ ; u∗ ≤ J˜ u∗ ; x , ∀x ∈ Ω (4.8)

with J˜ defined in (4.7), is called an NE solution of G(M, Ωi , Ji ).

Remark 4.9 The two definitions, Definitions 4.1 and 4.8 are equivalent. Indeed (4.8)
can be equivalently rewritten as for every given u∗−i ,


m
 m

Ji u∗−i , u∗i ≤ Ji u∗−i , xi , ∀x ∈ Ω
i=1 i=1

Thus it is immediately seen that u∗ satisfies Definition 4.8 if u∗ is an NE solu-

tion in the sense of Definition 4.1. Conversely, if u∗ satisfies Definition 4.8, then it
constitutes an NE solution in the sense of Definition 4.1 as shown next by contra-
diction. Assume to the contrary that such a u∗ is not an NE solution in the sense
∗
of Definition 4.1. This implies that for some  i ∈ M, there exists a ũi = ui such
that Ji (u∗−i , ũi ) < Ji (u∗−i , u∗i ). By adding j ∈M,j =i Jj (u∗−j , u∗j ) to both sides, the
following inequality holds:
 
J˜ u∗ ; ũ < J˜ u∗ ; u∗ ,

where ũ := (u∗−i , ũi ) and ũi = u∗i . This contradicts the hypothesis (4.8).

Based on the concept of NG cost function, (4.7), we give next a proof of Propo-
sition 4.5. The proof essentially follows arguments similar to those used in The-
orem 4.4 in [20]. We present it here to give readers a better interpretation of the
two-argument NG cost function, because this will prove a very useful analytical
vehicle in later chapters for games with coupled constraints (Chaps. 5 and 7). The
proof helps reformulate the necessary conditions (4.6) with respect to the NG cost
function.

Proof of Proposition 4.5 From (4.7), the two-argument NG cost function J˜(u; x)
is separable in the second argument x for every given u, i.e., each component cost
function in J˜(u; x) is decoupled in x for every given u. Therefore by using (4.7), for
every given u, the gradient of J˜(u; x) with respect to x is written as
⎡ ⎤
∂J1
∂x1 (u−1 , x1 )
⎢ ⎥
∇x J˜(u; x) := ⎢
⎣
..
.
⎥
⎦ (4.9)
∂Jm
∂xm (u−m , xm )
4.5 Existence and Uniqueness Results 53

The Hessian of J˜(u; x) with respect to x, ∇xx

2 J˜(u; x), is a diagonal matrix with
∂ 2 Ji
the diagonal elements being (u−i , xi ), i = 1, . . . , m. Under Assumption 4.3,
∂xi2
from the strict convexity of Ji (u−i , xi ) with respect to xi (for every given u−i ), it
follows that J˜(u; x) is strictly convex with respect to its second argument x, for
every given u. Moreover, J˜(u; x) is continuous in its arguments. Let us define a
reaction set:
 
R(u) = x ∈ Ω | J˜(u; x) ≤ J˜(u, w), ∀w ∈ Ω ,

or, equivalently,

R(u) = arg minJ˜(u; x),

x∈Ω

where minimization on the right-hand side is done with respect to the second argu-
ment x in J˜(u; x). Recall that Ω is compact. By the continuity and convexity prop-
erty of J˜, it follows from Theorem A.17 (Berge’s Maximum Theorem) that R is an
upper-semi-continuous mapping that maps each point u in Ω into a compact and
convex subset of Ω. Then by the Kakutani Fixed-Point Theorem (Theorem A.19),
there exists a point u∗ such that u∗ ∈ R(u∗ ), i.e., u∗ satisfies (4.8). By Definition 4.8,
u∗ is an NE solution of G(M, Ωi , Ji ). 

Thus as seen in the proof, an NE solution u∗ is a fixed-point solution that satisfies

u∗ ∈ R(u∗ ), i.e., u = u∗ is a solution to the implicit equation

u = arg min J˜(u; x) (4.10)

x∈Ω

Moreover, if u∗ is inner, then u = u∗ equivalently satisfies

∇x J˜(u; x)|x=u = 0, (4.11)

where the notation “|x=u ” denotes finding a fixed-point solution. By using (4.9), we
get the component-wise form of (4.11):

∂Ji 
(u−i , xi ) = 0, ∀i ∈ M, (4.12)
∂xi xi =ui

which are equivalent to the necessary conditions (4.6) with respect to Ji presented
in Proposition 4.6.
Based on this one can summarize the procedure to find an inner NE solution with
respect to J˜ as follows. As a first step solve ∇x J˜(u; x) = 0 for every given u (i.e.,
x is the only variable), which gives x as a function of u. Then look for a fixed-
point solution for these equations, this fixed-point solution being denoted by x = u.
Solving the resulting set of m equations, (4.12), leads to an inner NE solution. We
give next an illustration of this procedure.
54 4 Games with Continuous Action Spaces

Example 4.10 (Consider the two-player Nash game in Example 4.7 and its NG cost
function)

J˜(u; x) = J1 (u2 , x1 ) + J2 (u1 , x2 )

 
 1
= 2x12 − 2x1 − x1 u2 + x22 − x2 − u1 x2 (4.13)
2

Assume there exists an inner NE solution. Then given every u, in the first step we
solve ∇x J˜(u; x) = 0, i.e.,
∂J1
(u2 , x1 ) = 4x1 − 2 − u2 = 0
∂x1
∂J2 1
(u1 , x2 ) = 2x2 − − u1 = 0
∂x2 2
This leads to x as a function of u, i.e.,
 1
 1
0 4 2
x= u+
1 1
2 0 4

In the second step we solve for a fixed-point solution, by setting x = u in the above,
which leads to (u∗1 , u∗2 ) = ( 14
9 4
, 7 ).

While existence can be guaranteed under relatively standard conditions, unique-

ness of an NE depends on the particular situation and general results are hard to
arrive at. In games where an NE is not unique and NE(G) is not a singleton, one
may seek various refinements of the NE concept as described in the previous chap-
ter. These refinements could include testing an equilibrium for additional attributes
such as stability or robustness, among others. The reader can consult 4.3 in [20] for
such definitions. Besides refinements such as stability and robustness, sometimes it
is possible to partially order the set NE(G) by Pareto dominance. However, since
the best one can do is only partially order the set NE(G), we cannot expect in gen-
eral that even Pareto dominance will produce a unique solution. Work such as [54]
attempts to address the problem of equilibrium selection.

4.6 Notes
This chapter provided an overview of game with continuous action spaces and cost
functions (continuous kernels) and uncoupled constraints. A review of optimization
results is presented in the appendix.
Chapter 5
Computational Results for Games
with Coupled Constraints

Abstract This chapter provides some results for Nash games with coupled con-
straints, i.e., coupled action sets. Work on games with coupled action spaces has
been going on for more than 50 years. These are also called generalized Nash games,
games with coupled constraints, or social equilibria. Game theoretical formulations
of problems and computational approaches towards solving coupled or generalized
Nash games have been areas of much recent interest. We present some new results
mainly based on the Lagrangian approach extension proposed in Pavel (Automatica
43(2):226–237, 2007). We review a relaxation via an augmented optimization, the
Lagrangian extension in a game setup, followed by duality and hierarchical decom-
position in a game setup.

5.1 Introduction

As seen in the previous chapter, in Nash games with uncoupled constraints, the
action space is the Cartesian product of the individual action sets and players can
affect only the cost functions of the other players but not their feasible action sets.
On the other hand, in Nash games with coupled constraints, each player’s action
affects the feasible action sets of the other players. In Example 4.7, the action sets
are Ωi = [0, 8], i = 1, 2, such that the action space Ω is rectangular in R2 .
Now consider the following example with a modified action space.

Example 5.1 Consider the two-player Nash game in Example 4.7 with an action
space, Ω := Ω1 × Ω2 and ui ∈ Ωi = [0, 8], i = 1, 2. An additional constraint is
considered: u1 + u2 ≤ 8. Now the action space is modified to be

Ω = {u ∈ Ω | u1 + u2 − 8 ≤ 0}.

Figure 5.1 shows that Ω is rectangular (constraints have no coupling), while Ω is

triangular (constraints are coupled). In this latter case Ω, it is not possible to obtain
separate action sets from which the players can take actions independently, i.e.,

L. Pavel, Game Theory for Control of Optical Networks, 55

Static & Dynamic Game Theory: Foundations & Applications,
DOI 10.1007/978-0-8176-8322-1_5, © Springer Science+Business Media New York 2012
56 5 Computational Results for Games with Coupled Constraints

Fig. 5.1 Constraints:

rectangular and triangular

u1 ∈ [0, 8 − u2 ] and u2 ∈ [0, 8 − u1 ]. That is the action space Ω is coupled and this
game is called a two-player Nash game with coupled constraints.

Starting from this example, we present in this chapter some theoretically results
for Nash games with coupled constraints, i.e., coupled action sets. We call such
a game a coupled Nash game. Coupled Nash games are also called generalized
Nash games (games with non-disjoint strategy sets) [56], games with coupled con-
straints [128], social equilibria games or pseudo-Nash equilibria games [16, 37].
Game theoretical formulations of problems and computational approaches towards
solving coupled or generalized Nash games have been areas of much recent interest
[15, 40, 90, 91, 120, 135, 138]. The work in [40] shows that a generalized Nash
equilibrium can be calculated by solving a variational inequality and the results
express conditions in terms of variational inequality problem and Karush–Kuhn–
Tucker (KKT) conditions for the pseudo-gradient. Another related work is [90]
where the authors present a scheme that associates to a generalized variational in-
equality, a dual problem and KKT conditions, thus allowing to solve primal and dual
problems in the spirit of classical Lagrangian duality for constrained optimization
problems, using set theoretic concepts and set-valued operators. Extensions based
on this approach have been developed for non-compact and non-smooth settings in
[138].
The treatment is this chapter mostly follows the Lagrangian approach extension
proposed in [120]. This is a procedural method for computing a Nash equilibrium,
based on an extension of duality to a game theoretical framework [32]. The setting
of the construction uses the two-argument NG cost function (see Chap. 4) and re-
laxes also the constraints into a two-argument form. Thus the problem is enlarged
into a constrained optimization problem in a space of twice the dimension followed
by projection back into a one dimension (with a fixed-point solution). Moreover,
for convex constraints, duality leads to hierarchical decomposition into a lower-
level game with no coupled constraints and an optimization problem for Lagrangian
prices.
The chapter is organized as follows. In Sect. 5.2 some results on Nash equilibria
existence are reviewed and relaxation via an augmented optimization problem is
considered. This is followed by results for Lagrangian extension in a game setup in
Sect. 5.3. Sections 5.4 and 5.5 present results for duality extension and hierarchical
decomposition in a game setup.
5.2 Nash Equilibria and Relaxation via an Augmented Optimization 57

5.2 Nash Equilibria and Relaxation via an Augmented

Optimization
In order to formally define a coupled Nash game, let us consider the following cou-
pled inequality constraints:

gr (u) ≤ 0, r = 1, . . . , R (5.1)

or, in vector form, g(u) ≤ 0 where

⎡ ⎤
g1 (u)
⎢ ⎥
g(u) = ⎣ ... ⎦ (5.2)
gR (u)

for u ∈ Ω. The coupled constraint set associated with each gr (u) is denoted by
  
Ω r = u ∈ Ω  gr (u) ≤ 0 (5.3)

where Ω is defined (4.1), i.e., Ω = Ω1 × · · · × Ωm with Ωi = [mi , Mi ].

Thus the resulting action space Ω ⊂ Rm is coupled, defined as


R
  
Ω= Ω r = u ∈ Ω  g(u) ≤ 0 (5.4)
r=1

As before, for every player i ∈ M an individual cost function Ji : Ω → R is defined

that satisfies Assumption 4.3. For every given u−i ∈ Ω−i , a projection action set is
also defined for each i ∈ M,
  
Ωi (u−i ) = ξ ∈ Ωi  g(u−i , ξ ) ≤ 0 (5.5)

For each i ∈ M, this projection action set Ωi (u−i ) is the feasible action set under
the given u−i . A vector u = (u−i , ui ) is called feasible if u ∈ Ω. The resulting
coupled Nash game is denoted by G(M, Ω i , Ji ), and an NE solution is defined as
follows.

Definition 5.2 (NE for coupled Nash games) A vector u∗ ∈ Ω is called an NE

solution NE solution of G(M, Ω i , Ji ) if for all i ∈ M and for every given u∗ ,
−i
∗  ∗  ∗ 
∗
Ji u−i , ui ≤ Ji u−i , ui , ∀ui ∈ Ω i u−i , (5.6)
i (u∗ ) is defined in (5.5).
where Ω −i

Let us look again at Example 5.1. Note that (u∗1 , u∗2 ) = ( 14 , 7 ) ∈ Ω (see Fig. 5.2),
9 4
∗ ∗
where (u1 , u2 ) is the unique NE solution obtained in Example 4.7. Then obviously
(u∗1 , u∗2 ) is the NE solution of the associated Nash game with coupled constraints.
The following proposition (adapted from Theorem 4.4 in [20]) gives sufficient
conditions for existence of an NE solution.
58 5 Computational Results for Games with Coupled Constraints

Fig. 5.2 Reaction curves in

coupled Nash game: r1 (u2 )
and r2 (u1 )

Proposition 5.3 (Theorem 4.4, [20]) Let the action space Ω be a compact and
i , Ji ) admits an NE solution.
convex subset of Rm . Under Assumption 4.3, G(M, Ω

Next let us use the concept of NG cost function presented in Chap. 4 to charac-
terize an NE solution of G(M, Ω i , Ji ).
Recall the two-argument NG cost function J˜(u; x) defined in (4.7),


m
J˜(u; x) := Ji (u−i , xi )
i=1

and note that J˜(u; x) is in its second argument x. Then with respect to J˜(u; x), an
NE solution u∗ of G(M, Ω i , Ji ) satisfies
  
J˜ u∗ ; u∗ ≤ J˜ u∗ ; x , ∀x ∈ Ω, with g u∗−i , xi ≤ 0, ∀i ∈ M (5.7)

This result can be obtained by using Definition 5.2, the definition of J˜(u; x), (4.7),
i (u−i ), (5.5).
and the projection action set Ω
Now, let us also augment the coupled constraints g(u) in (5.2) into an equivalent
two-argument form, g̃,

m
g̃(u; x) = g(u−i , xi ), (5.8)
i=1

where g̃ = [g̃1 , . . . , g̃R ]T and


m
g̃r (u; x) = gr (u−i , xi ), ∀r = 1, . . . , R
i=1
5.2 Nash Equilibria and Relaxation via an Augmented Optimization 59

If x = u, then it follows that


m
g̃(u; u) = g(u) = m × g(u) (5.9)
i=1

Not only J˜ but g̃(u; x) also is in its second argument x.

We can remark that the use of the J˜ [20], defined on a space of twice the dimen-
sion of the original game is instrumental in what follows. This is because it allows
to find a solution of the original Nash game with coupled constraints by solving
a constrained optimization problem for J˜ and searching for a fixed-point solution.
Two main features allow this. Firstly, the is in the second argument x for every
given u, i.e., each component cost function Ji in J˜(u; x) is decoupled in x and de-
pends only on xi . Secondly, the constraints g have been augmented into a separable
two-argument form g̃ thus enlarging the search set. NG-feasibility is equivalent to
g̃(u; u) ≤ 0. Intuitively, by introducing the NG cost function J˜(u; x), the coupled
Nash game is related to a constrained optimization problem for J˜(u; x) that has a
fixed-point solution. Recall that this methodology has been used for uncoupled Nash
games in Chap. 4 where the associated optimization problem is unconstrained.
Herein, the optimization problem for J˜(u; x) is a constrained minimization of
J˜(u; x) with respect to x, with constraints g̃(u; x), (5.8). A solution u∗ of the con-
strained minimization of J˜(u; x) satisfies
  
J˜ u∗ ; u∗ ≤ J˜ u∗ ; x , ∀x ∈ Ω, with g̃ u∗ ; x ≤ 0, (5.10)

with g̃(u∗ ; u∗ ) ≤ 0.
Specifically one arrives at the following result.

Proposition 5.4 A solution u∗ of the constrained minimization of J˜(u; x) satisfying

i , Ji ).
(5.10) with g̃(u∗ ; u∗ ) ≤ 0 is an NE solution of G(M, Ω

This result has been proved by contradiction in [120]. We briefly review the argu-
ment here. By contradiction assume that u∗ is not an NE solution of G(M, Ω i , Ji ).
∗
It follows that for some i ∈ M, there exists an x̃i ∈ Ωi with g(u−i , x̃i ) ≤ 0, such
that
 
Ji u∗−i , x̃i < Ji u∗−i , u∗i

By adding a term j ∈M,j =i Jj (u∗−j , u∗j ) to both sides, the following inequality
holds:
 
J˜ u∗ ; x̃ < J˜ u∗ ; u∗ ,
with x̃ := (u∗−i , x̃i ) ∈ Ω. From g̃(u∗ ; u∗ ) ≤ 0 and (5.9), it follows that
g(u∗−j , u∗j ) ≤ 0, ∀j ∈ M, such that
   
g̃ u∗ ; x̃ := g u∗−j , u∗j + g u∗−i , x̃i ≤ 0
j ∈M,j =i
60 5 Computational Results for Games with Coupled Constraints

It follows that there exists an x̃ such that

  
J˜ u∗ ; x̃ < J˜ u∗ ; u∗ , x̃ ∈ Ω, with g̃ u∗ ; x̃ ≤ 0,

which contradicts the hypothesis in (5.10). Thus u∗ is an NE solution of

G(M, Ω i , Ji ) as needed.
As seen in Proposition 5.4, an NE solution of G(M, Ω i , Ji ) can be found by
˜
solving the problem for J (u; x) and searching for a fixed-point solution.

5.3 Lagrangian Extension in a Game Setup

As seen in the above section, for every given u ∈ Ω, the constrained minimization
(5.10) is a standard constrained optimization problem (see the appendix). In a game
context one can use a Lagrangian extension for a two-argument constrained opti-
mization as proposed in [116, 120].
Next we give a brief introduction of this methodology as it leads to an elegant
hierarchical decomposition. Some results reviewed here have further use in the fol-
lowing chapters. Readers are referred to [120] for some of the detailed proofs.
As in standard optimization, associated to (5.10) a two-argument Lagrangian
function L̃ is defined for J˜ and g̃, where

L̃(u; x; μ) = J˜(u; x) + μT g̃(u; x), (5.11)

with μ = [μ1 , . . . , μR ]T being the Lagrange multiplier vector.

The next result (extension to Propositions A.3 and A.4) gives optimality condi-
tions for an NE solution in terms of Lagrange multipliers.

Proposition 5.5 Consider G(M, Ωi , Ji ). Let the action space Ω be a compact and
convex subset of R and Assumption 4.3 is satisfied.
m

(a) (Necessity): If u is an NE solution of G(M, Ω i , Ji ), then there exists a unique

vector μ∗ ≥ 0 such that

∇x L̃ u; x; μ∗ x=u = 0 (5.12)

μ∗ T g(u) = 0, (5.13)

where the notation “|x=u ” is defined in (4.11) and denotes finding a fixed-point
solution.
(b) (Sufficiency): Let u∗ be a feasible vector together with a vector μ =
[μ1 , . . . , μR ]T , such that μ ≥ 0 and μT g(u∗ ) = 0. Assume that u∗ minimizes
the Lagrangian function, L̃ (5.11), over x ∈ Ω, as a fixed-point solution, i.e.,
u = u∗ satisfies
u = arg min L̃(u; x; μ), (5.14)
x∈Ω
5.3 Lagrangian Extension in a Game Setup 61

where minimization on RHS of (5.14) is done with respect to x in L̃. Then u∗ is

i , Ji ).
an NE solution of G(M, Ω

Note that if u is an inner NE solution, then μ∗ = 0 such that (5.12) is equivalent

to (4.11), the necessary condition for an NE solution in an uncoupled Nash game.

Remark 5.6 The Lagrangian optimality condition in Proposition 5.5 shows that u∗
is obtained by first minimizing the augmented Lagrangian function L̃(u; x; μ) with
respect to the second argument x, which gives x = φ(u) for every given u. The next
step involves finding a fixed-point solution u∗ of φ by setting x = u, i.e., solving
u = φ(u). This u∗ , a fixed-point solution to the minimization of L̃ over x ∈ Ω, the
following holds: L̃(u∗ ; u∗ ; μ) ≤ L̃(u∗ ; x; μ), ∀x ∈ Ω. Note that u∗ thus obtained
depends on μ, u∗ (μ). An optimal μ∗ is achieved by solving

μ∗ T g u∗ μ∗ = 0 and μ∗ ≥ 0 (5.15)

That is, in compact notation we write

  

L̃ u∗ ; u∗ ; μ = min L̃(u; x; μ)  (5.16)
x∈Ω x=u

The obtained u∗ (μ∗ ) is a solution of (5.10) and hence an NE solution of

i , Ji ). We shall call (u∗ (μ∗ ), μ∗ ) an NE solution–Lagrange multiplier pair.
G(M, Ω

Example 5.7 Consider the two-player Nash game presented in Example 5.1 with a
coupled constraint. The corresponding J˜(u; x) is
 
2  1
˜
J (u; x) = 2x1 − 2x1 − x1 u2 + x2 − x2 − u1 x2
2
2

The augmented constraint is

g̃(u; x) = g(u2 , x1 ) + g(u1 , x2 ) = (x1 + u2 − 8) + (u1 + x2 − 8) ≤ 0

Thus the Lagrangian function is obtained as

L̃(u; x; μ) = J˜(u; x) + μg̃(u; x)

 
2  1
= 2x1 − 2x1 − x1 u2 + x2 − x2 − u1 x2
2
2
+ μ(x1 + u2 − 8 + u1 + x2 − 8)

To find an NE solution, u, and the corresponding Lagrange multiplier vector, μ,

one needs to solve the necessary conditions, (5.12), with xi = ui , i = 1, 2. Then it
follows that
1
4u1 = u2 + 2 − μ and 2u2 = u1 + − μ
2
62 5 Computational Results for Games with Coupled Constraints

with u and μ satisfying μ(u1 + u2 − 8) = 0 and μ ≥ 0. Thus if μ = 0, then

(u1 , u2 ) = ( 14 , 7 ), which is an NE solution. If μ = 0, (u1 , u2 ) = ( 51
9 4 77
16 , 16 ) with
μ = − 16 < 0 such that ( 16 , 16 ) is not an NE solution.
95 51 77

Note that as Ji and gr are differentiable convex functions and Ω = Rm , the La-
grangian function L̃(u; x; μ) is convex with respect to x, so the Lagrangian mini-
mization is equivalent to the first order necessary condition. Thus in the presence of
convexity the first order optimality conditions are also sufficient.

5.4 Duality Extension

In this section we review duality results for Nash games with coupled constraints,
based on the Lagrangian extension. A dual cost function related to the minimization
of the associated Lagrangian function (cf. Proposition 5.5) is introduced, similar to
standard optimization [24]. For Nash games with convex coupled constraints, one
can show that duality enables decomposition into a lower-level Nash game with no
coupled constraints, and a higher-level optimization problem.
Consider a Nash game with coupled constraints and recall the associated La-
grangian function L̃ and its minimization in a fixed-point sense, as in (5.16). with
the resulting fixed-point solution as a function of μ, u∗ = u∗ (μ).
We define a function called the dual cost function D(μ) as

D(μ) := L̃ u∗ ; u∗ ; μ , (5.17)

where u∗ minimizes L̃ defined in (5.11) over x ∈ Ω as a fixed-point solution, i.e.,

u = u∗ satisfies
u = arg min L̃(u; x; μ)
x∈Ω
In a fixed-point notation, D(μ) can be written as
 

D(μ) := min L̃(u; x; μ)  ,
x∈Ω arg minx∈Ω L̃=u

where g̃(u; u) ≤ 0, and the dual optimal value is defined as

D ∗ = max D(μ) (5.18)

μ≥0

The primal and dual optimal solution pairs are characterized by the following
result (Theorem 2 in [120]).

Theorem 5.8 (u∗ ; μ∗ ) is an optimal NE solution–Lagrange multiplier pair in the

sense of (5.10) and (5.18), if and only if:
(1) u∗ ∈ Ω and g̃l (u∗ ; u∗ ) ≤ 0 (NG-feasibility);
5.4 Duality Extension 63

(2) μ∗ ≥ 0 (dual feasibility);

(3) u∗ = arg{minx∈Ω L̃(u; x; μ∗ )|x=u } (Lagrangian optimality);
(4) μ∗ T g̃(u∗ ; u∗ ) = 0 (complementary slackness).

Proof If (u∗ ; μ∗ ) is an optimal NE solution–Lagrange multiplier pair, then u∗ is

feasible and μ∗ is dual feasible and the first two relations follow directly. The last
two relations follow from Proposition 5.5.
For sufficiency, using Lagrangian optimality one obtains
  

L̃ u∗ ; u∗ , μ∗ = min L̃(u; x; μ∗ ) 
x∈Ω x=u

so that
 
L̃ u∗ ; u∗ , μ∗ ≤ L̃ u∗ ; x; μ∗ , ∀x ∈ Ω

Using (5.11) and complementary slackness yields

 
L̃ u∗ ; u∗ , μ∗ = J˜ u∗ ; u∗

Then from the foregoing inequality one can write

  
J˜ u∗ ; u∗ ≤ J˜ u∗ ; x , ∀x ∈ Ω, g̃ u∗ ; x ≤ 0

Therefore (5.10) holds and u∗ is an optimal NE game solution with J˜∗ = J˜(u∗ ; u∗ ).
Using (5.17), evaluated at μ∗ , and the foregoing relations yields
  
D μ∗ = min L̃ u; x; μ∗ x=u = J˜ u∗ ; u∗
x∈Ω

and for the optimal dual cost D ∗ , (5.18),

 
D ∗ ≥ D μ∗ = J˜ u∗ ; u∗ = J˜∗ 

If a Lagrange multiplier μ is known then all optimal NE solutions can be found

by minimizing the Lagrangian L̃ over x ∈ Ω, in the fixed-point sense as in (5.16).
However among those solutions u∗ (μ), there may be vectors that do not satisfy the
coupled NG-feasibility condition g(u∗ ) ≤ 0, so this has to be checked. Such an u∗
is a solution to (5.10) and hence an NE solution if the complementary slackness
condition in Theorem 5.8 holds. Note that u∗ thus obtained depends on μ, u∗ (μ).
To summarize, we saw that a Nash equilibrium for games with coupled con-
straints can be found by solving a constrained optimization problem for the two-
argument NG cost function. This can be used as a convenient vehicle for compu-
tation of a Nash equilibrium. The problem is relaxed into an optimization problem
in a space of twice the dimension and a fixed-point solution is used to project back
into the original space and game problem. The formalism of standard optimization
enables one to set up a Lagrangian extension and duality results. Conditions for the
existence of an NE solution are given in Proposition 5.3, the same as the ones for
64 5 Computational Results for Games with Coupled Constraints

the existence of a normalized equilibrium in [128]. In fact, u∗ (μ∗ ) is a normalized

equilibrium point with all ri in [128] equal to 1. Note that uniqueness of the Nash
equilibrium is not presumed. Rather this technique offers a procedural (computa-
tional) method to find one such solution.

5.5 Hierarchical Decomposition in a Game Setup

The separability in the second argument of both NG cost function and constraints
ensures that D(μ) in (5.17) can be decomposed. Such a decomposition result for
the minimization of L̃(u; x; μ) (Theorem 3, [120]) is presented next. The result
shows that the minimum of L̃(u; x; μ) with respect to x ∈ Ω can be obtained by
minimizing a set of one-argument Lagrangian functions. Thus the fact that both the
NG-game cost and the constraints are separable in the second argument is exploited
to show that the dual NG cost function D(μ) can be decomposed and, equivalently,
found by solving a modified Nash game with no coupled constraints.
The decomposition result is given next.

Proposition 5.9 (Theorem 3, [120]) Consider G(M, Ω i , Ji ). Let the action space
Ω be a compact and convex subset of R and Assumption 4.3 is satisfied. The
m

associated dual cost function D(μ), (5.17), can be decomposed as


m

D(μ) = Li u∗−i (μ), u∗i (μ), μ , (5.19)
i=1

where
Li (u−i , xi , μ) = Ji (u−i , xi ) + μT g(u−i , xi ) (5.20)
and u∗ (μ) = [u∗i (μ)] ∈ Ωminimizes a set of Li defined in (5.20) over xi ∈ Ωi as a
fixed-point solution, ∀i ∈ M. In other words, ui = u∗i (μ) satisfies

ui = arg min Li (u−i , xi , μ), ∀i ∈ M

xi ∈Ωi

Proof By Proposition 5.5, the necessary conditions for NE optimality with respect
to the Lagrangian L̃, (5.11), require one to solve

∇x L̃(u; x; μ)|x=u = 0 (5.21)

or, equivalently component-wise, ∂ L̃(u;x;μ)

∂xi |xi =ui = 0, i = 1, m. Using the defini-
tions for J˜ and g̃ one can write L̃, (5.11), as


m
L̃(u; x; μ) = Li (u−i , xi , μ) (5.22)
i=1
5.5 Hierarchical Decomposition in a Game Setup 65

with Li as in (5.20). Using (5.22) into the foregoing yields


m
∂
Lj (u−j , xj ; μ) = 0, i = 1, m
∂xi
j =1

Due to separability with respect to x, this yields

∂
Li (u−i , xi ; μ) = 0, i = 1, m (5.23)
∂xi
Therefore, component-wise (5.21) is the same as (5.23). To either (5.21) or (5.23)
one needs to find a fixed-point solution. Now (5.23) are the first order necessary
conditions for minimizing Li (5.20) with respect to xi . Since Ji and gr are convex,
they are also sufficient. For each given u−i , from (5.23) one obtains xi∗ = xi (u−i ),
so that
 
xi (u−i ) = arg min Li (u−i , xi , μ) , i = 1, m (5.24)
xi ∈Ωi

Moreover, because a fixed-point solution is sought one sets x = u, i.e., component-

wise one needs to solve

xi (u−i ) = ui , ∀i = 1, . . . , m

for a fixed-point vector denoted u∗ = [u∗i ] and x = [u∗i ], which depends on μ. With
this u∗ let us return now to the value functional in (5.22). The first step taken in
order to obtain u∗ was minimization with respect to x, so that from (5.22) one has


m
min L̃(u; x; μ) = min Li (u−i , xi , μ), x∈Ω
x∈Ω x∈Ω
i=1

for any given u, with Li as in (5.20). Since Ω = Ω1 × · · · × Ωm and the right-hand

side is separable with respect to x = [xi ], xi ∈ Ωi , it follows that


m
min L̃(u; x; μ) = min Li (u−i , xi , μ) (5.25)
x∈Ω xi ∈Ωi
i=1

for any given u. Now evaluating (5.25) at the fixed-point u∗ = [u∗i ], x = [u∗i ] ob-
tained as above, one can write
  m 
 
 
min L̃(u; x; μ)  = min Li (u−i , xi , μ) 
x∈Ω u=u∗ ,x=u∗ xi ∈Ωi ui =u∗i ,xi =u∗i
i=1


m

= Li u∗−i (μ), u∗i (μ), μ
i=1
66 5 Computational Results for Games with Coupled Constraints

We write this in a compact fixed-point notation

  m 
 
 
min L̃(u; x; μ)  = min Li (u−i , xi , μ) 
x∈Ω x=u xi ∈Ωi xi =ui
i=1

The proof is completed by using (5.17) and recalling that u∗ is a fixed-point solution
to the set of m optimizations (5.24), i.e., equivalently u∗ is an NE solution to the
Nash game with cost functions Li , (5.20). 

Proposition 5.9 yields a decomposition into a lower-level modified Nash game

with cost functions Li , (5.20), with no coupled constraints, and a higher-level opti-
mization problem. In general, u∗ (μ) may not be NE optimal for the given μ, in the
sense of attaining the minimum NG cost such that L∗i = Ji∗ . However , by Theo-
rem 5.8 there exists a dual optimal price μ∗ ≥ 0 such that u(μ∗ ) = [ui (μ∗ )] is NE
optimal. Hence μ∗ can be found as the maximizer in (5.18). A sufficient condition
is that the dual cost D(μ) is strictly concave in μ, for u∗ (μ) as obtained from the
lower-level game, (5.20). Alternatively, the price μ can be adjusted until the slack-
ness conditions in Theorem 5.8 are satisfied indicating that the dual optimal price
μ∗ is found.
This decomposition result has a hierarchical game interpretation [20]. At the
upper-level is a Stackelberg game ([20], p. 179): the system is the leader that sets
“prices” (Lagrange multipliers) and the m players are the followers. Given prices as
set by the leader, a Nash game is played at the lower-level between m players, with
cost functions Li , (5.20). Each player reacts to given “prices” and the price acts as
a coordination signal.
As in standard optimization, the hierarchical game decomposition may offer
computational advantages. For example, the lower-level game may admit a closed-
form explicit solution, or the higher-level problem may have a reduced dimension.
One such application of these results is presented in the following chapters.
Provided that the action space Ω is compact and convex and Assumption 4.3 is
satisfied, the NE solution–Lagrange multiplier pair (u∗ (μ∗ ), μ∗ ) can be obtained
by solving a lower-level uncoupled Nash game with individual cost functions Li ,
(5.20), so that u∗ (μ) is obtained and a higher-level problem with respect to the
Lagrange multiplier, i.e., solving (5.15).
The duality approach offers a natural way to hierarchically decompose a Nash
game with coupled convex constraints into a lower-level Nash game with individ-
ual cost functions Li (u−i , xi , μ) and with no coupled constraints, and a higher-
level system optimization problem. In effect, the interpretation is that a procedural
method for finding a solution to a Nash game with coupled constraints can be based
on solving a modified game with no coupled constraints and an optimization prob-
lem.
5.6 Notes 67

5.6 Notes

Game theoretical formulations of problems and computational approaches towards

solving generalized or coupled Nash games have been areas of much recent interest.
The study of conditions for existence and uniqueness of Nash equilibrium [95] in
pure strategies continues to be a fundamental issue. Only sufficient conditions for
existence are available, while for the study of uniqueness by now only partial results
are available. Uniqueness results exist only for special classes of games. Recent
work on this topic has focused on S-modular games [10], potential games [135],
routing games in parallel links [17]. Uniqueness of a normalized equilibrium point
is studied in [128]. From a computation point of view, the study of generalized
Nash equilibrium presents severe analytical difficulties [40]. Insightful theoretical
results have been obtained for computation of equilibria in classes of games with
structures, such as two-player polynomial games [113], separable games [151] or
potential games [89].
Duality has also received interest from the perspective of games. Duality and
dual games are studied for repeated two-player zero-sum games in [35]. The results
in [120] and their extension herein have provided a way of finding one such Nash
equilibrium, assuming that at least one exists but not presuming uniqueness of the
Nash equilibrium. The related similar results in [40] that have appeared almost at the
same time as [120] in parallel and independently indicate continued interest in this
area after 40 years. In the following chapters we shall mostly follow this approach.
 Part II
Game Theory in Optical Networks
Chapter 6
Optical Networks: Background and Modeling

Abstract This chapter provides an overview of basic background on transmission

in optical networks and on general topologies to be studied. Most of the material is
adapted from Agrawal (Fiber-optic Communication Systems, 3rd edn. Wiley, New
York, 2002) and Ramaswami and Sivarajan (Optical Networks: A Practical Perspec-
tive, 2nd edn., Academic Press, San Diego, 2002), and the reader is referred to these
references for more in-depth material. The concepts of OSNR and link power ca-
pacity constraint in optical networks are introduced as a preface to the remaining
chapters.

6.1 Introduction
This book is focused on methodologies and game theory for optical networks
from a control perspective. The general setup is that given an optical communica-
tion network, some performance measures to be optimized among many networks
units/players/channels, one must design an algorithm that achieves as good a per-
formance as possible for each channel. In the following two chapters we consider
the problem of developing such algorithms from a game theoretical perspective.
Channels seek to re-adjust parameters such that they maintain an optimal quality of
service (QoS) (OSNR). Because of optical networks’ characteristics, game theoretic
formulations and results cannot be ported directly from other application domains
(e.g. wireless networks or congestion control). These characteristics are either phys-
ical constraints or network specific topologies that translate into various mathemat-
ical constraints in the game formulation: coupled as opposed to disjoint action sets,
constraints on player’s interaction (global versus localized interaction, i.e., one shot
end-to-end games versus stage or partitioned games), constraints on players’ ac-
tions (global constraints versus propagated/modified constraints). In this chapter we
present the background and mathematical modeling on optical networks needed to
develop these game theoretically inspired algorithms.
Optical wavelength-division multiplexed (WDM) communication networks are
evolving beyond statically designed point-to-point links. That is, one goal is how
to realize reconfigurable networks with arbitrary topologies, while at the same time
maintaining network stability, optimal channel transmission performance and qual-
ity of service (QoS), [92].

L. Pavel, Game Theory for Control of Optical Networks, 71

Static & Dynamic Game Theory: Foundations & Applications,
DOI 10.1007/978-0-8176-8322-1_6, © Springer Science+Business Media New York 2012
72 6 Optical Networks: Background and Modeling

At the physical transmission level, one parameter that directly determines chan-
nel performance and QoS is the bit-error rate (BER). BER in turn, depends on op-
tical signal-to-noise ratio (OSNR), dispersion, and nonlinear effects, [2]. Typically
in link optimization, OSNR is considered as the dominant performance parameter,
with dispersion and nonlinearity effects being kept low by proper link design. This
is called the noise-limited regime versus the dispersion-limited regime as these two
regimes can be treated separately. Detailed OSNR computations are needed to de-
termine whether the channel OSNR on a given optical device or route is acceptable
[152]. Therefore, OSNR optimization can be directly translated to QoS optimiza-
tion.
The dominant impairment in OSNR is given by noise accumulation in chains of
optical amplifiers and its effect on OSNR, [45]. A traditional approach uses a static
budget of fiber and device impairments along a fiber link with sufficient tolerance
margins added [126]. Therefore at least a desired OSNR is achieved for each chan-
nel. The process is repeated manually after reconfiguration. An optimum map of the
amplifier gain distribution has been proposed in [85]. The map not only minimized
the effects of the nonlinearity, but also gave rise to the minimum OSNR for a given
average power and the minimum average power for a given OSNR. In fact, power
control has been used to solve OSNR equalization problems since the 1990s. The
primary objective of power control is to obtain a satisfactory channel OSNR by reg-
ulating channel input power. In [34], a heuristic algorithm was proposed for on-line
OSNR equalization in single point-to-point links, while an analysis of the perfor-
mance limits associated with end-to-end equalization was performed in [154, 155].
The algorithm in [34] is centralized, in that each channel needs to know the OSNR of
all other channels. OSNR equalization has been formulated as a static optimization
problem in a point-to-point link in [45]. Optimal channel optical power vector at the
transmitter is found by solving an eigenvalue problem for the system transmission
matrix, composed of link gains of all channels. The proposed algorithm, developed
only for a single point-to-point link, requires centralized, global link information.
Current approaches that are developed for single links are not appropriate for
reconfigurable optical networks, where the length of the links and the number of
devices in a link are changing. Moreover, in these networks arbitrary topologies
can be formed, different channels can travel via different optical paths, and also
different channels can have different levels of QoS (OSNR) requirements. There-
fore one seeks to adjust network parameters (optical power, gains) in an optimal
way, based on on-line network feedback (from receiver to transmitter and various
nodes). This adjustment can result in increased network flexibility and capacity,
so that the needed quality of service (QoS) is ensured whenever a new channel is
added/dropped.
These observations justify the need for on-line network control algorithms to-
wards OSNR optimization that have provable convergence properties for general
network configurations. Moreover, particularly useful are decentralized algorithms,
such that channel power at the transmitter (Tx), can be adjusted based on feedback
from the corresponding receiver (Rx) only, plus other channel measurements. Fig-
ure 6.1 depicts one such feedback system (encapsulating the mesh network above)
6.1 Introduction 73

Fig. 6.1 Network control

block

with a power control algorithm. We take channel signal power at the sources (chan-
nel power) as input to the optical network system and channel OSNR output and
other network measurements as feedback signals.
This problem on how to design such algorithms can be approached from an opti-
mization perspective or from a game theoretical perspective. A game theoretical one
is suited particularly because it naturally enables one to capture user preferences and
player incentives and leads to distributed algorithm. This is the approach we take in
this book, although from time to time we refer to and compare it to an optimization
approach.
Such power control algorithms can be implemented in an iterative feedback man-
ner. Each channel OSNR is measured by the optical spectrum analyzer (OSA) at the
optical switches and the destination. Feedback network measurements are trans-
mitted via optical service channels (OSCs) in optical networks. Each source has
a computation unit which performs an algorithmic calculation and determines the
corresponding channel input power.
This problem is similar to power control in wireless communication systems, a
topic which has been explored extensively, via either centralized approaches [164],
or decentralized, noncooperative game approaches [8, 41, 61]. There are several
differences that make this problem more challenging in optical networks. In wireless
networks, channels are characterized by static loss/gain only, with single Tx to Rx
links, or multiple links with no gains (ATM networks). Optical networks bring a
new combination of challenges: amplified spans, multiple links, accumulation, and
self-generation of optical (ASE) noise, as well as cross-talk generated at the routing
elements.
In the following chapters we consider this problem of developing algorithms
from a game theoretical perspective for re-adjusting network parameters such that
channels maintain an optimal QoS (OSNR), while at the same time taking into ac-
count constraints imposed by dispersion and nonlinearity. The material is based
on mathematical modeling, presented in this chapter, organized as follows. In the
next two sections (Sects. 6.2 and 6.3) we provide basic background on transmis-
sion basics in optical networks and on general topologies to be studied. In Sect. 6.4
the analytical mathematical model for OSNR is presented for point-to-point link
(Sect. 6.4.1) and for network topology (Sect. 6.4.2), both as recursive relations and
end-to-end relations. Link power capacity constraint is introduced in Sect. 6.4.3.
74 6 Optical Networks: Background and Modeling

Fig. 6.2 A point-to-point WDM fiber link

6.2 Transmission Basics

In this preliminary section we present some basic principles of optical communica-
tions. The textbook [126] gives a broad introductory background on optical com-
munication networks, as well as optical components. On the other hand [2] provides
a much more detailed study of the physics involved in optical communication sys-
tems, and the reader is referred to these references for more in-depth treatment.
Optical networks are high-capacity communication networks where optical fiber
is used as the transmission medium. A set of channels corresponding to a set of
wavelengths, are transmitted across the network by intensity modulation of the cor-
responding wavelength of light. The signals (channels) are multiplexed into a single
beam of light using wavelength-division multiplexing (WDM) before being trans-
mitted over the fiber. Each channel is assigned a unique wavelength in the beam of
light.
The network medium may be a single point-to-point WDM fiber link (see
Fig. 6.2), or a network of optical switches and WDM fiber links. In turn the trans-
mitter block consists of one or more transmitters. Each transmitter consists of a laser
and a laser modulator to carry out the E/O conversion, one for each wavelength.
In first-generation optical networks, electronic switches were used; this means
that a transmitted signal might go through electrical-to-optical (E/O) and optical-
to-electrical (O/E) conversion many times before reaching its destination. Optical
networks are called all-optical if a transmitted signal remains in the optical form
from its source (transmitter, Tx) to its destination (receiver, Rx) without undergoing
any O/E conversions.
Signals at the different wavelengths are combined onto a single fiber by using an
optical multiplexer. Individual channel powers can be adjusted at the signal sources
(Tx). Signals are demultiplexed at their destination points (Rx) to be processed out-
side the optical domain.
A receiver block may consist of a demultiplexer and a set of photodetectors. Each
photodetector receives a signal at a wavelength and carries out the O/E conversion.
As an optical signal propagates through the fiber, its optical power (which is pro-
portional to the magnitude of the light intensity) is attenuated due to absorption
and scattering. Optical amplifiers (OAs) are deployed along fiber links throughout
the network, about every 80 to 100 kilometers in an optical link, in order to am-
plify channel powers. Due to the broad spectrum of an optical amplifier gain as
6.3 Topologies and Setup 75

well as to there being virtually no cross-talk (slow response-time), a single OA can

provide simultaneous amplification of many wavelengths (channels). However the
amplifier does not have individual control of the gain or the power of each wave-
length (channel); the spectral shape of the gain dictates the amount of gain that
each channel will experience. While providing such benefits as simultaneous chan-
nel amplification, optical amplifiers have the disadvantage of introducing amplified-
spontaneous emission (ASE) noise that gets itself amplified and cascaded, leading
to optical signal-to-noise (OSNR) degradation [2, 126, 161].
Another important physical impairment is dispersion, which characterizes the
propagation of different components of a optical pulse (wavelengths/frequencies,
polarization states, modes, etc.) with different speeds in the fiber and this leads to
pulse spreading. Effective means of dispersion control have to be imposed for high-
performance transmission by using static or tunable dispersion control.
Nonlinear effects appear at transmission of high-intensity light (high power).
Due to the fact that the refractive index depends nonlinearly on the light intensity,
negative effects such as four-wave mixing (generation of additional frequencies), or
coupling of light from one frequency to another (e.g. by stimulated Raman scatter-
ing) or self- or cross-phase modulation of the optical signal can develop. Nonlinear
effects are negligible if the fiber link is designed properly with optimized total input
power of all signals. Such a condition is regarded as a link capacity constraint. The
effects of nonlinearity can also be minimized by a map of optical amplifier gain
distribution as proposed in [85].
Thus network system performance can be improved by dispersion management
techniques and optimization of the total input power. For further details on fiber
nonlinearities, dispersion, and their effects, readers are referred to [19, 29, 33, 62,
88, 124, 152, 167], while an in-depth treatment of these effects can be found in [2].
Typically in link optimization, OSNR is considered as the dominant performance
parameter, with dispersion and nonlinearity effects being kept low by proper link de-
sign. This is called the noise-limited regime versus the dispersion-limited regime, as
these two regimes can be treated separately. ASE noise generated and accumulated
in chains of OAs degrades associated channel OSNR.
Detailed OSNR computations are needed to determine whether the channel
OSNR on a given optical device or route is acceptable [152]. This OSNR model
will be the focus of Sect. 6.4. The next section, Sect. 6.3 gives some background on
the type of topologies used in optical networks.

6.3 Topologies and Setup

We consider typical topologies for wavelength-routed optical networks.

A lightpath (also referred to as channel) is established for each transmitted signal
from its associated transmitter to the corresponding receiver as per the set of trans-
mission requests. A light path is routed and switched by optical switches on each
intermediate fiber link.
76 6 Optical Networks: Background and Modeling

Fig. 6.3 A single-sink multi-link topology

The collection of lightpaths is called the virtual topology in the optical layer.
The optical switch may be equipped with a wavelength converter. Then some chan-
nels may be converted from one wavelength to another as well along their route by
wavelength conversion.
Consider an optical network that is defined by a set of optical fiber links L =
{1, . . . , L} connecting optical switch nodes. A link l ∈ L is composed of Nl optical
amplified spans. Each span includes an optical fiber followed by an optical amplifier.
These nodes allow channels in the network to be added, dropped or routed and
can also provide the flexibility of channel power adjustments [2].
Optical switch nodes could be optical cross-connects (OXCs) or optical add/drop
multiplexers (OADMs). An OXC deals with multiple wavelengths and selectively
drops some of these wavelengths locally or adds selected wavelengths while letting
others pass through. An OADM provides a similar function but at a much smaller
size.
There are various topologies that an optical network can be configured into, the
simplest one being the point-to-point link topology (Fig. 6.2).
Another simple topology is a single-sink topology where channels are added at
different links, but dropped off at the output of the same link (e.g. Fig. 6.3), multi-
link topology (Fig. 6.4), moving to the most complex, mesh topology (Fig. 6.5(a)).
A set of channels, M = {1, . . . , m}, corresponding to a set of wavelengths,
are transmitted across the network by intensity modulation and wavelength-
multiplexing. We denote by Ml the set of channels transmitted over link l ∈ L.
Also, we denote by Ri , i ∈ M, the set of links from the associated Tx to the corre-
sponding Rx that channel i uses in its optical route. For a link l ∈ Ri , we sometimes
use the notation Ri,l to denote the set of links previous to link l on the path Ri , i.e.,
for ∀l ∈ Ri ,
 
Ri,l := l ∈ L | l ∈ Ri , l precedent to l
We denote by ui , n0i , pi and ni the channel signal power at Tx, the chan-
nel noise power at Tx, the channel signal power at Rx and the channel power
at Rx, respectively, for channel i ∈ M (illustrated in Fig. 6.5(a)). We let u =
6.3 Topologies and Setup 77

Fig. 6.4 Example: three-link topology with three channels

Fig. 6.5 A wavelength-routed optical network configuration: mesh, link, and span

[u1 , . . . , ui , . . . , um ]T or u = [ui ] denote the vector form. We also use u = (u−i , ui )

with u−i = [u1 , . . . , ui−1 , ui+1 , . . . , um ]T .
Sometimes we use u = [ui ] so that [u]i = u(i) = ui .
We sometimes use for convenience the following notation. For two vectors p and
n of the same dimension, we denote by p./n element-by-element division, such that

[p./n]i = [p]i /[n]i = pi /ni

We sometimes also use .∗ to denote element-by-element vector multiplication, such

that

[y. ∗ n]i = [y]i ∗ [n]i = yi ∗ ni

78 6 Optical Networks: Background and Modeling

Fig. 6.6 A point-to-point WDM fiber link

6.4 Power Control and OSNR Model

As we mentioned in the previous sections, the bit-error rate of the transmission
depends on the optical signal-to-noise ratio (OSNR) of the signal channels at re-
ception, Rx. As the signal power in a channel increases, the OSNR value of the
channel increases, and the bit-error rate decreases. That is, maximization of channel
OSNR values is particularly important in optical networks. However, this channel
OSNR optimization is not straightforward, in that a trade-off exists on the attainable
OSNR values of each channel. As one channel increases its signal power, thereby
increasing its OSNR, it may adversely affect the OSNR values of the other chan-
nels in the network. This will become more apparent after this section, where we
present the modeling of channel OSNR in optical networks. The material in this
section is based mostly on [2, 45, 119]. We start with the basic OSNR model for a
point-to-point link, and then move on towards the network model.

6.4.1 Point-to-Point Link Model

Consider a single point-to-point WDM link as shown in Fig. 6.6, composed of N

cascaded optical amplified spans, i.e., optical fiber and amplifiers.
A set M = {1, . . . , m} of channels are transmitted over this link. Each channel
i corresponds to a wavelength λi , i ∈ M. We denote by pi,s and ni,s the optical
signal and noise power of channel i at the output of span s, respectively. For each
s = 1, . . . , N , we denote these in vector form by ps = [pi,s ], ns = [ni,s ], where
i ∈ M. Let channel signal and noise power at the input of the link be denoted by ui
and n0i , respectively, in vector form u = [ui ], n0 = [n0i ], and by convention [alph]
ui = pi,0 , u = p0 (6.1a)
n0i = ni,0 , n0 = n0 (6.1b)

Let channel signal and noise power at the output of the link (at Rx) be denoted by
pi , nout out
i , respectively, or in vector form p and n . These are taken to be the same
as the channel signal and noise power at the output of span N , i.e.,
pi = pi,N , p = pN (6.2a)

i = ni,N ,
nout nout = nN (6.2b)
6.4 Power Control and OSNR Model 79

The optical signal-to-noise ratio (OSNR) at Rx, for the ith channel, i ∈ M, denoted
by yi is defined by
pi
yi := out or y = p · /nout (6.3)
ni
in vector form.
The following basic propagation relations will be used. An optical amplified
span s, s = 1, . . . , N on each link is composed of an optical fiber with attenuation
(loss) coefficient, Ls , which is wavelength independent, and an optical amplifier. An
optical amplifier simultaneously increases the optical power of all channels i ∈ M
multiplexed (transmitted) over the link, with a wavelength-dependent gain (spectral
gain shape).
Let us consider an amplifier on span s and let the gain it provides to channel i be
denoted by gi,s . Then the span transmission of span s for channel i is defined as

hi,s = gi,s Ls , ∀s = 1, . . . , N (6.4)

denoted in vector form by hs = [h1,s . . . hi,s . . . hm,s ]T . Similarly let transmission

up to span s for channel i be denoted as Hi,s and defined by


s
Hi,s = hi,q (6.5)
q=1

and in vector form by Hs = [H1,s . . . Hi,s . . . Hm,s ]T .

An optical amplifier introduces broadband amplified-spontaneous emission noise
(ASE noise). This is in fact the dominant impairment affecting channel performance
and OSNR, i.e., accumulation of ASE noise in amplifiers. The reason is that typi-
cally dispersion and nonlinearity effects are considered to be limited (see [45, 85]).
ASE noise generated by an optical amplifier is gain dependent which means that it
is wavelength-dependent also, due to the spectral gain shape of the amplifier.
The following assumption is typically used with respect to ASE noise [45].

Assumption 6.1 ASE noise does not contribute to the amplifier gain saturation.

ASE noise power generated within the bandwidth of the ith channel by the sth
amplifier is denoted by ASEi,s and is given as [2]

ASEi,s = 2nsp [gi,s − 1]hνi Bo (6.6)

where nsp > 1 is a constant factor called amplifier excess noise factor, h is the Planck
constant, Bo is the optical bandwidth, and νi is the optical frequency corresponding
to wavelength λi , where λi = c/νi , with c the speed of light. Typical values are λi
in the 1550 nm (infrared) range and νi in the THz range. In vector form we denote
ASEs = [ASE1,s . . . ASEi,s . . . ASEm,s ]T .
Noise is generated additively at each amplifier. Moreover in cascaded amplifiers,
each OA amplifies simultaneously signal and noise power, since ASE is in-band
80 6 Optical Networks: Background and Modeling

noise (same wavelength or optical frequency as the signal channel). Thus, channel
signal and noise power at the output of span s on the link, denoted by pi,s , ni,s are
given for ∀s = 1, . . . , N as

pi,s =hi,s pi,s−1 , (6.7a)

ni,s =hi,s ni,s−1 + ASEi,s , (6.7b)

where as before pi,0 = ui , ni,0 = n0i , or in vector form

ps = Diag(hs )ps−1 , (6.8a)

ns = Diag(hs )ns−1 + ASEs , (6.8b)

where Diag(v) denotes the diagonal matrix with the elements of vector v as diagonal
entries.
In the following we present the OSNR model, i.e., its relation to the input optical
signal and noise power. We consider only forward propagation of signal and noise in
steady-state, [45], i.e., we do not consider amplifier gain dynamics. This is justified
for any OSNR game or optimization problem where power adjustments need to be
made at steady-state, after updating network topology.
We can consider separately two cases depending on the amplifier operation
mode: gain control mode (Lemma 6.1), and power control mode (Lemma 6.2) in
[119]. In the gain control mode, the gain of an amplifier is kept constant, while in
the power control mode the total output power is kept constant. The case of gain
control mode leads to a simpler mathematical model (given in Lemma 6.1): OSNR
on a particular channel does not depend on the power of the rest of the channels.
However, the typical operation mode is the automatic power control mode because
this ensures that a constant power is launched into the next span of a link, which
leads to uniform total power distribution across a link and thus limits the nonlinear
effects [2, 85]. The model for this case is given in Lemma 6.2.
The following lemma gives the model for gain control mode.

Lemma 6.1 Let u, n0 be the optical signal and noise power at the input (Tx) of a
point-to-point optical link with N spans and amplifiers in gain control mode. Under
Assumption 6.1 the optical signal power p, noise power nout and the OSNR at Rx y
(6.3) are given in vector form as

p = Diag(HN )u (6.9a)


N
 −1
n out
= Diag(HN ) n +0
Diag(Hr ) ASEr (6.9b)
r=1

and

! 
N
 −1
y = u. n0 + Diag(Hr ) ASEr (6.10)
r=1
6.4 Power Control and OSNR Model 81

where Hr is transmission up to span r, (6.5), HN is the link transmission, and

Diag(v) denotes the diagonal matrix with the elements of vector v as diagonal en-
tries.

Proof The proof follows by developing propagation relations for both signal and
noise. Using (6.8a), (6.8b) yields

ps = Diag(hs ) Diag(hs−1 )ps−2 ,

ns = Diag(hs ) Diag(hs−1 )ns−2 + Diag(hs )ASEs−1 + ASEs

Using these recursively after s together with (6.1a), (6.1b), yields for ∀s = 1, N


s
ps = Diag(hq )u,
q=1
 s

s 
s−2 
ns = Diag(hq )n0 + Diag(hv ) ASEs−r−1 + ASEs ,
q=1 r=0 v=s−r

or

s
ps = Diag(hq )u,
q=1


s
ns = Diag(hq )n0
q=1
 s s−r−1 −1

s−2  
+ Diag(hv ) Diag(hv ) ASEs−r−1 + ASEs
r=0 v=1 v=1
"s
After some manipulation, factoring out v=1 Diag(hv ), denoting s − r − 1 by new
index r we obtain

s
ps = Diag(hq )u,
q=1
   r −1

s 
s 
ns = Diag(hq ) n +0
Diag(hv ) ASEr
q=1 r=1 v=1

Now from (6.5) one can write


s
Diag(Hs ) = Diag(hs ) Diag(hs−1 ) · · · Diag(h1 ) = Diag(hq )
q=1
82 6 Optical Networks: Background and Modeling

so that, for ∀s = 1, . . . , N , the foregoing is written as

ps = Diag(Hs )u,


s
 −1
ns = Diag(Hs ) n + 0
Diag(Hr ) ASEr
r=1

Using these for s = N with p = pN , nout = nN (see (6.2a), (6.2b)) gives (6.9a),
(6.9b). Then (6.10) follows immediately by using these into (6.3). 

Next one can use this result and develop a model for the case when amplifiers are
operated in automatic power control mode, which is the typical operation mode for
optical amplifiers when the network is in steady-state, [2]. We will see that this case
leads to a more complex mathematical model; the inherent scaling on the total power
translates into coupling between all channels’ powers, and OSNR on a particular
channel does depend on all other channels’ powers.
The following assumption is used for optical amplifiers.

Assumption 6.2 All spans on each link have equal length and all optical amplifiers
are operated in automatic power control mode with the same total power target P 
and have the same gain spectral shape [2, 85].

This assumption of same gain spectral shape and power target is a reasonable
assumption if all spans in a link have the same length, i.e., for uniformly designed
optical links, [2]. Then gi,s = gi for s = 1, . . . , N (same spectral shape), and by (6.6)
the ASE noise generated is the same at each amplifier on the link. In vector form
let g = [g1 . . . gi . . . gm ]T or g = [gi ] denote the vector spectral gain shape of any
amplifier on the link. Similarly for each s = 1, . . . , N let gs = [g1s . . . gis . . . gm
s ]T or

g = [gi ].
s s

Also by Assumption 6.2 at the output of each span s = 1, . . . , N one can write

,
1Tm ps = P (6.11)

and at the output of the link

,
1Tm p = P (6.12)
where 1m = [1 1 . . . 1]T is the m-dimensional all-ones vector. The above condition
is called the system constraint. One benefit of keeping a constant total power at
the output of the amplifier, or a constant total launching power into next span, is
the compensation of variations in fiber-span loss across a link [45]. Moreover, if
the total power target is selected to be below the threshold for nonlinear effects, an
optimal gain distribution is achieved across the link [85].
Mathematically, in automatic power control mode a scaling of the gain is per-
formed such that the same total power is kept at the output of each amplifier [45].
This can be realized by using a variable optical filter after each amplifier, this filter
6.4 Power Control and OSNR Model 83


having the loss coefficient ηs adjusted to achieve the constant total power target P
[45]. Thus under Assumption 6.2, the effective gain of an optical amplifier can be
written for s = 1, . . . , N , i ∈ M, as

gi,s = gi ηs , (6.13)

where gi is the corresponding spectral gain value for channel i, and ηs the filter loss.
As we will see next the scaling via ηs introduces coupling between channel powers,
and channel OSNR is no longer independent of other channels’ powers.
The following result gives the OSNR model for a point-to-point WDM link de-
veloped based on (6.2a), (6.2b), (6.11), (6.20), and (6.7a), (6.7b). An earlier similar
simple model can be found in [45].

Lemma 6.2 Let u, n0 be the signal and noise power at the input (Tx) of a point-
to-point optical link with N spans and amplifiers in power control mode. Under
Assumptions 6.1 and 6.2, the optical signal power p and the OSNR at Rx y (6.3) are
given in vector form as
p = Diag(HN )u (6.14)

where HN = P
[gN ]T u
gN and

y = u./ n0 + Γ u (6.15)
where Γ = [Γi,j ] is the (m × m) link system matrix

N
 −1 ASE r T
Γ = Diag gr g

P
r=1

and Diag(v) denotes the diagonal matrix with the elements of vector v as diagonal
entries.

Component-wise one can write from (6.15)

ui 
yi =  := fi u, n0 , (6.16)
n0i + j ∈M Γi,j uj

where

N gs
j ASE i
Γi,j = (6.17)

gis P
s=1

and HN = [Hi,N ], i ∈ M

giN P
Hi,N =  N
j ∈M gj uj
84 6 Optical Networks: Background and Modeling

Proof From Lemma 6.1, (6.9a), (6.9b), one can write for ∀s = 1, . . . , N

ps = Diag(Hs )u, (6.18a)



s
 −1
ns = Diag(Hs ) n0 + Diag(Hr ) ASEr , (6.18b)
r=1

where ASEr = ASE, for r = 1, . . . , N by Assumption 6.2. Using (6.18a) into (6.11)
gives
 = 1Tm Diag(Hs )u = HTs u
P (6.19)
Recall that Hs = [Hi,s ], i ∈ M where Hi,s is defined in (6.5) as


s
Hi,s = hi,q
q=1

From (6.4) and (6.13) hi,q can be written as

hi,q = gi Lq ηq = gi δq , ∀q = 1, . . . , N, (6.20)

where we denoted δq = Lq ηq (scalar), or in vector form

hq = gδq , ∀q = 1, . . . , N

where hs = [h1,s . . . hi,s . . . hm,s ]T = [hi,s ], g = [gi ]. Substituting (6.20) into the
expression above for Hi,s yields


s 
s
Hi,s = gi δq = gis δq , ∀s = 1, . . . , N,
q=1 q=1
"s
or, by denoting ρs = q=1 δq ,

Hi,s = gis ρs , ∀s = 1, . . . , N

Then in vector form for Hs = [Hi,s ], i ∈ M one can write

H s = gs ρ s , ∀s = 1, . . . , N (6.21)

where the notation gs = [gis ] was used and ρs is scalar.

Now using (6.21) in (6.19) yields
 
 = ρs gs T u,
P ∀s = 1, . . . , N

so that

P
ρs = , ∀s = 1, . . . , N
[gs ]T u
6.4 Power Control and OSNR Model 85

Hence

P
Hs = gs , (6.22)
[gs ]T u
which when used in (6.18a) for s = N yields (6.14). Note that component-wise one
can write

gs P
Hi,s = m i s
j =1 gj uj

Next, for the second part, observe that using (6.22) one can write for the last term
in (6.18b)
 −1 1 −1 T
Diag(Hr ) ASE = Diag gr ASE gr u

P
where the fact that (gr )T u is a scalar was used. Thus replacing this into (6.18b)
yields

ns = Diag(Hs ) n0 + Γs u , (6.23)

where Γs is defined by

s
1 −1 T
Γs = Diag gr ASE gr

P
r=1

Thus, for OSNR, y = p./nout , (6.3), using p in (6.14) and nout = nN in (6.23) for
s = N , yields

y = p./n = u./ n0 + Γ u

with Γ = ΓN , or component-wise
pi ui
yi = OSNRi = = 0 
ni ni + j ∈M Γi,j uj

and the proof is completed. 

Remark 6.3 This is the OSNR model typically used in optical links. Mathematically
the OSNR model in (6.16) is similar to the wireless signal-to-interference ratio (SIR)
model [8]. However, it has a richer system structure: Γ with Γi,j defined in (6.17)
is a full matrix, with cross-coupling terms, non-zero diagonal elements and all el-
ements dependent on network parameters (e.g., the gain and ASE noise of optical
amplifiers). We can rewrite (6.16) as
ui
yi := OSNRi = , (6.24)
X−i + Γi,i ui
86 6 Optical Networks: Background and Modeling

where

X−i = n0i + Γi,j uj . (6.25)
j ∈M,j =i

Hence OSNR is no longer a linear function of channel input power like SIR as in [8].
Furthermore, since optical amplifiers are often cascaded along the link, ASE noise
accumulates over many amplifiers and degrades channel OSNR as the number of
amplifiers increases, which is reflected by (6.16).

6.4.2 Network Model

Let us look now at the network model. The same assumptions are used and similar
notations. Thus consider an optical network with a set L = {1, . . . , L} of links. A set
M = {1, . . . , m} of channels transmitted over the network. Let Ml = {1, . . . , ml }
be the set of channels transmitted over link l ∈ L. Each link l is composed of Nl
cascaded optical amplifiers and optical fiber spans and has P̂l as its constant total
power target. We denote by gi,l the gain experienced by channel i due to the spectral
gain shape (wavelength dependency) of the amplifier on link l. By Ri we denote the
route of channel i from Tx to Rx. Let the optical fiber from each Tx to the optical
switch that is connected be defined as a virtual optical link (VOL) (see Fig. 6.7).
Thus the set of VOLs, Lv , is equivalent to M. Therefore, the set of all optical links
and VOLs is denoted by L = L ∪ Lv = {1, . . . , L }, where L = L + m.
We define two network connection matrices. First, let A = [Ai,l ]m×L denote the
channel transmission matrix, where
#
1, channel i uses link or virtual link l;
Ai,l =
0, otherwise

Similarly, let B = [Bk,l ]L ×L denote a system connection matrix defined as

⎧
⎨ 1, link/virtual link k is connected to link l,
Bk,l = i.e., the channel transmission direction between k and l is k → l;
⎩
0, otherwise

Let A = [Al ] with Al = [A1,l , . . . , Am,l ]T and B = [Bl ] with Bl =

[B1,l , . . . , BL ,l ]T . The channel transmission matrix and the system connection ma-
trix completely describe channel routing and system interconnection, i.e., the net-
work configuration.
Next we introduce the following notations for channels. Let ul denote the m-
dimensional input power vector at each link l (all channels), and ui denotes the
L -dimensional input power vector for each channel (all links), respectively,

ul = [u1,l . . . ui,l . . . um,l ]T , l = 1, . . . , L ,

ui = [ui,1 . . . ui,l . . . ui,L ]T , i = 1, . . . , m
6.4 Power Control and OSNR Model 87

Fig. 6.7 Network configuration: mesh

Similar notations are used for the output powers: pl denotes the m-dimensional
output power vector at each link l (all channels), and pi denotes the L -dimensional
output power vector for each channel (all links), respectively.
Based on the network connection matrices, the following result gives intercon-
nection relation for all the channel powers at link l, i.e., how the input channel
powers ul are obtained from the other links or from the virtual optical links (VOL).
The first case we treat is the end-to-end case where we assume that channel pow-
ers are adjustable only at the transmitter sites. Later on in this section we treat the
recursive case where channel powers are individually adjustable also at the input of
each link by some adjustment factors γ , hence called γ -link. In this first case the
channel signal power at the input of link l, denoted by ui,l is identical to the signal
power at the output of the preceding link l of link l on route Ri , i.e.,

ui,l = pi,l (6.26)

Lemma 6.4 The optical signal powers at the input of link l, ul , l ∈ L, are given in
vector form as
ul = VlT p, (6.27)
where p = [p1 . . . pi . . . pm ]T and pi = [pi,1 . . . pi,k . . . pi,L ]T , with
#
pi,k , k∈L
pi,k =
ui , k ∈ Lv

where ui is the
' signal power of channel i at Tx, and the matrix Vl is the direct sum
of Vi,l , Vl = mi=1 Vi,l , where ∀i = 1, . . . , M, l ∈ L,

Vi,l = [Ai,l B1,l Ai,1 , . . . , Ai,l BL ,l Ai,L ].

88 6 Optical Networks: Background and Modeling

Proof For channel i, i ∈ M its signal power launched into link l, l ∈ L is trans-
mitted either from one of previous links of link l or one of virtual links (Txs). Let
k ∈ L be such a link. Then Bk ,l = 1, Ai,k = 1 and

ui,l = Ai,l (Bk ,l Ai,k pi,k ).

Notice that there is one and only one k such that both Bk ,l and Ai,k are non-zero.
So the above equation can be rewritten as
   
ui,l = Ai,l Bk ,l Ai,k pi,k + Bk,l Ai,k pi,k = Ai,l γi,l Bk,l Ai,k pi,k
k∈L ,k =k k∈L

where
#
pi,k , k ∈ L,
pi,k =
ui , k ∈ Lv ,
and ui is the signal power of channel i at Tx. Since L = {1, . . . , L }, ui,l can be
expressed as
⎡ ⎤
pi,1
⎢ ⎥
ui,l = [Ai,l B1,l Ai,1 , . . . , Ai,l BL ,l Ai,L ] ⎣ ... ⎦ = γi,l · Vi,l · pi (6.28)
pi,L
'm
Recalling that Vl = i=1 Vi,l = Diag(Vi,l ), the vector form of the foregoing
yields (6.27). 

Remark 6.5 The matrix Diag(Vi,l T ) is an augmented system configuration matrix,

indicating not only the connections between links but also channel routing condi-
tions. From the system control point of view, for the network system with channels
transmitted over it, input of this system is the signal power at the input side of each
link and the output is the output signal power of each link.

Remark 6.6 Since noise is propagated in the same manner via the optical switch (no
O/E conversion), it can be immediately seen that, as in (6.26),

i,l = ni,l .
nin out

Thus the input noise power nin

i,l is given similarly as
 
i,l = Ai,l
nin Bk,l Ai,k nout
i,k , ∀i ∈ M, l ∈ L,
k∈L

where
#
nout
i,k , k ∈ L,
nout =
i,k n0i , k ∈ Lv ,
6.4 Power Control and OSNR Model 89

and in vector form, similar to (6.27), the following holds:

l = Vl n.
nin T

In the following we present the framework for OSNR modeling in networks. As

before, an optical amplified span s on link l is composed of an optical fiber with
coefficient, Ll,s , which is wavelength independent, and an optical amplifier with
gain gi,l,s . The optical amplifier introduces amplified-spontaneous emission (ASE)
noise, denoted by ASEi,l,s . Both the gain and ASE noise are wavelength dependent.
Let pi,l,s and ni,l,s denote the channel signal and noise power, respectively, at the
output of span s on link l (illustrated in Fig. 6.5(c)).
Under similar Assumptions 6.1 and 6.2 and the notations, a constant total power
is launched into each span of a link. Under Assumption 6.2, the following relation
holds:

l , ∀l ∈ L,
pi,l = P (6.29)
i∈Ml

or in vector notation
l ,
eT pl = P ∀l ∈ L,

where e is an m-dimensional vector such that for each i ∈ M, [e]i = 1, if i ∈ Ml or

[e]i = 0 else. As before the above condition is called the system constraint.
Next we describe a typical optical link extracted from the network and the OSNR
at the output of this link.
In Fig. 6.8, pi,l and nout
i,l are the signal and noise power at the output of link l,
respectively, and ui,l and nin i,l are the signal and noise power at the input of link l,
respectively, or in vector form pl and ul . The matrix Γl is the link system matrix of
link l.
At the output of link l with l ∈ Ri , we denote the OSNR of channel i by yi,l =
pi,l
nout
, or in vector form
i,l

yl := pl ./nout
l

Applying directly Lemma 6.2 yields the following result.

Proposition 6.7 Let ul , nin

l be the signal and noise power at the input of a link l
with Nl spans and ml channels. Under Assumptions 6.1 and 6.2, the optical signal
power pl and the OSNR yl at the output of the link l are given in vector form as

pl = Diag(HNl )ul (6.30)

and

yl = ul ./ nin
l + Γl ul (6.31)
90 6 Optical Networks: Background and Modeling

Fig. 6.8 Typical optical link

where Γl = [Γli,j ] is the link l system matrix


Nl
 −1 ASEl r T
Γl = Diag grl gl
l
P
r=1

and HNl = [Hi,Nl ] with

Nl 
gi,l Pl
Hi,Nl =  Nl
j ∈Ml gj,l uj,l

Component-wise

pi,l ui,l
yi,l := = in  (6.32)
ni,l ni,l + j ∈Ml Γli, j uj,l

where

Nl s
gj,l ASEi,l
Γli,j = (6.33)
s
gi,l Pl
s=1

Based on (6.31) a recursive OSNR model for the network can be obtained.

Lemma 6.8 Consider a link l and a channel i, i ∈ Ml , and let l be the link prece-
dent to link l for channel i along path Ri . Let yi,l , and yi,l denote the channel
OSNR at the output of link l and l , respectively. Then the following OSNR recursive
relation holds for ∀i ∈ Ml :

1 1  uj,l
= + Aj,l Γli,j (6.34)
yi,l yi,l ui,l
j ∈M

Proof Let us write (6.32) with the help of the connection matrix A, such that for
∀i ∈ Ml
ui,l
yi,l = 
nin
i,l + j ∈M Aj,l Γli,j uj,l
6.4 Power Control and OSNR Model 91

Fig. 6.9 End-to-end model

so that
1 nin
i,l
 uj,l
= + Aj,l Γli,j
yi,l ui,l ui,l
j ∈M

Using (6.26) we have

nin
i,l nout
i,l 1
= = (6.35)
ui,l pi,l yi,l
the last equality following from the definition of yi,l (at the output of link l ). Sub-
stituting 6.35 into the foregoing completes the proof. 

A similar model as in Lemma 6.2 can be developed for the network end-to-end
OSNR albeit with a richer network system matrix Γ (Fig. 6.9). The model can be
derived in a similar way based on recursive use of propagation relations. The reader
is referred to [119] for the full proof of this result as well as for extension to the case
when cross-talk at the optical nodes (OXC or OADM) is included (Lemmas 2 and 3
[119]).

Proposition 6.9 (Lemma 2, [119]) Under Assumptions 6.1 and 6.2, the OSNR of
channel i at Rx, yi is given as
ui
yi =  , (6.36)
n0i + j ∈M Γi,j uj

where Γ = [Γi,j ] is the network system matrix with

"
 q∈Rj,l Hj,Nq
Γi,j = " Γli,j ,
l∈Riq∈Ri,l Hi,Nq

where Γli,j and Hi,Nq are defined in Corollary 6.7 and Rl,i denotes the set of links
on the path Ri before link l.

The network OSNR model (6.36) has the same form as the link OSNR model
(6.2), but with a more complex system matrix Γ = [Γi,j ], which we refer to as net-
work system matrix. The OSNR model shows that as input power of one channel
92 6 Optical Networks: Background and Modeling

Fig. 6.10 Link l

increases, thereby increasing its OSNR, the noise in the other channels increases,
thus decreasing their OSNRs. Based on the OSNR model, we take the channel in-
put power as input to the optical network system and OSNR output and other net-
work measurements as feedback signals to design power control algorithms such
that channel OSNRs can be optimized.
Now we consider the second case, whereby in an optical network, channel pow-
ers are adjustable not only at Txs but also at optical switches [2]. Thus channel
powers can be individually adjusted in the beginning of each optical link via an
adjustment parameter for channel i on link l is denoted by γi,l , γi,l ∈ [γmin , γmax ]
(Fig. 6.10). This is the case called γ -link topology. Then

ui,l = γi,l pi,l , ∀i = 1, . . . , M, l ∈ L

where l is the link precedent to link l for channel i on route Ri .

For l = l − 1 we have the configuration in the following figure (Fig. 6.11),
called γ -link topology, whereby each γ -link is composed of cascaded OAs, with
adjustable power per channel at each stage.
Similar to the previous results we have the immediate extensions below. First we
have the following interconnection result similar to Lemma 6.4.

Lemma 6.10 The optical signal powers at the input of link l, ul , l ∈ L, are given
in vector form as
ul = Diag(γ l )VlT p, (6.37)
where γ l = [γ1,l , . . . , γm,l ]T , and Vl , p are defined as in Lemma 6.4.

Proof In this case

ui,l = Ai,l γi,l (Bk ,l Ai,k pi,k )
or
   
ui,l = Ai,l γi,l Bk ,l Ai,k pi,k + Bk,l Ai,k pi,k = Ai,l γi,l Bk,l Ai,k pi,k
k∈L ,k =k k∈L

and the proof follows immediately as in Lemma 6.4. 

6.4 Power Control and OSNR Model 93

Fig. 6.11 A γ -link topology

Now, following Remark 6.6, since the adjustment factors affect both input signal
and noise power simultaneously, one can write

i,l = γi,l ni,l ,

nin ∀i = 1, . . . , M, l ∈ L
out

That is, for noise power a relation similar to (6.37) holds, i.e.,

l = Diag(γ l )Vl n
nin T

Moreover, let us note that

nin
i,l γi,l nout
i,l 1
= =
ui,l γi,l pi,l yi,l

Thus one can see that in this second case with γi,l adjustment factors (γ -link), the
same recursive OSNR model as in (6.35) and Lemma 6.8 is valid on a per-link basis.

6.4.3 Link Capacity Constraint

Recall that in Sect. 6.2 we mentioned that nonlinear effects on optical fiber along
links limit the total input power. After each span of optical fiber, the total power is
influenced by the gain distribution of optical amplifiers [33]. Under Assumption 6.2,
the same total power is launched into each span of a link. This leads to a uniform
94 6 Optical Networks: Background and Modeling

total power distribution along the link, which minimizes the effects of optical non-
linearity [85].
However, this optimal gain distribution of optical amplifiers is not applicable
to limit nonlinear effects on the optical fiber at the beginning of each link, i.e.,
the fiber segment between Txs (or optical switches) and the first optical amplifier
along the link. Since the effects of the nonlinearity can be reduced by lowering
the total launched power, the following condition is imposed as the coupled power
constraint, also called the link capacity constraint in optical networks:

ui,l ≤ P̂l , ∀l ∈ L (6.38)
i∈Ml

where ui,l is the signal power of channel i at the input of link l. When channel i is
transmitted from its associated Tx directly to link l, ui,l = ui , where ui is the signal
power of channel i at Tx. Otherwise, due to the power propagation along links and
among networks, ui,l is a function of ui , or u where u is defined as the signal power
vector of all channels at Tx. This is similar to capacity constraints in flow control
[13]. However, capacity constraints in flow control are unchanged along links, while
link coupled power constraints in optical networks are propagated along links. The
foregoing (6.38) can be rewritten as

gl (u) = ui,l − P̂l ≤ 0, ∀l ∈ L (6.39)
i∈Ml

We are interested in the convexity of the constraint (6.39).

Example 6.11 In a single point-to-point WDM fiber link (Fig. 6.6), channel i is
transmitted from its associated Tx directly to the link. Then one can drop the link
index and (6.39) is reduced to

g(u) =  ≤ 0.
ui − P (6.40)
i∈M

The function g(u) in the single link case is always convex.

In optical networks, the coupled power constraints are propagated along links,
and due to this, convexity of (6.39) is not automatically ensured, as shown in the
next example.

Example 6.12 Consider the network shown in Fig. 6.12 with three links (L = 3)
and three channels (m = 3) transmitted over links. Each link l has Nl spans and a
l . Let us study the convexity of the constraint on the third link,
total power target P
g3 (u):
3
g3 (u) = u3,1 + u3,2 − P
6.5 Notes 95

Fig. 6.12 Three-link topology with three channels

By using (6.26), i.e., ui,l is identical to pi,l with l being the preceding link of l on
route Ri , we have
3
g3 (u) = p2,1 + p2,2 − P
Then by using the system constraint (6.29) on link 2, we get
3 = −p2,3 + (P
g3 (u) = p2,1 + p2,2 − P 2 − P
3 )

By using (6.14) in Lemma 6.2 and (6.26), we have

2 (G2,3 )N2 u3
P
p2,3 = ,
(G2,1 )N2 u2,1 + (G2,2 )N2 u2,2 + (G2,3 )N2 u3
1 (Gi,1 )N1 ui
P
u2,i =pi,1 = , i = 1, 2
(G1,1 )N1 u1 + (G1,2 )N1 u2

As can be easily seen g3 (u) is a more intricate function of u. Moreover, when com-
2 2
puting the Hessian matrix of g3 (u), the sign of ∂ g3 2(u) and ∂ g3 2(u) may be different.
∂u1 ∂u2
Thus the Hessian matrix ∇ 2 g3 (u) may not be positive semidefinite, that is, g3 (u)
may not remain convex independently of link parameters.

6.5 Notes

This chapter provided an overview of basic background on transmission in optical

networks and on general topologies to be studied, as well as modeling of OSNR.
The reader is referred to [2] and [126] for more in-depth material. The concepts of
OSNR and link power capacity constraint in optical networks are introduced as a
preface to the remaining chapters.
Chapter 7
Games in Point-to-Point Topologies

Abstract This chapter provides the basic formulation of a game framework to-
wards solving the OSNR optimization problem in optical networks. We restrict
the analysis to single point-to-point optical links, as the simplest network topol-
ogy. A Nash game played among channels is employed towards maximizing OSNR
firstly without coupled link capacity constraint. Then for incorporating the coupled
power constraint, two approaches are considered—an indirect and a direct one,
based on Lagrangian pricing and duality extension. Sufficient conditions are de-
rived for the existence and uniqueness of an NE solution for both approaches. Two
convergent iterative algorithms are developed towards finding the NE solution.

7.1 Game Formulation

Noncooperative game theory represents a powerful tool that can capture strategy in-
teractions among self-interested players. Game-theoretic models have started to be
used recently in networks [163]–[8] in the context of dynamic resource allocation,
as an alternative to traditional system-wide optimization [164]–[68]. In large-scale
networks decisions are often made by users independently [78], each according to
its own performance objective. This is also appropriate for large-scale optical net-
works, where it is difficult to maintain a centralized system for transmitting real-time
information between all channels, and cooperation among channels is impractical.
This makes noncooperative game theory a suitable framework [20, 96]. A game-
theoretic approach can be employed to optimize channel OSNR. The interest in
such an approach is motivated by the departure from the assumption of cooperation
among channels, particularly in large-scale optical networks. Here too decisions are
often made independently by channels according to the users’ own performance ob-
jectives. Moreover, a beneficial feature of a game formulation is that it leads itself
to iterative distributed algorithms for the computation of an NE solution.
This problem belongs to a subclass of resource allocation in general communi-
cation networks [4, 134]. In optical network systems, a signal over the same fiber
link can be regarded as an interfering noise for others, which leads to OSNR degra-
dation. A satisfactory OSNR at Rx for each channel may be achieved by regulating
the input power per channel at Tx. As a first step towards solving the OSNR opti-
mization problem in optical networks, in this chapter we study such a problem in

L. Pavel, Game Theory for Control of Optical Networks, 97

Static & Dynamic Game Theory: Foundations & Applications,
DOI 10.1007/978-0-8176-8322-1_7, © Springer Science+Business Media New York 2012
98 7 Games in Point-to-Point Topologies

a single point-to-point optical link. A Nash game played among channels is em-
ployed towards maximizing OSNR firstly without coupled link capacity constraint
(Sect. 7.2). Channel optical signal-to-noise ratio (OSNR) optimization is formulated
as an m-player noncooperative game. Conditions for the existence and uniqueness of
the game equilibrium solution are given. We discuss an iterative algorithm for power
control, which uses only channel feedback measurements and which is shown to
converge to the equilibrium solution [115, 118]. Then for incorporating this link ca-
pacity coupled constraint, two approaches are considered: an indirect penalty-based
approach (Sect. 7.3) and a direct Lagrangian pricing approach (Sect. 7.4). Sufficient
conditions are derived for the existence and uniqueness of an NE solution for both
approaches. Two convergent iterative algorithms are developed towards finding the
NE solution.

7.2 Games Without Coupled Constraints

7.2.1 Utility and Nash Equilibrium Solution

The OSNR game problem is similar to power control via noncooperative game ap-
proaches in wireless networks, a topic which has been explored extensively in works
such as [8, 133]. The problem needs to be formulated and addressed separately for
optical networks, as there are several differences when compared to wireless net-
works [8]. In wireless networks, channels are characterized by uncoupled loss/gain
with single Tx to Rx links. Optical networks have cascaded amplified spans, accu-
mulation and self-generation of optical noise, cross-talk and possible coupling, and
saturation [2].
Considering these differences and physical features of optical networks, relevant
questions that we address in this section are: what is a natural way of formulating
a tractable OSNR optimization game in optical links and then networks of general
topology? How is this game compared to the SIR optimization game in wireless
networks? Is there a natural decomposition that leads to iterative algorithms with
provable convergence, decentralized with respect to channels?
We concentrate on a basic game formulation for an end-to-end optical link, i.e., a
point-to-point link topology. Conditions for uniqueness of the Nash equilibrium as
well as proof techniques and conditions for selecting pricing parameters are different
than in the wireless case. This chapter focuses on a point-to-point link topology and
it provides a starting point for the next chapter where optical network topologies are
considered.
We study a point-to-point WDM fiber link shown in Fig. 6.6 in Chap. 6. We
assume that reconfiguration is finished, i.e., channels will not be added or dropped
while performing adjustments. A set M = {1, . . . , m} of channels are transmitted
over the link. The link consists of N cascaded spans of optical fiber each followed
by an OA (Fig. 7.1). All OAs have the same gain spectral shape and the gain value
for channel i is gi . We denote ui and n0i the signal power and noise power of channel
7.2 Games Without Coupled Constraints 99

Fig. 7.1 A point-to-point link

i ∈ M at Tx, respectively. Similarly, we denote pi and ni the signal power and noise
power of channel i ∈ M at Rx, respectively. Let u = [u1 , . . . , um ]T = [ui ] denote
the vector form of the signal power at Tx. Equivalently, we write u = (u−i , ui )
with u−i = [u1 , . . . , ui−1 , ui+1 , . . . , um ]T in some context to represent the same
vector u. The signal power at Tx is typically limited for every channel. That is, ui
is in a bounded set Ωi = [0, umax ] where umax is a positive constant. We use Ω to
represent the Cartesian product of Ωi , i ∈ M, Ω = Ω1 × · · · × Ωm . We also let
Ω−i = Ω1 × · · · × Ωi−1 × Ωi+1 × · · · × Ωm . Thus ui ∈ Ωi and u ∈ Ω.
The OSNR of channel i at Rx denoted by yi , yi = pnii , is given as in (6.16), i.e.,

ui
yi =  , (7.1)
n0i + j ∈M Γi,j uj

with Γ as in (6.17). Equivalently, (7.1) can be rewritten as

ui 
yi = with X−i = Γi,j uj + n0,i (7.2)
X−i + Γi,i ui
j ∈M,j =i

where X−i denotes the total interference on channel i due to other channels’ power.
The OSNR model reflects that the signal power of one channel can be regarded
as an interfering noise for others, which leads to OSNR degradation. Regulating the
optical powers at Tx, i.e., allocating optical power as a resource among channels,
aims to achieve a satisfactory OSNR for each channel at Rx.
Let us now formulate a game-theoretic approach to solve an OSNR optimiza-
tion problem from the user optimization point of view, based on this OSNR model.
100 7 Games in Point-to-Point Topologies

Specifically we formulate a Nash game where a player is identified with a channel.

The objective of each player is to maximize its utility function related to individual
channel OSNR. Each channel adjusts its power towards this goal in the presence of
all other channels. The game settles at an equilibrium when no channel can improve
its utility unilaterally.
Each channel i, i ∈ M is a player that minimizes its own cost function Ji (or
maximizes its utility function), by adjusting its transmission power, in response to
the other channels’ (players’) actions. Such a game is denoted by G(M, Ωi , Ji ).
This means that the relevant concept is the noncooperative Nash equilibrium (NE)
(see Chap. 4), i.e., a vector u∗ such that
 
Ji u∗−i , u∗i , ≤ Ji u∗−i , ui , ∀i ∈ M,

for any given u∗−i . Recall that NE optimality means that no player has an incentive
to change its action, since no further individual improvement in its cost is possible.
Let us consider that each individual cost function Ji is defined as the difference
between a pricing function Pi and a utility function Ui

Ji (u) = Pi (u) − Ui (u) (7.3)

The utility Ui is related to channel’s performance criteria, while the pricing term Pi
is used to penalize a channel for using too large an action (power). Minimizing the
cost function is equivalent to maximizing the net utility, i.e., the difference between
utility and pricing.
In general a pricing mechanism is known to improve the NE efficiency, and a
linear pricing is the simplest one [133]. For such a simple linear pricing term each
channel minimizes the cost function Ji :

Ji (u−i , ui ) = αi ui − Ui (u−i , ui ), ui ∈ [0, umax ] (7.4)

where channel power is limited to be below some umax threshold on transmitter

power. This linear pricing term in the individual channel input signal power can
be interpreted as the “price” a channel pays for using the system resources [133],
reflecting the fact that increasing one channel’s power degrades the OSNR of all
other channels. This price can be set by the network/link and dictated to each chan-
nel. From a system performance point of view, this linear pricing term limits the
interference of this channel on the other ones [134]. Intuitively speaking, a large αi
means that a large penalty is put by the network on the use of power.
With OSNR as a performance criterion, the utility Ui should reflect a channel’s
preference for high OSNR or for low OSNR degradation. We define an OSNR-like
channel utility function Ui (u) that reflects the channel’s preference for maximizing
OSNRi , yi . We consider a utility function appropriately defined for optical net-
works, a generalization of the one used in the wireless case. The following assump-
tions are used.
(A.ii.1) The utility function Ui (u) is a continuously differentiable function in ui ,
monotone increasing and strictly concave in ui .
7.2 Games Without Coupled Constraints 101

Fig. 7.2 Ui vs. yi

(A.ii.2) ui = 0, ui = umax are not solutions to the minimization of the cost func-
tion Ji .
We construct a utility function that satisfies (A.ii.1), (A.ii.2). Note that OSNRi ,
i.e., yi in (7.2), is a strictly increasing function with respect to ui , and tends to 1/Γi,i ,
for infinite channel power. It is apparent that relation (7.2) bears a striking similarity
with the wireless SIR model, [8], even though a different physical mechanism is
present (ASE noise accumulation). The system matrix in the SIR model [8] has a
special structure with equal rows, which is instrumental in obtaining uniqueness
results. In contrast, for optical networks, Γ is a full general structure matrix, with
coupling due to all channels and all spans. Moreover, OSNRi is no longer a linear
function of ui and a direct logarithmic utility function of associated SIR as in the
wireless case cannot be applied. For the general full matrix Γ , one can define a
more general utility function Ui (u) here chosen to be a logarithmic function of the
associated channel’s OSNR, yi (u),
 
yi (u)
Ui (u) = βi ln 1 + ai , ∀i ∈ M (7.5)
1 − Γi,i yi (u)

where ai , βi > 0 are channel-dependent parameters, ai a scaling parameter for flex-

ibility and βi quantifying the desire to maximize the OSNR, respectively.
The utility function Ui defined in (7.5) is monotonically increasing in yi , so that
maximizing utility is related to maximizing channel OSNR, yi . This relationship is
illustrated in Fig. 7.2. Equivalently, using (7.2)
 
ui
Ui (u−i , ui ) = βi ln 1 + ai (7.6)
X−i

where X−i is given as in (7.2), as a function of the full system-matrix Γ . This de-
pendence will be instrumental in the following results. Therefore, the cost function
102 7 Games in Point-to-Point Topologies

to be minimized, (7.4), is given as

 
ui
Ji (u−i , ui ) = αi ui − βi ln 1 + ai (7.7)
X−i

Herein αi is the price set by the network/link and βi is set by each channel, re-
spectively. These act as weighting factors, quantifying the trade-off between pricing
(penalizing a channel for using large power) and the desire to maximize its utility.
From (7.6) it follows immediately that Ui satisfies (A.ii.1), i.e., the utility func-
tion Ui (u−i , ui ) is twice continuously differentiable in its arguments, monotonically
increasing and strictly concave in ui . Using (7.7), it can be shown that there exists a
non-empty interval from which to select βi /αi , such that (A.ii.2) holds. Thus αi , βi
are selected such that an NE solution is an interior point of the action set, and hence
is an inner NE (see remark after Definition 4.1 in Chap. 4). This is what we assume
in the following.
One can note that this is only a particular utility function that has certain nice
properties being logarithmic, in that it allows for closed-form expression of the
Nash equilibrium solution. Other examples can be given such as a linear function in
OSNR. A logarithmic function is analytically useful and moreover is widely used
as a utility function in flow control [7, 67, 139] and power control [3, 6, 8] for
general communication networks. In some cases, the logarithmic utility function is
intimately associated with the concept of proportional fairness [67].
Let us recall from Chap. 4 that a point u∗ is an NE when u∗i solves individ-
ual optimization problem Ji , given all channels on its path have equilibrium power
levels, u∗−i . Existence of an NE solution depends on existence of a common in-
tersection point for all players’ reaction curves, while uniqueness depends on the
particular problem. The following result characterizes the Nash equilibrium (NE)
solution.

Theorem 7.1 Consider G(M, Ωi , Ji ) with individual cost functions Ji , (7.7). This
game admits a unique NE solution u∗ if ai are selected such that

Γi,j < ai , ∀i ∈ M (7.8)
j =i

The optimal NE solution u∗ is inner and

u∗ = Γ(−1 b̃ (7.9)

where Γ( = [Γ(i,j ] and b̃ = [b̃i ] are defined as

#
a, j =i ai βi
Γ(i,j = i b̃i = − n0i
Γi,j , j = i αi

and Γi,j being the link system matrix.

7.2 Games Without Coupled Constraints 103

This result shows that while similar in some respects to wireless case, [8], for
optical networks a more general uniqueness condition is needed for uniqueness of
the NE solution.
2
Proof From (7.4, 7.7) and (A.ii.1) it follows directly that ∂ J2i > 0. Since the cost
∂ui
function Ji is strictly convex in ui , there exists a minimizing u∗i , for any given u−i ,
such that

J u−i , u∗i < J (u−i , ui ), ∀ui = u∗i

on the closed and bounded (compact) set [0, umax ]. Furthermore by (A.ii.2) u∗i is
inner.
To find u∗i we solve the necessary conditions ∂u
∂Ji
i
= 0. From (7.7) one obtains

ai βi
ai u∗i + X−i
∗
= , ∀i (7.10)
αi

which defines the ith player’s reaction curve or reaction function ri .

Note now that a linear reaction function greatly simplifies the closed-form ex-
pression of the NE. From Chap. 4 and Theorem 4.3 in [20], the vector solution of
(7.10) is an NE solution to the m-player game. Use the definition of X−i∗ , (7.2), to

rewrite (7.10) as
 ai βi
ai u∗i + Γi,j u∗j = − n0,i , ∀i
αi
j =i

Equivalently, in matrix form, this is written as

Γ(u∗ = b̃ (7.11)

where matrix Γ( and vector b̃ are defined as Γ( = [Γ(i,j ] and b̃ = [b̃i ] with
#
a, j =i ai βi
Γ(i,j = i b̃i = − n0,i
Γi,j , j = i αi

Therefore a unique NE solution u∗ exists if the matrix Γ( is invertible. Recall that

Γ is a positive-entry matrix. If (7.8) holds, then Γ(, (7.11), is strictly diagonally
dominant; from Gershgorin’s Theorem [59], it follows that Γ( is invertible, and the
unique NE solution is u∗ = Γ(−1 b̃. 

Remark 7.2 For a given Γ , the ai factors can be selected such that the diagonal
dominance condition (7.8) holds on Γ(. If Γ is itself diagonal dominant, then a pos-
sible choice is ai = Γi,i . Once Γ changes (e.g. upon reconfiguration) these factors
can be adjusted to satisfy (7.8).
104 7 Games in Point-to-Point Topologies

7.2.2 Iterative Algorithm

In this section we discuss a decentralized iterative algorithm developed by assuming

that all players follow best response strategy, i.e., based on best response dynamics.
It can be shown that this algorithm converges to the NE solution if the diagonal dom-
inance condition in Theorem 7.1 holds. Other control update laws can be considered
such as gradient dynamics, etc. (see Chap. 4) with similar results. Consider

βi X−i (n)
ui (n + 1) = − , ∀i (7.12)
αi ai
as recursive relation for updating transmitter power level, based on (7.10). Thus
(7.12) corresponds to a parallel adjustment scheme (PUA) based on best response
(BR) (see Chap. 4 and [20]), whereby each player responds optimally to the pre-
viously selected action of the other players. Relation (7.12) requires the total inter-
ference factor X−i , which from (7.2) depends on all channel powers, uj , and all
channel gains, i.e., centralized information. However, using (7.2) one can express
(7.12) in terms of yi , i.e,
 
βi 1 1
ui (n + 1) = − − Γi,i ui (n) (7.13)
αi ai yi (n)

where n denotes the iteration number. This corresponds to a decentralized algorithm

as in [46, 72], since the only information feedback is the individual channel yi ,
which can be measured in real-time, and the channel “gain”, Γi,i .
The following result gives convergence conditions for this algorithm.

Lemma 7.3 If (7.8) holds, then algorithm (7.13) converges to the NE solution.

Proof Let ei (n) = ui (n) − u∗i where u∗ = [u∗i ] is the NE solution. Using (7.10, 7.12,
7.13) yields

1 
ei (n + 1) = − Γi,j ej (n) (7.14)
ai
j =i

so that
  
) )   1  
)e(n + 1))  
= max ei (n + 1) ≤ max 
Γi,j ej (n)
∞ i i ai
j =i

where the fact that Γ is component-wise positive was used.

Using |ej (n)| ≤ e(n)∞ , ∀j , yields
  
) ) ) )
)e(n + 1)) ≤ max 1 Γ i,j
)e(n))
∞ i ai ∞
j =i
7.2 Games Without Coupled Constraints 105

Using (7.8) on the right-hand side of the foregoing yields

) ) ) )
)e(n + 1)) < )e(n)) , ∀n = 0, 1, 2, . . .
∞ ∞

By a contraction mapping argument this shows that e(n) → 0 and u(n) → u∗ . 

Next we present some discussion on parameter selection via pricing strategies.

(a) Proportional Pricing This is the case when pricing parameters αi are selected
to be proportional to the system matrix entry for channel i, Γi,i , i.e., αi = Γi,i ki , and
all βi are equal to 1. It can be shown that the scaling factors ki can be selected such
that all channels achieve some desired OSNR level, γi∗ . Using (7.2) and (7.10) for
the NE solution, one can write
1 ∗ ai 1
u = (Γi,i − ai )u∗i + , ∀i
γi∗ i Γi,i ki
In matrix vector form this yields

Σu∗ = v with Σ = diag(ε1 , . . . , εi , . . . εm ), v = [vi ] (7.15)

where εi = 1
γi∗ + ai − Γi,i , and vi = ai 1
Γi,i ki . Using the NE solution, (7.11), u∗ =
Γ˜ −1 b̃, and b̃ = v − n0 , yields after some manipulation

I − Γ˜ Σ −1 v = n0 (7.16)

assuming that εi = 0. Then if ρ(Γ˜ Σ −1 ) < 1, where ρ is the matrix spectral radius,
equation (7.16) has a unique solution v. Moreover, for εi = ε = 0, using (7.15), one
can rewrite this condition as an upper bound condition on the desired OSNR level
γi∗ , i.e.,
1
γi∗ < , ∀i (7.17)
ρ(Γ˜ ) + (Γi,i − ai )
Therefore, if (7.17) holds v can be uniquely found as in (7.16), and hence the ki
factors can be found such that the desired OSNR level γi∗ is achieved. This corre-
sponds to a centralized pricing strategy, and as in [8], shows the trade-off between
“gain” of the system matrix, in terms of the spectral radius, and the level of OSNR
achieved. Unlike [8], for the general model Γ these parameters and the upper bound
for the desired OSNR level are different for each channel.

(b) Decentralized Pricing For the case when βi can be adjusted individually, one
can show that if βi satisfies the lower and upper bounds
    
1 + (ai − Γi,i )γi∗ αi αi
βi > X −i , β < u a + Γ + n 0 , ∀i
1 − Γi,i γi∗
i max i i,j
ai ai
j =i

then each channel will achieve at least γi∗ , i.e., yi > γi∗ , with ui ≤ umax at each
iteration.
106 7 Games in Point-to-Point Topologies

(c) Minimum OSNR Level Finally, consider the case when all ui = umax and
yi ≥ γmin . Using the OSNR model in Chap. 6, it can be shown that a sufficient
condition for this is
1
γmin ≤ 
maxi∈M ( j ∈M Γi,j )

which shows the trade-off between minimum achievable OSNR, γmin , and the norm
(gain) of system matrix Γ . Unlike the wireless case, [8], the system matrix Γ plays
an essential role, i.e., a condition independent of the system matrix cannot be ob-
tained.
As long as coupling constraints are not considered these point-to-point link re-
sults are directly extendable to end-to-end games in optical network topologies (see
Chap. 8). In the next section we show how distributed links can be treated.

7.3 Games with Coupled Constraints: Indirect Penalty

Approach

As we saw in Chap. 6 in optical networks cascaded optical amplified spans are

present, as well as accumulation and self-generation of optical ASE noise. These
realistic physical features have been considered in OSNR modeling (see Chap. 6).
In order for game methods to be practical, they must incorporate realistic constraints
arising from the underlying network systems. One such important constraint is the
link capacity constraint (see Sect. 6.4.3 in Chap. 6), which is a coupled power con-
straint on channels that share an optical fiber. As we recall, the total power launched
into the fiber needs to be restricted below the nonlinearity threshold [2, 85]. This
constraint is satisfied at intermediary amplifier sites, which are operated in the auto-
matic power control (APC) mode, but not at Txs. Thus unlike the previous section,
herein we consider a Nash game with coupled constraint.
Thus, incorporating the link capacity constraint (6.38), the action space Ω is
#   *

Ω = u∈Ω   
uj − P ≤ 0 (7.18)
j ∈M

This Ω is coupled in the sense that one player’s action affects the feasible action
sets of the other players. The feasible action set for each channel i is the projection
set
#   *

i (u−i ) = ξ ∈ Ωi 
Ω u + ξ − 
P ≤ 0 (7.19)
 j
j ∈M,j =i

The first approach we consider in order to treat such a game is based on incor-
porating the coupled constraint into each cost function, thus in effect an indirect
approach. Part of the results in this section are based mainly on [105, 108, 110].
7.3 Games with Coupled Constraints: Indirect Penalty Approach 107

Let us consider then that for each channel i ∈ M an individual cost function
Ji : Ω → R is assigned which is defined as the difference between a pricing function
Pi : Ω → R and the same utility function as before , Ui : Ω → R:

i − Ui
Ji = P (7.20)

Specifically, the utility function Ui is the same as in (7.5), while the new pricing
function consists of two terms: a linear pricing term (as in Sect. 7.2.1) and a new
regulation (penalty) term, i.e.,

i (u−i , ui ) = αi ui + 1
P  , ∀i ∈ M, (7.21)
−
P j ∈M uj

where αi > 0 is a pricing parameter determined by the system.

The regulation term is constructed by considering the link capacity constraint.
It penalizes any violation of the constraint as following: the regulation term tends
to infinity when the total power approaches the total power target P, so the pricing

function Pi (u−i , ui ) increases without bound. Hence the system resource is pre-
served by forcing all channels to decrease their input powers and indirectly satisfies
the link capacity constraint.
Thus the m-player Nash game is defined in terms of the new cost functions
Ji (u−i , ui ), i ∈ M, played within the action space Ω. This game is denoted by
G(M, Ω i , Ji ).

7.3.1 Nash Equilibrium Solution

A solution u∗ of G(M, Ω i , Ji ) is called an NE solution in the sense of Definition 5.2

for coupled games. If in addition, such a solution is not on the boundary of the action
space Ω, then it is an inner NE solution.
Notice that
1 
 →∞ −
as ui → P uj
−
P j ∈M uj j ∈M,j =i

Thus the points on the hyperplane {u ∈ Rm | j ∈M uj = P } are not NE solutions
of G(M, Ω i , Ji ). In addition, since ui = 0 means channel i is inactive in the link,
an NE solution u∗ with a zero component, say, u∗1 = 0, of G(M, Ω i , Ji ) implies
that channel 1 does not have any effect on the game. So in this case, the game
is equivalent to the one in which (m − 1) channels play and the NE solution to
the (m − 1)-player Nash game does not have a zero component. In this sense, we
assume that any NE solution u∗ to the m-player OSNR Nash game does not have
zero components, thus is always inner.
The following result provides sufficient conditions for existence and uniqueness
of an inner NE solution.
108 7 Games in Point-to-Point Topologies

i , Ji ) admits a unique inner NE solution u∗ if

Theorem 7.4 (See [105]) G(M, Ω
∀i ∈ M,

ai > Γi,j , (7.22)
j =i,j ∈M

βmin
βi <  Γj,i
, (7.23)
j =i,j ∈M aj
+
, 
, βi Γj,i
αi >αmax - , (7.24)
βj aj
j =i,j ∈M

where βmin = minj ∈M βj and αmax = maxj ∈M αj .

Proof (Existence) The action space Ω is a compact and convex set with a non-
empty interior. Each cost function Ji (u−i , ui ) is continuous and bounded and the
first and second partial derivatives of J
i (u−i , ui ) with respect to ui are well defined
on Ω except the hyperplane {u ∈ Rm | j ∈M uj = P }, given as

∂ Ji (u) 1 βi ai
= αi +  − , ∀i ∈ M (7.25)
∂ui 
(P − j ∈M uj )2 X−i + ai ui

∂ 2 Ji (u) 2 βi ai2

=  + , ∀i ∈ M (7.26)
∂u2i  − j ∈M uj )3 (X−i + ai ui )2
(P

∂ 2 Ji (u)
It follows that is positive and therefore Ji (u−i , ui ) is strictly convex in ui .
∂u2i

Since each point on the hyperplane {u ∈ Rm | j ∈M uj = P } is not an NE solution,
by Proposition 5.3, the game with the action space Ω admits an NE solution, which
is the inner NE solution of G(M, Ω i , Ji ).
Since an NE solution is inner, it follows from Proposition 4.6 that (4.6),
∂ Ji (u)  
∂ui = 0, ∀i ∈ M, holds. The vector form is ∇ J (u) = 0, where ∇ J (u) is defined
as in (A.3).
(Uniqueness) The uniqueness part is proved by contradiction and involves rather
long manipulations. The reader is referred to [105, 110] for the detailed steps. 

Based on the results in Theorem 7.4, let us briefly discuss parameter selection
strategies. The link sets fixed channel prices and each channel decides its willing-
ness βi to obtain a satisfactory OSNR, yi , denoted by βi (yi ). From the necessary
i (u)
condition, ∂ J∂u i
= 0, using (7.25) one obtains βi as a function of yi to be

αi  X−i + ai ui (yi )
βi (yi ) = X−i + ai ui (yi ) +  , (7.27)
ai 
ai (P − j ∈M,j =i uj − ui (yi ))2
7.3 Games with Coupled Constraints: Indirect Penalty Approach 109

where
X−i
ui (yi ) =
1/yi − Γi,i
For a given lower OSNR bound 
γi , one can show that if βi is adjusted to satisfy the
lower bound
αi 1 + (ai − Γi,i )
γi
βi > X−i
ai 1 − Γi,i 
γi
(1 − Γi,i γi )(1 + (ai − Γi,i )
γi )
+   X−i ,
−
ai (P 
j ∈M,j =i uj − (P − j ∈M,j =i uj Γi,i + X−i )
γi )2
(7.28)

so that each channel has yi > 

γi .
Theorem 7.4 shows the existence of a unique inner NE solution under certain
conditions, but without giving an explicit expression of this solution. Some prop-
erties of the NE solution, which directly lead to the development of two iterative
algorithms towards finding the NE solution numerically, are characterized in the
following.
i (u)
Consider the necessary conditions ∂ J∂u i
= 0, i.e.,

1 βi ai
αi +  = , ∀i ∈ M (7.29)
−
(P j ∈M uj )
2 X−i + ai ui

The unique inner NE solution of G(M, Ω i , Ji ) satisfies (7.29).

Alternatively, the necessary condition can be written in terms of reaction func-
tions. Recall that the action space Ω is compact and convex and the cost function
Ji (u−i , ui ) is twice continuously differentiable over Ω in its arguments and strictly
convex in ui . A reaction function ri for channel i is defined as

ri (u−i ) := arg min Ji (u−i , ui ), ∀i ∈ M (7.30)

i (u−i )
ui ∈Ω

Then it follows that for every given u−i , ri (u−i ) is single-valued. Furthermore, by
Berge’s Maximum Theorem (Theorem A.17), ri (u−i ) is continuous. Thus, there is
a unique point, ui = ri (u−i ) for every i ∈ M. In vector form,

u = r(u), (7.31)

where r := [r1 (u−1 ), . . . , rm (u−m )]T .

As discussed in Chap. 4, an NE solution u∗ ∈ Ω satisfies component-wise

u∗i = ri u∗−i , ∀i ∈ M, (7.32)

or u∗ = r(u∗ ). Unlike the case with no coupling constraints in Sect. 7.2.1 where a
closed-form expression was found, here the reaction function ri is highly nonlinear
and the NE solution is analytically intractable.
110 7 Games in Point-to-Point Topologies

The reaction function is in fact the implicit solution of (7.29). Notice that over
Ω, the left-hand side of (7.29) is a monotonically increasing function with respect
to ui , while the right-hand side of (7.29) is monotonically decreasing. Therefore,
for every i ∈ M, there exists a unique intersection between the left-hand side and
the right-hand side of (7.29), which is the NE solution u∗ . One can get insights into
the NE solution’s characteristics by studying the reaction function.

Lemma 7.5 (See [110]) The reaction function r(u) has the following properties:
– r(u) ≥ 0 (non-negativity);
– If u > u , then r(u) < r(u ) (monotonicity).

It follows from Lemma 7.5 that the reaction function r(u) is not standard. As
defined in [163], a function I (p) is standard if for all p ≥ 0, the following properties
are satisfied: I (p) > 0; if p ≥ p , then I (p) ≥ I (p ); for all α > 1, αI (p) > I (αp).
Such a property typically helps in developing iterative algorithms.

7.3.2 Iterative Algorithm and Convergence Analysis

In this section we present two iterative algorithms towards finding the unique inner
NE solution: a parallel update algorithm (PUA) and a gradient algorithm (GA). PUA
is developed using the reaction functions and GA is developed based on the gradient
descent method.
In the parallel update algorithm each channel i updates its input power ui based
on its associated reaction function ri (u−i ). Let
   
u(n) := u(0), u(1), . . . , u(n), . . .

denote the sequence of channel input power vectors, where u(0) is the initial chan-
nel input power vector. Then at each iteration time (n + 1), channel input power
ui (n + 1) is updated by

(PUA) ui (n + 1) = ri u−i (n) , ∀i ∈ M, (7.33)

or equivalently, in a vector form,


u(n + 1) = r u(n) (7.34)

If the reaction function r(u) were standard, PUA would be the standard power
control algorithm [163] and the convergence results in the synchronous case could
be directly applied to PUA. However, this is not the case here and convergence is
really difficult to prove. Results have been obtained only for the case of 2-players
(m = 2) and extension to m-player case, m > 2 is an open problem [110].
7.3 Games with Coupled Constraints: Indirect Penalty Approach 111

Remark 7.6 PUA may not converge when more than two channels (m > 2) exist.
The intuitive reason is that each channel updates its optical power only based on
instant costs and parameters, ignoring future implications of its action. Therefore,
at some iterations the total optical power of all other channels will exceed the target
power P  when m > 2. To overcome these power fluctuations, one can use a relaxed
PUA, in which a relaxation parameter is used to determine the step size that each
channel takes towards finding the NE solution at each iteration step. For example,
(7.33) can be modified as

ui (n + 1) = (1 − μi )ui (n) + μi ri u−i (n) ,

where the coefficient 0 < μi < 1 and it can be shown that the relaxed PUA converges
if μi is restricted.

PUA is developed by using the reaction function r(u) but convergence can be
shown only for m = 2. As alternative a gradient algorithm (GA) can be designed
based on the gradient descent method [24]. Convergence of this algorithm can be
proved even when m > 2. For convenience, particularly for the simplification of
convergence proof, we follow the approach in [107, 109] and develop the algorithm
in the continuous-time domain.
We consider a model where each channel uses a gradient algorithm (GA) to up-
date its power, given as
  
dui ∂ Ji (u−i , ui )
u̇i (t) = = −μ , ∀i ∈ M, (7.35)
dt ∂ui (t)
where t is the continuous-time variable and the coefficient μ > 0.
Using (7.3) and (7.25), one can rewrite (7.35) as
 
1 βi ai
(GA) u̇i (t) = −μ αi +  −
 − j ∈M uj (t))2 m Γ˜i,k uk (t)
(7.36)
(P k=1

where
#
Γi,k , k = i
Γ˜i,k =
ai , k=i
Recalling the definition of OSNR, one can rewrite (7.36) as
 
1 βi ai
u̇i (t) = −μ αi +  − (7.37)
 − j ∈M uj (t))2 ( 1 + ai − Γi,i )ui (t)
(P yi (t)

Hence, at each iteration, individual channels need only the sum of power of all
channels and local measurements, namely, its own power and current OSNR level.
Thus the iterative algorithm (7.37) is distributed.
Next let us define a set Ω δ , which is a subset of the action space Ω. We slightly
modify the bounded set Ωi = [0, umax ] to be
(i = [umin , umax ],
Ω ∀i ∈ M
112 7 Games in Point-to-Point Topologies

(i is given by Ωδ . The set Ω δ is

where 0 < umin < umax . The Cartesian product of Ω
defined as
#   *

Ω δ = u ∈ Ωδ   
uj − P ≤ 0
j ∈M

We set umin to be sufficiently small such that Theorem 7.4 holds, i.e., u∗i > umin for
all i ∈ M and

∂ Ji 
(u−i , ui ) < 0, ∀u−i , ∀i ∈ M (7.38)
∂u i ui =umin

Notice that because of the capacity constraint, the following statement is always
true:

∂ Ji 
(u−i , ui ) 
> 0, ∀u−i , ∀i ∈ M (7.39)
∂ui − j ∈M,j =i uj
ui =P

Thus the set Ω δ is invariant under the algorithm (7.36). That is, the trajectory
remains inside Ω δ . It follows that the trajectory lies in Ω if the initial state is
in Ω δ . Moreover, the equilibrium of (7.36) in Ω δ is the unique NE solution u∗
of G(M, Ω i , Ji ).
The following result proves the convergence of algorithm (7.36).

Theorem 7.7 For G(M, Ω i , Ji ), let the initial condition u(0) ∈ Ω δ . Then the up-
date scheme (7.35) converges to the NE solution u∗ if
  2 
2P
amin > max Γj,i , (7.40)
umin i∈M
j ∈M, =i

βi > S · βmax , (7.41)

where amin = minj ∈M aj , βmax = maxj ∈M βj and

 2 
2P maxi∈M j ∈M,j =i Γj,i
S= (7.42)
umin amin

Proof The proof is based on a Lyapunov approach. Let φi (u) := u̇i (t), where u̇i (t)
is defined in (7.35) or (7.36), and define a candidate Lyapunov function,

1  2
V (u) := φi (u) (7.43)
2
i∈M

Note that V (u) is restricted to the set, Ω δ . Because of the uniqueness of the NE
solution u∗ , φi (u) = 0, ∀i ∈ M, if and only if u = u∗ . Therefore, V (u) is strictly
positive for all u = u∗ .
7.3 Games with Coupled Constraints: Indirect Penalty Approach 113

From (7.36), taking the second derivative of ui with respect to time t, yields
 2 βi ai Γ˜i,j

üi (t) = −μ  +  φj (u) (7.44)
 − j ∈M uj (t))3 (
(P ˜ 2
j k∈M Γi,k uk (t))

It is obvious that φ̇i (u) = üi (t).

Taking the derivative of V (u), (7.43), with respect to time t, yields

V̇ (u) = φi (u) · φ̇i (u)
i∈M
  2 βi ai Γ˜i,j

= −μ φi (u)  +  φj (u),
−
(P j ∈M uj (t))
3 ( k∈M Γ˜i,k uk (t))2
i∈M j ∈M
(7.45)

where (7.44) is used. For simplicity, let

2 βi ai
a(t) =  , bi (t) =  ,
−
(P j ∈M uj (t))3 ( k∈M Γ˜i,k uk (t))2

θij (t) = a(t) + bi (t)Γ˜i,j

Therefore, (7.45) is rewritten as

  
V̇ (u) = −μ a(t) + bi (t)Γ˜i,j φj (u)φi (u)
i∈M j ∈M
 
= −μ φi (u) θij (t)φj (u)
i∈M j ∈M

= −μφ T (u)Θ(t)φ(u), (7.46)

where vector φ = [φi ] and matrix Θ(t) = [θij (t)]m×m are appropriately defined. If
Θ(t) is uniformly positive definite then V̇ (u) in (7.46) is negative definite, and the
system is asymptotically stable by Lyapunov’s stability theorem [69]. This is the
main idea of the proof and the reader is referred to [107] for the detailed steps. Then
φi (u(t)) = u̇i (t) → 0, ∀i ∈ M which implies that ui (t) converges to the unique NE
solution point u∗ . 

A gradient algorithm has been studied for stability in a congestion control game
[4, 5] where a special structure matrix is automatically uniformly positive definite in
the proof. In contrast, herein the corresponding matrix Θ(t) is more general. System
dependent conditions are required for Θ to be positive definite.
Compared the conditions (7.40) and (7.41) for the stability of GA with the suf-
ficient conditions (7.22)–(7.24) for the existence of a unique NE solution, it read-
ily follows that βi is not only upper-bounded by (7.23), but also lower-bounded
by (7.41).
114 7 Games in Point-to-Point Topologies

As we have seen in this section a relatively tedious process has to be followed to

treat coupled constraints indirectly, with respect to both NE existence and unique-
ness conditions, as well as with respect to NE computation (algorithm). In the next
section we will use a direct approach to treat these constraints, which, as we will
see, offers reduced complexity.

7.4 Games with Coupled Constraints: Lagrangian Pricing

Approach

7.4.1 Lagrangian Pricing and Duality Extension

In this section we consider Nash games with coupled constraints and their solv-
ability by a direct approach based on Lagrangian extension. Specifically we apply
directly the procedure developed in Chap. 5. We note that this direct approach has
advantages compared to the indirect one in the previous section. As we saw the indi-
rect, penalty-based approach does not allow for an analytically tractable NE solution
and developing decentralized algorithms is not immediate.
As in Sect. 7.2, each channel i ∈ M minimizes the original individual cost func-
tion Ji : Ω → R, with Ji defined in (7.4), not the penalized cost Ji as used the
previous section. The cost Ji is composed of a linear pricing and a channel utility
Ui related to OSNR maximization defined as before. Unlike Sect. 7.2, where cou-
pled constraints were not considered, here the action set is coupled as in (7.18),
i.e.,
 

Ω = u ∈ Ω | 1T u ≤ P (7.47)

and the action set of channel i ∈ M is the projection of Ω on channel i’s direction,
namely as in (7.19):
  m 


i (u−i ) = xi ∈ Ωi  
Ω uj + xi ≤ P (7.48)

j =i

We denote the game by G(M, Ω i , Ji ), which is in the class of m-player Nash games
with coupled utilities and coupled constraints. It follows from Sect. 7.2 that Ji is
continuously differentiable in its arguments and convex in ui . Note that the over-
all coupled action space Ω is compact and convex as well. Then from Proposi-
tion 5.3, G(M, Ω i , Ji ) admits an NE solution. Moreover, due to the coupled con-
straint (7.47), solving directly for an NE solution of this game requires coordination
among possibly all channels and is impractical as seen in the previous section.
In the following, we use Lagrangian extension and decomposition results in
Chap. 5 as a natural way to obtain a hierarchical decomposition and compute an NE
7.4 Games with Coupled Constraints: Lagrangian Pricing Approach 115

solution of G(M, Ω i , Ji ). For this game, using the notations as defined in Chaps. 4
and 5, consider the separable NG cost function J((u; x), (4.7),


m
J((u; x) = Ji (u−i , xi ),
i=1

the separable augmented constraint (

g (u; x), (5.8),


m
(
g (u; x) = g(u−i , xi ),
i=1

( x; μ), (5.11),
together with the augmented Lagrangian function L(u;

( x; μ) = J((u; x) + μT (
L(u; g (u; x)

and the dual cost function D(μ), (5.17),


( u∗ ; u∗ ; μ ,
D(μ) = L (7.49)

( x; μ). In a fixed-point
where u∗ is such that u = u∗ satisfies u = arg minx∈Ω L(u;
notation,
 
D(μ) = min L(u;( x; μ)  ,
x∈Ω (
arg min L=u
x∈Ω

or
 
( x; μ) 
D(μ) := min L(u;
x∈Ω x=u

Then G(M, Ω i , Ji ) is related to a constrained minimization of J(, (4.7), (5.8), with

respect to the second argument x, which admits a fixed-point solution. In particular
individual components of a solution u∗ to this constrained minimization constitute
an NE solution to G(M, Ω i , Ji ) in the sense of Definition 5.2. From Remark 5.6,
∗
we know that u is obtained by first minimizing L(u; ( x; μ) with respect to x, x(u).
The next step involves finding a fixed-point solution x(u∗ ) = u∗ which depends
on μ, u∗ (μ). An NE solution–Lagrange multiplier pair (u∗ (μ∗ ), μ∗ ) is obtained if
μ∗ T g(u∗ (μ∗ )) = 0 and μ∗ ≥ 0.
Thus D(μ) is related to the minimization of the associated Lagrangian func-
( x; μ). For G(M, Ω
tion L(u; i , Ji ) with convex coupled constraints, Proposition 5.9
yields a decomposition into a lower-level Nash game with no coupled constraints
and a higher-level problem for pricing. Thus based on Proposition 5.9 applied to the
OSNR game G(M, Ω i , Ji ), with separable NG cost function and linear constraints,
one has the following decomposition result.

Proposition 7.8 Consider the coupled OSNR Nash game G(M, Ω i , Ji ) with cost
functions Ji (u−i , ui ), (7.4), subject to the linear constraint (7.47), i.e., over Ωi .
116 7 Games in Point-to-Point Topologies

Then the dual cost function D(μ), (7.49), can be decomposed as


m
∗  m

D(μ) = ∗
Li u−i , ui (μ), μ + 
μ eT u∗−i − P (7.50)
i=1 i=1

where u∗ (μ) = [u∗i (μ)] is an NE solution to the Nash game G(M, Ωi , Li ) with cost
functions Li (7.51) and no coupled constraints.

Li (u−i , xi , μ) = Ji (u−i , xi ) + μxi , i∈M (7.51)

Proof Each cost function Ji , (7.4), is continuously differentiable and convex in ui

and the constraints are linear so that one can apply Proposition 5.9. The linear con-
straints (7.47) are rewritten in a two-argument form


g(u−i , xi ) = eT u−i + xi − P (7.52)

where e = [1, . . . , 1]T is the (m − 1) × 1 all ones vector. Then by using Proposi-
tion 5.9, D(μ) can be decomposed as


m

D(μ) = Li u∗−i (μ), u∗i (μ), μ
i=1

where using (7.52) Li is given here as



Li (u−i , xi , μ) = Ji (u−i , xi ) + μxi + μ eT u−i − P (7.53)

Recall that in D(μ) in Proposition 5.9 one has to minimize first with respect to
xi on the right-hand side, and then solve for a fixed-point solution. From (7.53)
it can be seen that only the first two terms depend on xi . Hence, substituting for
Li (u−i , xi , μ), (7.53), on the right-hand side of D(μ) and isolating the terms that
are independent of xi , yields


m 
m

D(μ) = min Li (u−i , xi , μ)|xi =ui + 
μ eT u−i − P
xi ∈Ωi
i=1 i=1

where Li is defined as in corollary statement (7.51). A fixed-point solution

u∗ = [u∗i ] to the set of m optimizations on the right-hand side of the foregoing
is an NE solution to the Nash game with cost functions, Li , (7.51), and the last part
of the claim follows. 

Proposition 7.8 leads to a hierarchical decomposition of G(M, Ω i , Ji ) into a

lower-level modified Nash game G(M, Ωi , Li ) with cost functions Li (7.51) and
no coupled constraints, and a higher-level optimization problem used for coordina-
tion. This decomposition is computationally simpler as shown below. For a given
7.4 Games with Coupled Constraints: Lagrangian Pricing Approach 117

pricing μ, the lower-level game admits a closed-form explicit solution. Specifically

using (7.51, 7.4, 7.7) we see that Li satisfies
 
ui
Li (u−i , xi , μ) = (αi + μ)ui − βi ln 1 + ai
X−i

i.e., Li is the same as Ji , (7.7) for αi replaced by αi + μ, ∀i. Therefore, for each
given μ, the NE solution u∗ (μ) to the lower-level game G(M, Ωi , Li ) with cost Li
is unique and can be obtained as in Theorem 7.1 as
  
u∗ (μ) = Γ(−1 Diag 1./(α + μ) b0 − n0 (7.54)

where b0 = [ai βi ], n0 = [n0i ] and Diag[1./(α + μ)] = Diag([1/(αi + μ)]). This is

formalized in the following result.

Proposition 7.9 For each given μ ≥ 0, G(M, Ωi , Li ) admits an NE solution if ai

in Li (7.51) satisfies

Γi,j < ai , ∀i ∈ M (7.55)
j =i

Then the inner NE solution u∗ (μ) is unique and given as

u∗ (μ) = Γ(−1(
b(μ),

where Γ( = [Γ(i,j ] and (

b(μ) = [(
bi (μ)] are defined as
#
( a j = i, ( ai βi
Γi,j = i bi (μ) = − n0i , (7.56)
Γi,j j = i, αi + μ

with Γi,j being the system matrix.

Proof The result can be proved by applying Proposition 4.5. We rewrite Li (7.51)
as
 
ai ui
Li (u−i , ui , μ) = (αi + μ)ui − βi ln 1 + 0  , ∀i ∈ M (7.57)
ni + j =i Γi,j uj

It can be seen that for any given μ ≥ 0, Li is jointly continuous in all its arguments
and
∂ 2 Li
> 0, ∀ui ∈ Ωi
∂u2i

It follows that Li is strictly convex in ui . Recall that for each i ∈ M, Ωi is a closed,

bounded, and convex subset of R. Then by Proposition 4.5, G(M, Ωi , Li ) admits
an NE solution.
118 7 Games in Point-to-Point Topologies

∂Li
An inner NE solution can be found by solving the necessary conditions, ∂ui = 0.
From (7.57), one can obtain
 ai βi
ai u∗i (μ) + Γi,j u∗j (μ) = − n0i , ∀i ∈ M
αi + μ
j =i

Equivalently, in a matrix form, this is written as

Γ( · u∗ (μ) = (
b(μ), (7.58)

where matrix Γ( and vector ( b(μ) are defined in (7.56). Therefore a unique solution
of (7.58) exists if the matrix Γ( is invertible. Notice that Γ( is a positive-entry matrix.
If (7.55) holds, then Γ( is strictly diagonal dominant. From Gershgorin’s Theorem
(Theorem A.7), it follows that Γ( is invertible and a unique solution of (7.58) exists,

u∗ (μ) = Γ(−1 · (
b(μ),

which is an inner NE solution of G(M, Ωi , Li ). 

Next, based on the explicit solution (7.54) and on pricing coordination at the
higher level, a recursive hierarchical algorithm is discussed. By Theorem 5.8 applied
to the coupled OSNR game G(M, Ω i , Ji ) with costs Ji and coupled constraints
∗ ∗
(7.47), (u , μ ) is an optimal NE solution–Lagrange multiplier pair if and only if
u∗ is NG-feasible,


m
,
u∗i (μ) ≤ P u∗i ∈ Ωi , i ∈ M (7.59)
i=1

μ∗ ≥ 0, μ∗ ( m ∗ 
i=1 ui − P ) = 0 (complementary slackness condition) and the La-
grangian optimality condition
 
( u; x; μ∗ 
u∗ = arg min L (7.60)
x∈Ω x=u

holds. By Proposition 7.8 and (7.50), note that u∗ (μ) solving (7.60) can be found as
an NE solution to the modified Nash game G(M, Ωi , Li ) with costs Li (7.51), with
no coupled constraints. For every given price μ, this NE solution u∗ (μ) is unique
as in (7.54). Furthermore, from (7.54) it is seen that all components of u∗ (μ) de-
crease with μ. One can exploit the linear constraint and adjust the price μ to satisfy
the slackness condition. Instead of maximizing D(μ), the optimal price μ∗ can be
obtained such that the slackness condition holds, i.e.,as the point of interception
between the curve representing total power, u∗T (μ) = m ∗ 
i=1 ui (μ), with the level P
(Fig. 7.3). This method has the interpretation of a coordination mechanism. The link
as the coordinator sets the price at the optimal value μ∗ . The channels respond by
adjusting their power levels to u∗i (μ∗ ), which minimizes their own cost.
7.4 Games with Coupled Constraints: Lagrangian Pricing Approach 119

Fig. 7.3 Graphical plot of

total power versus price

7.4.2 Iterative Algorithm

A hierarchical adjustment algorithm for both coordinating link price (higher-level)

and channel powers (lower-level) is discussed.

Link Algorithm Every K iterations of the channel algorithm, the new link price
is computed based on the received total power for all channels in the link uT (K) =
μ
j =1 uj (K) as
 
μ(k̄ + 1) = μ(k̄) + η  +
uT (K) − P (7.61)

where k̄ is the link iteration number, η is the step-size and [z]+ = max{z, 0}.
This simple price update based on Fig. 7.3 requires only measurement of total
power. Moreover it corresponds to a gradient descent technique if link price is ad-
justed slower than channel powers. At the higher level, μ(k̄) acts as a coordination
signal that aligns individual optimality with the system constraint, (7.47) or (7.59).
This is the new price given to the channels, who repeat K iterations of the following
algorithm.

Channel Algorithm Based on a pricing μ(k̄) from the link, the optimal channel
power vector u∗ (μ(k̄)) can be found explicitly as in (7.54). This requires global
centralized information. However, the following iterative update algorithm can be
used:
 
βi 1 1
ui (n + 1) = − − Γi,i ui (n) (7.62)
αi + μ(k̄) ai yi (n)
where n is the channel iteration number. This is again a decentralized algorithm,
since the only information feedback is the individual channel yi , which can be mea-
sured in real-time, and the channel “gain”, Γi,i . For fixed μ this algorithm converges
to the optimal NE solution (7.54). Convergence of the combined algorithms will be
proved later on in the context of network topologies (see Chap. 8). Even if the op-
timal solution is coupled as in (7.54), it can be iteratively computed by using the
decentralized algorithm (7.62). Thus individual channels do not have to coordinate
with other channels at the lower-level game.
120 7 Games in Point-to-Point Topologies

7.5 Notes

In this chapter, we presented a Nash game framework towards OSNR optimiza-

tion in point to point links. We developed first the basic game formulation, with-
out capacity constraints (Sect. 7.2). Then we considered indirectly the link capac-
ity constraint (Sect. 6.4.3) by adding a penalty term to each channel cost function
(Sect. 7.3). Conditions for the existence of a unique inner NE solution to this Nash
game were given. The NE solution is analytically intractable and developing itera-
tive algorithms is not immediate. We have studied the properties of the NE solution
and presented two distributed algorithms: PUA and GA, towards finding the unique
inner NE solution. The algorithms use only local measurements and the current load
of the link.
In the second part, Sect. 7.4, we applied directly the results in Chap. 5 to a Nash
game with coupled constraints in optical point-o-point links. The duality approach
offered a natural way to hierarchically decompose the coupled OSNR Nash game
into a lower-level Nash game, with no coupled constraints, and a higher-level link
problem. Moreover, the lower-level Nash game is analytically tractable and its solu-
tion can be found iteratively, decentralized with respect to channels. This approach
led to a solution that made full use of the input power resource. We discuss extension
of these approaches to general network configurations in the next chapter.
Chapter 8
Games in Network Topologies

Abstract This chapter provides approaches on how to deal with games in more
complicated network topologies, starting from the basic games in single point-to-
point WDM fiber links studied in Chap. 7. The multi-link topologies studied are
representative for selected paths extracted from a mesh configuration in which no
closed loops are being formed by channel optical paths. In network configurations,
coupled constraints are propagated along fiber links and constraint functions be-
come complicated from the end-to-end point of view. The non-convexity introduces
additional complexities for analysis. In this chapter, we present a partition approach.
More precisely, we partition the general multi-link structure into stages each stage
being a single link. Then we formulate a partitioned Nash game for general multi-
link topologies composed of ladder-nested stage Nash games. We also show that
convexity is ensured in single-sink topologies, so that a partition approach could be
based on single-sink stages.

8.1 Introduction
We have seen that in their most complete form games in optical networks belong
to the class of m-player games with coupled utilities and constraints. For single
point-to-point link topologies these were studied in Chap. 7. The purpose of this
chapter is to treat multi-link and mesh topologies. In fact end-to-end games without
coupled constraints are direct extensions of the point-to-point games in Chap. 7,
with the appropriate Γ network system matrix (as in Chap. 6) replacing the link
system matrix. Because of this we shall not consider them here. Instead we shall
focus on how to deal with games with coupled constraints in network topologies.
When coupled constraints are considered in network configurations, constraint
functions become complicated from an end-to-end point of view. This is due to
power propagation and constraint propagation along fiber links as shown in Exam-
ple 6.12 in Chap. 6. The approach presented in Sect. 7.3 of Chap. 7 cannot easily be
extended to such configurations. One reason is the difficulty in building an appro-
priate cost function that needs a complicated penalty pricing function term for the
constraints. On the other hand the Lagrangian extension and decomposition results
as developed in Sect. 7.4 of Chap. 7 provide a good path to follow. One thing to keep
in mind is that the approach can be applied under convex constraints. Since convex-

L. Pavel, Game Theory for Control of Optical Networks, 121

Static & Dynamic Game Theory: Foundations & Applications,
DOI 10.1007/978-0-8176-8322-1_8, © Springer Science+Business Media New York 2012
122 8 Games in Network Topologies

ity may not be preserved across the network (see Example 6.12), this is unlike ca-
pacity constraints in flow control [21, 68]. This non-convexity introduces additional
difficulties in directly applying the approach to network topologies from an end-to-
end point of view. A possible approach is a partitioned game formulation and this
is the approach we present. The main idea is that a general multi-link structure can
be partitioned into stages whereby each stage is a γ -link or a single-sink structure,
for which convexity is satisfied. A partitioned Nash game can be formulated for a
general multi-link topology as being composed of o series of ladder-nested stage
Nash games. Stage Nash games are played sequentially and solutions are intercon-
nected. Based on Lagrangian extension and decomposition results, a hierarchical
iterative algorithm towards computing a solution of the partitioned Nash game can
be developed.
The remainder of this chapter is organized as follows. As a building block for the
partitioned game approach, in Sect. 8.2 focuses on games in a γ -link topology. This
is a simple network topology where power is adjustable not only at the transmitter
sites but at all nodes (see Sect. 6.4.2). For simplicity we consider games with no cou-
pled constraints in a γ -link so that the partitioned game approach is shown better.
Then in Sect. 8.4 a partitioned game is presented for the case with constraints and
general mesh and multi-link topologies. This follows the approach in Sect. 8.2 for
the partitioned game where each stage game is a link game. Coupled capacity con-
straints are treated as in Sect. 7.4 for each stage game. Alternatively in Sect. 8.3 we
show that for single-sink topologies convexity is maintained and thus approach in
Sect. 7.4 can be directly applied to an end-to-end game. The material in this chapter
is mostly based on [106, 112].

8.2 Games in γ -Link Topologies

As a stepping stone for solving games with coupled constraints in general network
topologies in this section we consider a relatively simple topology of interconnec-
tion of L point-to-point links linked via network elements such as OXC or dynamic
gain equalizers [126]. Thus parameters are adjustable at all nodes that have these
elements (see Sect. 6.4.2). This allows us to develop a partitioned game approach
into stages that will be used later on in multi-link and mesh topologies. For sim-
plicity we treat only the case with no coupled constraints. We will consider coupled
constraints from the following sections onwards (Sects. 8.3 and 8.4).

8.2.1 Partitioned Game Formulation

In this section we study γ -link topologies with a set of links L = {1, . . . , L} where
channel powers are adjustable not only at Txs but also at some K bifurcation nodes
via the adjustment factors γ (see Fig. 6.11, Sect. 6.4.2 and Fig. 8.1 below).
8.2 Games in γ -Link Topologies 123

Fig. 8.1 A γ -link topology

We assume for simplicity that this is possible at each link so that K = L. We

also assume for simplicity that the same set M = {1, . . . , m} of channels is trans-
mitted over all links. We partition the multi-link structure into L stages depending
on the nodes where channels are adjusted. The case where this is possible only at
fewer nodes, i.e., K < L is treated in [121]. Each stage has a point-to-point struc-
ture and stages are ladder-nested. A nested structure is an information structure in
which each player has an access to the information acquired by all his precedents,
i.e., players situated closer to the beginning of the decision process. If the differ-
ence between the information available to a player and his closer precedent involves
only his actions, then the structure is ladder-nested [77]. The ladder-nested structure
enables decomposition that in turn results in recursive procedure for its solution.
In effect we define a partitioned Nash game where we exploit the structure of
each stage and the ladder-nested form of the game. The game is played in the fol-
lowing way. Channels transmitted over the same stage compete for input power
resource. Actions of competition are taken in a fixed order among stages and so-
lutions are interconnected. In the following we will formulate a partitioned Nash
game composed of ladder-nested stage Nash games. We will also discuss iterative
computation of equilibria and develop a three-level hierarchical algorithm.
First, we make some new notations for each stage. A multi-link structure of L
links is partitioned into L stages, L = {1, . . . , L} and for each stage l ∈ L, denote
by ul = [ui,l ] and pl = [pi,l ], i ∈ M = {1, . . . , m} the input and output signal power
l = [ni,l ] and nl = [ni,l ] the input and
vector, respectively. Similarly, denote by nin in out out

output noise power vector, respectively. Sometimes we write ul as ul = (ul,−i , ui,l )

124 8 Games in Network Topologies

Fig. 8.2 A γ -link topology (recursive interconnection)

with the vector ul,−i obtained by deleting the ith element from ul . Let γl = [γi,l ]
denote the vector of channel adjustments at stage l such that

ui,l = γi,l pi,l , ∀i ∈ M

where stage l is the stage precedent to stage l for channel i, i.e., l = l − 1 (Fig. 8.2).
The adjustment parameter γi,l is bounded within [γmin , γmax ]. Each stage is com-
posed of a link. Thus the OSNR for channel i, i ∈ M, at the output of stage l, l ∈ L,
is defined as
pi,l
yi,l = out .
ni,l
Based on this partition, the OSNR of channel i at the output of stage (link) l, can be
obtained from Proposition 6.7 as
ui,l
yi,l =  , (8.1)
nin
i,l + j ∈M Γli,j uj,l

where Γl = [Γli,j ] is the link (stage) system matrix. Based on (8.1), we define

1 1 1  uj,l
:= − = Γli,j (8.2)
δQi,l yi,l yi,l ui,l
j ∈M

where stage l is the link precedent to stage l for channel i. In this case of γ -
link topologies this is simply l = l − 1. It follows that δQ1i,l measures the OSNR
degradation of channel i from stage l to stage l. Thus, instead of maximization of
OSNR from Tx to Rx, one can consider minimization of OSNR degradation be-
tween stages.
We formulate a partitioned Nash game, composed of lower-level ladder-nested
stage Nash games. On stage l, a Nash game is played with each channel attempting
to minimize its individual cost over m channels. Games on stages are played in
sequence and solutions are interconnected as will be shown below.
We first specify channel individual cost function Ji,l in the lower-level stage Nash
game. Channel individual cost function Ji,l has a similar form as Ji (7.3) (Chap. 7),
defined as a difference between a pricing function αi,l ui,l and a utility function.
8.2 Games in γ -Link Topologies 125

Unlike the games in Chap. 7, herein the utility function reflects the associated
channel’s OSNR degradation. Based on the OSNR model (8.1) and the new measure
variable δQ1i,l (8.2) for stage l, let us define the cost function Ji,l as

Ji,l = αi,l ui,l − Ui,l , ∀i ∈ M (8.3)

where in this case

 
ai,l
Ui,l = βi,l ln 1 + (8.4)
1
δQi,l − Γli,i

That is, the utility function Ui,l reflects a preference for lower OSNR degradation of
channel i on stage l. Substituting (8.2) into (8.3) yields
 
ui,l
Ji,l (ul,−i , ui,l ) = αi,l ui,l − βi,l ln 1 + ai,l , (8.5)
X̃l,−i

where

X̃l,−i = Γli,j uj,l (8.6)
j ∈M,j =i

It follows that Ji,l (ul,−i , ui,l ) is continuously differentiable in its arguments and
convex in ui,l . Let us denote the lower-level stage Nash game by G(M, Ωi , Ji,l ).
Recall that the overall action space Ω is convex, then from Proposition 4.5
G(M, Ωi , Ji,l ) admits an NE solution.
We exploit the partitioned Nash game composed of L lower-level stage Nash
games, given by G(M, Ωi , Ji,l ). Each G(M, Ωi , Ji,l ) is played by m channels such
that δQ1i,l is minimized for each channel i on stage l. The lower-level stage Nash
games are played sequentially
 (in a precedence order) with the interpretation that
across all L stages, l∈R i Ji,l is related to the overall OSNR degradation for chan-
nel i. Solutions of all G(M, Ωi , Ji,l ) are interconnected, explained as follows.
Let u∗l = [u∗i,l ] be an NE solution of G(M, Ωi , Ji,l ). Recall that channel powers
are adjustable at bifurcation nodes and γi,l is the adjustable parameter for channel i
on stage l. Given the precedent actions of channel i, i.e., u∗i,l and p∗i,l , the adjust-
ment for channel i on stage l is obtained as

∗
u∗i,l
γi,l =
p∗i,l

In a vector form, γl∗ = [γi,l

∗ ], i ∈ M, satisfies


u∗l = diag γl∗ · p∗−l ,

where p−l consists of the corresponding channel signal powers from the output of
other stages for all i ∈ M. Recall that each adjustment parameter is bounded within
126 8 Games in Network Topologies

∗ ∈ [γ
[γmin , γmax ]. The partitioned Nash game admits a solution if each γi,l min , γmax ].
∗
Alternatively, γl can be written as

γl∗ = Fl u∗l , p∗−l ,

where Fl is a one-to-one mapping. The reader is referred to [117, 121] for a detailed
proof (Theorem 1).

Remark 8.1 Note that using a system-like cost interpretation the overall L×m game
between γ -links and channels has a cost function


L 
m
Jt = Ji,l (8.7)
l=1 i=1

This could be interpreted as the sum of net utilities over all γ -links and over chan-
nels. After changing the summation order this gives


m 
L
Jt = J˜i , with J˜i = Ji,l (8.8)
i=1 l=1

Thus Jt can be interpreted as the sum of net utilities over channels, each channel
with cost J˜i
L 
  
ai,l
J˜i = αi,l ui,l − βi,l ln 1 +
l=1
1
δQi,l − Γli,i

An interesting parallel can be made between J˜i and the cost Ji (7.3) (Chap. 7), used
in the end-to-end game. This relation is more evident if one considers the case of
identical γ -links. In this case the parameters αi,l , βi,l can be taken to be the same
for all γ -links, αi,l = αi , βi,l = βi , ai,l = ai and J˜i can be rewritten as


L 
L  
ai
J˜i = αi ui,l − βi ln 1 + (8.9)
l=1 l=1
1
δQi,l − Γi,i

Comparing this with Ji (7.3) (Chap. 7), it can be seen that J˜i , (8.9), is similarly
expressed as the difference between a linear pricing term and a logarithmic util-
ity term. In fact J˜i , (8.9), generalizes Ji since it captures in pricing and utility the
contribution of each γ -span from Tx to Rx.

Remark 8.2 We started the formulation from a game with action space in R mL that
was further decomposed. This formulation has a computational advantage when
compared to an overall link m-player game with action space in R L , and a possi-
ble cost J˜i . For the latter game formulation one cannot find an explicit, analytically
tractable NE solution, due to the coupling between channel powers at one γ -link and
8.2 Games in γ -Link Topologies 127

all previous spans. Alternatively, the game formulation thus developed corresponds
to a ladder-nested structure: in the game between the γ -spans the lth player decision
is taken after the (l − 1)th player’s action. The overall NE solution is still coupled
but it has decoupled necessary existence conditions. Moreover, the specific triangu-
lar structure of these conditions enables a recursive computation of the unique NE
solution. For more details the reader is referred to [121], where a similar partitioned
Nash game formulation was proposed for the number of stages K < L. As in [121]
herein the number of channels at each γ -link (stage) was identical for simplicity.
In the following two sections this formulation is extended to multi-link and mesh
topologies which are more complex. Firstly, at each stage, the number of channels at
different stages (links) can be variable and, secondly, more importantly the coupled
link capacity constraint has to be considered.

Next we discuss an iterative algorithm. On each stage l, the following distributed

iterative channel algorithm is used:
 
βi,l 1 1 ui,l (n)
ui,l (n + 1) = − − − Γli,i (8.10)
αi,l yi,l (n) yi,l ai,l

where n is the iteration number, stage l = l − 1 is the precedent of stage l and

yi,l is invariable during the channel iteration on stage l. The algorithm (8.10) is
distributed in the sense that each channel i on stage l updates its signal power ui,l
based on the feedback information, i.e., its OSNR at the output of stage l, yi,l ,
and fixed parameters, μl,l (t) and yi,l . It is worth mentioning that the algorithm
operates synchronously. It is possible to consider asynchronous updates and extend
the analysis in that case.

Theorem 8.3 If for all i ∈ M, ai,l are selected such that (8.11)

Γli,j < ai,l , ∀i ∈ M (8.11)
j =i

then channel algorithm (8.10) converges to the inner NE solution u∗l .

Proof By Theorem 7.1 (point-to-point link) applied to link l, if on stage (link) l, ai,l
is selected satisfying (8.11), then an inner unique NE solution u∗l exists given as in
(7.9). Component-wise, (7.9) is written for i ∈ M as

βi,l 1 
u∗i,l = − Γli,j u∗j,l (8.12)
αi,l ai,l
j ∈M,j =i

The rest of the proof follows directly as the proof of Lemma 4 in [121], stated here
for completeness. Let
ei,l (n) := ui,l (n) − u∗i,l (μl )
128 8 Games in Network Topologies

The corresponding vector form is el (n) = [. . . , ei,l (n), . . .]T . Let us also define
) )  
)el (n)) := maxei,l (n)
∞ i∈M

Assuming that yi,l is stationary it follows that

1 
ei,l (n + 1) = − Γli,j ej,l (n)
ai,l
j ∈M,j =i

Using this it can be shown that under condition (8.11), el (n + 1)∞ ≤ C0 el (n)∞ ,
where 0 ≤ C0 < 1 and el (n)∞ ≤ C0n el (0)∞ , such that the sequence {el (n)}
converges to 0. Therefore channel algorithm (8.10) converges to the inner NE solu-
tion u∗l . 

Note that for a given γ -link matrix Γl , the selected ai,l factors can be set such
that the diagonal dominance condition holds. If Γl is itself diagonal dominant, then
a possible choice is ai,l = Γli,i . If all γ -links are identical then the same ai,l = ai
can be used.

8.3 Games in Single-Sink Topologies

In the simple network topology in the previous section, i.e., γ -link topology, we did
not consider coupled constraints. Towards treating the full case of network topolo-
gies with coupled constraints, we now turn to games with coupled constraints but
we restrict the network topology to another simple case, called single-sink topology.
As we will see this particular topology has the feature of maintaining convexity of
the coupled constraints.
In a single-sink topology with a set L = {1, . . . , L} of links and a set M =
{1, . . . , m} of channels, channels are added at different links, but dropped off at
the output of the same link (e.g. Fig. 8.3).
Compared to the point-to-point topology here we can have different number
channels over different links. We order the m channels in the following way: a set
Ma1 = {1, . . . , m1 } of m1 channels are added into the network at the first link, a set
 l−1 
 
l
Mal = mj + 1, . . . , mj
j =1 j =1

of ml channels are added at link l and


L
ml = m
l=1
8.3 Games in Single-Sink Topologies 129

Fig. 8.3 A single-sink topology

As before the input power of each channel i ∈ M at Tx is denoted by ui ∈ Ωi =

[umin , umax ] with umax > umin ≥ 0. For each channel i ∈ M, let ui,l and pi,l denote
the signal power of channel i at the input and output of link l, respectively. Each
l , thus overall there are L coupled constraints.
link l has a total power target P

8.3.1 Convexity Analysis (Coupled Constraints)

In this section we analyze the convexity of constraints in a class of multi-link topolo-

gies where only one sink exists.

Example 8.4 Consider an example shown in Fig. 8.3, where three channels (m = 3)
are transmitted over a single-sink three-link network (L = 3).
Similar to Example 6.12 in Chap. 6, the link capacity constraints gl (u) can be
written as

2
g1 (u) = u1,i − P10 = u1 + u2 − P10 ≤ 0
i=1


4 
2

g2 (u) = u2,i − P20 = p1,i + u3 + u4 − P20 = u3 + u4 − P20 − P10 ≤ 0
i=1 i=1


6 
4

g3 (u) = u3,i − P30 = p2,i + u5 + u6 − P30 = u5 + u6 − P30 − P20 ≤ 0,
i=1 i=1

where the system constraint (6.29) was used twice. Thus the .coupled constraint set
for link l ∈ L, Ω l = {u ∈ Ω | gl (u) ≤ 0} is convex and Ω = l∈L Ω l is also convex.

This result can be generalized for single-sink multi-link topologies with m chan-
nels, presented in the following lemma.
130 8 Games in Network Topologies

Lemma 8.5 Consider a single-sink topology with a set of links L. For any link
l ∈ L,.the coupled constraint set Ω l is convex. The overall coupled action space
Ω = l∈L Ω l is a convex set.
l−1 l
Proof Let us denote m̄l−1 = j =1 mj and m̄l = j =1 mj . Then

Mal = {m̄l−1 + 1, . . . , m̄l−1 + ml }

Thus the link capacity constraint (6.39) can be rewritten as


ml
gl (u) = l ≤ 0,
um̄l−1 +j − P (8.13)
j =1

where P l is a modified power constraint related to the coupled propagated con-

straints. In fact Pl is the remaining total power constraint after the propagated
optical power of channels is considered. In Example 8.4, P20 = P20 − P10 . One can
rewrite gl (u) as

m̄l
gl (u) = l
uj − P (8.14)
j =m̄l−1 +1
or
l
gl (u) = eTml u − P (8.15)
where
 
T
em l
= 0Tm̄l−1 , 1Tml , 0T (8.16)
is an 1 × m vector with 1ml , an ml × 1 all ones vector, and 0, a vector of all 0
components with appropriate dimension. Thus Ω l is convex for all l, and so is Ω. 

Remark 8.6 From (8.14), one can see that in the inequality constraint for link l,
gl (u) ≤ 0, only channels in the set Mal are coupled, i.e., channels that are added
into the network at link l are coupled. This is advantageous for a hierarchical de-
composition, as will be shown later.

8.3.2 Hierarchical Decomposition

Based on the convexity result in Lemma 8.5, let us consider now an m-player Nash
game in a single-sink topology where there are L coupled constraints. The action
space is
 
Ω= u ∈ Ω | gl (u) ≤ 0 ,
l∈L
8.3 Games in Single-Sink Topologies 131

where Ω is the Cartesian product of Ωi , i ∈ M. The action set of channel i ∈ M is

the projection of Ω on channel i’s direction, namely,
 
i (u−i ) = ξ ∈ Ωi | gl (u−i , ξ ) ≤ 0, ∀l ∈ L .
Ω

As before in the Nash game formulation, each channel i ∈ M minimizes its indi-
vidual cost function Ji : Ω → R,
Ji (u−i , ui ) = αi ui − Ui (u−i , ui ), (8.17)
with a utility Ui indicating preference for better OSNR:
   
ai ai ui
Ui (u−i , ui ) = βi ln 1 + 1 = ln 1 + 0  ,
yi − Γi,i
ni + j =i Γi,j uj
where αi , βi > 0 are weighting factors, ai > 0 is a channel specific parameter and
Γ = [Γi,j ] is the network system matrix being defined in (6.36).
We denote the game by G(M, Ω i , Ji ), which is in the class of m-player Nash
games with coupled utilities and coupled constraints. It follows from (8.17) that
Ji is continuously differentiable in its arguments and convex in ui . Note that the
overall coupled action space Ω is compact. By Lemma 8.5, Ω is convex as well.
Then from Proposition 5.3, G(M, Ω i , Ji ) admits an NE solution. In the following,
we use Lagrangian extension and decomposition results presented in Chap. 5 to
compute an NE solution.
An NE solution of G(M, Ω i , Ji ) can be computed by using the Lagrangian ex-
tension to the game-theoretic framework. We continue using the notations defined
in Chap. 5 for the separable NG cost function J((u; x), (4.7),

m
J((u; x) = Ji (u−i , xi ),
i=1
the separable augmented constraint (
g (u; x), (5.8),

m
(
g (u; x) = g(u−i , xi ),
i=1
( x; μ), (5.11),
the augmented Lagrangian function L(u;
( x; μ) = J((u; x) + μT (
L(u; g (u; x)
and the dual cost function D(μ), (5.17),

( u∗ ; u∗ ; μ ,
D(μ) = L (8.18)
( x; μ). In a fixed-point
where u∗ is such that u = u∗ satisfies u = arg minx∈Ω L(u;
notation,
 
D(μ) = min L(u;( x; μ)  ,
x∈Ω (
arg minx∈Ω L=u

where (
g (u; u) ≤ 0.
132 8 Games in Network Topologies

Then G(M, Ω i , Ji ) is related to a constrained minimization of J(, (4.7), (5.8),

with respect to the second argument x, that admits a fixed-point solution. Individual
components of a solution u∗ to this constrained minimization constitute an NE so-
lution to G(M, Ω i , Ji ) in the sense of Definition 5.2. From Remark 5.6, we know
that u∗ is obtained by first minimizing L(u;( x; μ) with respect to x. The next step
involves finding a fixed-point solution which depends on μ, u∗ (μ). An NE solution–
Lagrange multiplier pair (u∗ (μ∗ ), μ∗ ) is obtained if μ∗ T g(u∗ (μ∗ )) = 0 and μ∗ ≥ 0.
The dual cost function D(μ) is related to the minimization of the associated La-
( x; μ). For G(M, Ω
grangian function L(u; i , Ji ) with convex coupled constraints,
Proposition 5.9 yields a decomposition into a lower-level Nash game with no cou-
pled constraints and a higher-level problem for pricing. Particularly, Remark 8.6
states that only channels that are added into the network at link l, are coupled. This
benefits the hierarchical decomposition for D(μ) by Proposition 5.9.

Theorem 8.7 Consider G(M, Ω i , Ji ) on a single-sink multi-link topology with L

/Ml , ofa channels are added into network at link
links (L coupled constraints). A set, a

l ∈ L, L = {1, . . . , L} and M = l∈L Ml . The dual cost function D(μ), (8.18),

can be decomposed as


m
 m 
L

D(μ) = Li u∗−i (μ), u∗i (μ), μr(i) + l (8.19)
μl eTml ,−i u∗−i (μ) − P
i=1 i=1 l=1

where r(i) refers to link r where channel i is added, eml ,−i is an (m − 1) vector
obtained by deleting the ith element of the m vector eml , and u∗ (μ) is an NE solution
to G(M, Ωi , Li ) where

Li (u−i , xi , μr(i) ) = Ji (u−i , xi ) + μr(i) xi . (8.20)

This is similar to the point-to-point case (see Chap. 7) but herein each of the L
constraints is accounted for.

Proof Let us order the L links and m channels in the same way as in the proof of
Lemma 8.5. Then the constraint function (8.15) can be written in a two-argument
form
 T
em ,−i u−i + xi − P l , l = r(i)
T 
gl (u−i , xi ) = eml · u − Pl = T l (8.21)
eml ,−i u−i − Pl , l = r(i)

Since Ji and gl are continuously differentiable and convex functions, by using

Proposition 5.9, the dual cost function D(μ) can be decomposed as


m

D(μ) = Li u∗−i (μ), u∗i (μ), μ
i=1
8.3 Games in Single-Sink Topologies 133

where
Li (u−i , xi , μ) = Ji (u−i , xi ) + μT g(u−i , xi )
Using (8.21) one finds that, ∀i ∈ M, Li is given as

Li = Ji (u−i , xi ) + μr(i) gr(i) (u−i , xi ) + μl gl (u−i , xi )
l =r(i)
 
= Ji (u−i , xi ) + μr(i) xi + l
μl eTml ,−i u−i − P (8.22)
l

The following arguments are similar to those in Corollary 1 in [120], outlined here.
Let us minimize RHS of (8.22) with respect to xi and then solve for a fixed-point
solution xi = ui . Since only the first two terms on RHS of (8.22) depend on xi ,
isolating the terms without xi yields
m 
 

D(μ) = min Li (u−i , xi , μr(i) ) 
xi ∈Ωi arg minxi ∈Ωi Li =ui
i=1


m 
L

+ l ,
μl eTml ,−i u−i − P (8.23)
i=1 l=1

with Li as in (8.20). After minimization in each subproblem on RHS of (8.23), xi∗

is obtained as a function of u−i , namely,

xi∗ = xi (u−i )

A fixed-point solution denoted by xi = ui can be obtained by setting xi (u−i ) = ui

and solving simultaneously the m equations for a vector solution denoted by u∗ , or
u∗ (μ) since it depends on μ. Substituting u∗ (μ) on RHS of (8.23) yields (8.19). 

Theorem 8.7 leads to a hierarchical decomposition. An NE solution–Lagrange

multiplier pair (u∗ (μ∗ ), μ∗ ) can be obtained by solving a lower-level uncoupled
Nash game G(M, Ωi , Li ) and a higher-level problem for coordination (i.e., setting
price μ). The hierarchical game decomposition offers computational advantages.
For a given price μ ≥ 0, the lower-level game with no coupled constraints admits
an inner closed-form explicit NE solution. The following result gives conditions for
the existence of the NE solution.

Proposition 8.8 For each given μ ≥ 0, G(M, Ωi , Li ) admits an NE solution if ai

satisfies

Γi,j < ai , ∀i ∈ M (8.24)
j =i
134 8 Games in Network Topologies

Fig. 8.4 A mesh topology

with three channel optical
paths

The inner NE solution u∗ (μ) is unique, given as

u∗ (μ) = Γ(−1 · (
b(μ),

where Γ( = [Γ(i,j ] and (

b(μ) = [(
bi (μ)] are defined as
#
a j = i, ( ai βi
Γ(i,j = i bi (μ) = − n0i , (8.25)
Γi,j j = i, αi + μr(i)

with Γi,j defined in (6.36).

The proof follows similar to the proof of Proposition 7.9 for the point-to-point
topology except that it involves the network matrix and μr(i) .
Based on this single-sink case and the method in Sect. 8.2, a partitioning method
for general multi-link topologies was proposed in [101, 112]. Such a topology is
partitioned into stages where each stage has a single-sink structure and algorithms
developed based on the extragradient method [70, 71]. However, this partition does
not fully exploit the power adjustment flexibility, is not completely scalable and
cannot be easily extended to treat mesh topologies. In the next section we present
an approach that addresses this most general case, of how to treat mesh topologies.

8.4 Games with Coupled Constraints: Mesh Topologies

After these preliminary simple network topologies (γ -link in Sect. 8.2 and single-
sink topologies in Sect. 8.3), we now consider games with coupled constraints in
general mesh and multi-link network topologies. A possible case with three channel
optical paths is shown in Fig. 8.4, where each channel is represented by a different
color (wavelength).
8.4 Games with Coupled Constraints: Mesh Topologies 135

Fig. 8.5 Graph with adjustable nodes for Fig. 8.4

For such general mesh or multi-link topologies, an NE solution of the end-to-

end game with coupled constraints is intractable because of the non-convexity of
propagated constraints. Thus the results in Sect. 8.3 cannot be directly extended
to general multi-link or mesh topologies. We will use an approach that builds on
the simpler results in Sect. 8.2. In particular we will use a partition technique as in
Sect. 8.2 together with a hierarchical decomposition (for coupled constraints) as in
Chap. 7 or Sect. 8.3. The same notations as in Sects. 8.2 and 8.3 are used.
Recall that the starting point for setting an end-to-end game was the fact that
channel powers are assumed to be independently adjustable only at Txs. In fact,
this restriction is relaxed. In optical networks, optical switches not only perform
the function of selectively dropping some of wavelengths or adding selected wave-
lengths while letting others pass through, but also provide the flexibility of optical
power adjustment for each channel [2]. For the example in Fig. 8.4 these five nodes
are shown in Fig. 8.5.
The simplest example we consider is the one obtained by extracting from Fig. 8.4
a single quasi-ring. Thus consider the quasi-ring topology shown in Fig. 8.6 as the
simplest example. In Fig. 8.6, we use Txi /Rxi to indicate the Tx/Rx that channel i
uses. We also use li to refer link l. The quasi-ring topology is a ring-type topology
with partially closed loops being formed by channel optical paths [2]. In Fig. 8.6, the
optical paths of channels 1 and 3 are l1 → l2 → l3 and l3 → l1 → l2 , respectively.
Note that each of the links can be either the intermediate or the end link on a spe-
cific channel optical path. Thus how to define a single-sink structure that potentially
can be partitioned is not immediate. Another issue of concern is the interconnection
among stages. However, the simplest way is to consider just a link by link partition-
ing. This alternative approach is scalable and easily applicable, and moreover fully
136 8 Games in Network Topologies

Fig. 8.6 Quasi-ring topology

uses the flexibility of power adjustment at each node. Thus the network is parti-
tioned as in Sect. 8.2 into stages composed only of single optical links. By breaking
the loop and selecting one stage as the start, stages can be placed sequentially in a
ladder-nested form. One benefit of such a partition is that the convexity of coupled
constraints propagated along links on each stage is automatically satisfied.
In the partitioned Nash game, each link Nash game is played towards minimizing
channel OSNR degradation. Based on such a partition, the hierarchical decomposi-
tion is applicable to each link game, leading to a lower-level uncoupled game for
channels on each link and a higher-level problem for link pricing. Computation of
equilibria is based on a three-level hierarchical algorithm. Such a partition simplifies
the structure of each stage, makes it regular and scalable, and benefits the develop-
ment of a link pricing algorithm. This approach can be thus used for general mesh
topologies.
We can apply Lemma 6.8 and use the recursive OSNR model for the network.
Consider link l in Fig. 6.10. The OSNR of channel i at the output of link l is given
as
1 1  uj,l
= + Aj,l Γli,j , i ∈ Ml , (8.26)
yi,l yi,l ui,l
j ∈M

where link l is the link precedent to link l for channel i. Using (8.26), let us define

1 1 1  uj,l
= − = Aj,l Γli,j (8.27)
δQi,l yi,l yi,l ui,l
j ∈M

It follows that δQ1i,l measures the OSNR degradation of channel i from link l to
link l. Thus instead of maximization of yi from Tx to Rx, consider minimization of
each δQ1i,l between links, i.e., minimization of individual OSNR degradation. In the
next section, by using the similar methodology as in Sect. 8.2, a partitioned Nash
game composed of stage Nash games is presented. Each stage is a link and the stage
Nash game is formulated to minimize the OSNR degradation, and moreover the
coupled constraints are considered.
8.4 Games with Coupled Constraints: Mesh Topologies 137

8.4.1 Partitioned Game Formulation

In this section we discuss a partitioned game framework with link capacity con-
straints in mesh topologies. As mentioned above the network is partitioned into
stages composed of single links. By partitioning, each stage game is defined for a
link, thus a link game, as in Sect. 8.2. Each link Nash game is played towards min-
imizing channel OSNR degradation. Then a partitioned Nash game composed of
link games is defined. The set of OSNR degradation minimizations on stages/links
is related to OSNR maximization from Tx to Rx (see Remark 8.1 in Sect. 8.2). This
setup is described below.
On each link l ∈ L, the set of channels is Ml = {1, . . . , ml }. Let us consider the
link capacity constraint on link l:

gl (ul ) = l ≤ 0, ∀l ∈ L,
Ai,l ui,l − P (8.28)
i∈M

where Ai,l = 1 if channel i is transmitted on link l and otherwise, Ai,l = 0. Let Ω l

denote the action space on link l,
 
Ω l = ul ∈ Ω | gl (ul ) ≤ 0 , (8.29)

where Ω denotes the Cartesian product of Ωi . The action set of individual channel
i ∈ Ml is defined as the projection of Ω l on channel i’s direction, namely,
 
Ωi,l (ul,−i ) = ξ ∈ Ωi | gl (ul,−i , ξ ) ≤ 0 ,

where ul,−i is obtained by deleting ui,l from vector ul . It can be seen that both the
action space Ω l and the individual action set Ω i,l (ul,−i ) are compact and convex.
The link Nash game is played with each channel attempting to minimize its indi-
vidual cost with respect to its OSNR degradation. We specify a channel cost func-
tion Ji,l for channel i on each link l. Similar to the channel cost function defined in
Sect. 8.2, Ji,l is defined as a difference between a pricing function Ai,l αi,l ui,l and
a utility function Ui,l which reflects the associated channel’s OSNR degradation,
namely,
Ji,l = Ai,l αi,l ui,l − Ui,l , ∀i ∈ M, (8.30)
with
 
Ai,l ai,l
Ui,l = βi,l ln 1 + , (8.31)
1
δQi,l − Γli,i
where βi,l > 0 indicates the strength of the channel’s desire to minimize its OSNR
degradation and ai,l > 0 is for scalability. Substituting (8.27) into (8.30) yields
 
Ai,l ui,l
Ji,l = Ai,l αi,l ui,l − βi,l ln 1 + ai,l , ∀i ∈ M, (8.32)
Xl,−i
138 8 Games in Network Topologies

where Xl,−i = j =i,j ∈M Aj,l Γli,j uj,l . It follows that Ji,l is continuously differen-
tiable in its arguments and convex in ui,l .
We denote such a link Nash game by G(M, Ω i,l , Ji,l ). Note that the individ-
ual cost function Ji,l is generally defined for each channel i ∈ M. If channel i is
not transmitted on link l, i.e., i ∈ / Ml , then Ai,l = 0 and Ji,l = 0, which means
that the decision of channel i, i ∈ / Ml , does not affect the decisions made by
other channels j ∈ Ml . Thus G(M, Ω i,l , Ji,l ) is equivalently to a reduced Nash
game played among ml channels. Furthermore, the existence of an NE solution of
G(M, Ω i,l , Ji,l ) is guaranteed by Proposition 5.3.
We exploit the partitioned Nash game. By partitioning and selecting one stage
as the start,
 stages can be sorted sequentially with the interpretation that across all
stages, l∈Ri Ji,l is related to the overall OSNR degradation for channel i. So-
lutions of all G(M, Ω i,l , Ji,l ), l ∈ L are interconnected similar to Sect. 8.2. The
explanation is given as follows.
Recall that channel powers are adjustable at optical switches and γi,l is the ad-
justable parameter for channel i on stage l. The vector form is γl = [γi,l ]. Let
u∗l = [u∗i,l ] be an NE solution of G(M, Ω i,l , Ji,l ). The corresponding signal power
vector at the output of link l is p∗l = [pi,l ∗ ] and the corresponding augmented output

power vector defined in Lemma 6.10 is p∗ . Note that for those channel i ∈ / Ml , val-
ues of u∗i,l and pi,l ∗ are randomly set. By using (6.37) in Lemma 6.10, optimal γ
i,l
can be obtained by solving the corresponding component-wise equation in
 T ∗
u∗l = diag γl∗ · diag Vi,l ·p .
∗ ∈ [γ
Finally, the partitioned Nash game admits a solution if each γi,l min , γmax ].
Compared to Sect. 8.2 the interconnection matrix diag(Vi,l ) for the full network
topology appears as well as the coupled constraints.

8.4.2 Hierarchical Decomposition

In this section, we first use Lagrangian extension presented in Chap. 5 to obtain an

NE solution. Then we discuss an iterative hierarchical algorithm for computation of
equilibria of the partitioned Nash game. Recall that if channel i is not transmitted
on link l, its decision does not affect the decisions made by other channels j ∈ Ml .
Thus G(M, Ω i,l , Ji,l ) is equivalently to a reduced Nash game played among ml
channels. The computation of equilibria is based on this fact and the algorithm is
developed for the equivalent reduced Nash game, i.e., G(Ml , Ω i,l , Ji,l ). We use the
−
mark “ ” to indicate the associated reduced vector. For example, ūl is the reduced
vector obtained by removing those elements ui,l , i ∈ / Ml , from ul . Sometimes we
write ūl as ūl = (ūl,−i , ui,l ) with the vector ūl,−i obtained by deleting the ith ele-
ment from the reduced vector ūl .
On each link l ∈ L, G(Ml , Ω i,l , Ji,l ) admits an NE solution From Proposi-
tion 5.3. An NE solution can be computed by using the Lagrangian extension to
8.4 Games with Coupled Constraints: Mesh Topologies 139

the game-theoretic framework presented in Chap. 5. We first define several aug-

mented functions for G(Ml , Ω i,l , Ji,l ). The augmented NG cost function J(l (ūl ; x̄l )
is defined as

J(l (ūl ; x̄l ) = Ji,l (ūl,−i , x̄i,l ),
i∈Ml

where Ji,l is defined in (8.32) with Ai,l = 1. The two-argument constraints are given
as

gl (ūl ; x̄l ) =
( gl (ūl,−i , x̄i,l )
i∈Ml

Both J(l (ūl ; x̄l ) and (

gl (ūl ; x̄l ) are separable in the second argument x̄l . The associ-
ated two-argument Lagrangian function L (l is defined as

(l (ūl ; x̄l ; μl ) = J(l (ūl ; x̄l ) + μl (

L gl (ūl ; x̄l ), (8.33)

where the scalar μl is the Lagrange multiplier. The associated dual cost function
Dl (μl ) is defined as

(l ū∗l ; ū∗l ; μl ,
Dl (μl ) = L (8.34)
where ū∗l is such that ūl = ū∗l satisfies

( ūl ; x̄l ; μl )
ūl = arg min L(
x̄l ∈Ω

where Ω denotes the Cartesian product of Ωi , i ∈ Ml .

The G(Ml , Ω i,l , Ji,l ) is related to a constrained minimization of J(l (ūl ; x̄l ) with
respect to x̄l , which admits a fixed-point solution ū∗l . Individual components of ū∗l
constitute an NE solution to G(Ml , Ω i,l , Ji,l ). In the Lagrangian optimality condi-
∗
tion, ūl is obtained by first minimizing L (l (ūl ; x̄l ; μl ) with respect to x̄l . The next
step involves finding a fixed-point solution ū∗l by setting x̄l = ūl . Since ū∗l depends
on μl , an NE solution–Lagrange multiplier pair (ū∗l (μ∗l ), μ∗l ) is obtained if

μ∗l gl ū∗l = 0 and μ∗l ≥ 0.

The minimization of L (l (ūl ; x̄l ; μl ) can be decomposed by Proposition 5.9. Then

for each link l applying Proposition 7.8 in Chap. 7 yields a decomposition in terms
of Li,l . Alternatively since each stage l is a single optical link, which is the sim-
plest single-sink structure, the decomposition results (Theorem 8.7) for single-sink
topologies can be directly used. Thus G(Ml , Ω i,l , Ji,l ) can be naturally decom-
posed into a lower-level Nash game with no coupled constraints and a higher-level
link problem for pricing (Lagrange multiplier). By using Theorem 8.7, after decom-
position, the lower-level Nash game is obtained with the following individual cost
function:

Li,l (ūl,−i , x̄i,l , μl ) = Ji,l (ūl,−i , x̄i,l ) + μl x̄i,l , ∀i ∈ Ml , (8.35)

140 8 Games in Network Topologies

where Ji,l is defined in (8.32) and an uncoupled action set Ωi . We denote this game
by G(Ml , Ωi , Li,l ). Notice that Li,l (8.35) has a similar form with Li (7.51) in
Chap. 7. By using Proposition 7.9 we obtain the following result characterizing the
NE solution of G(Ml , Ωi , Li,l ).

Corollary 8.9 For each given μl ≥ 0, G(Ml , Ωi , Li,l ) admits an NE solution if ai,l
in Li,l (8.35) satisfies

Γli,j < ai,l , ∀i ∈ Ml (8.36)
j =i,j ∈Ml

The inner NE solution ū∗l (μl ) is unique, given as

ū∗l (μl ) = Γ(l−1 · b̄l (μl ) (8.37)

where Γ(l = [Γ(li,j ] with

#
a , j =i
Γ(li,j = i,l i, j ∈ Ml ,
Γli,j , j = i

where b̄l (μl ) = [bi,l (μi )] with

ai,l βi,l
bi,l (μl ) = , i ∈ Ml
αi,l + μl

It follows from (8.37) that i∈Ml u∗i,l (μl ) decreases when μl increases. This mono-
tonicity property will be used in the development of an iterative hierarchical algo-
rithm.

8.4.3 Primal–Dual (Channel-Link) Iterative Algorithm

We discuss an iterative hierarchical algorithm based on the NE solution (8.37) of

the lower-level Nash game and coordination at the higher level.
For given μl ≥ 0, ū∗l (μl ) in Corollary 8.9 is a solution to solve the Lagrangian
optimality condition. Recall that (ū∗l (μ∗l ), μ∗l ) is an NE solution–Lagrange multi-
plier pair if

μ∗l gl ū∗l = 0 and μ∗l ≥ 0.
By using (8.28), μ∗l ≥ 0 can be obtained by solving
 
∗
μl∗ ∗ 
ui,l μl − Pl = 0 (8.38)
i∈Ml

Based on these, an iterative hierarchical algorithm can be developed for both link
pricing and channel power adjustment.
8.4 Games with Coupled Constraints: Mesh Topologies 141

Channel Algorithm Based on the similarity between G(Ml , Ωi , Li,l ) and

G(M, Ωi , J i,l ) developed for γ -point topologies in Sect. 8.2, one can adapt the
channel algorithm:
 
βi,l 1 1 ui,l (n)
ui,l (n + 1) = − − − Γli,i , ∀i ∈ Ml (8.39)
αi,l + μl (k̄) yi,l (n) yi,l ai,l

where link l is the precedent of link l. Note that both yi,l and μl (k̄) are invariable
during the channel iteration on link l. Each channel i on link l updates its signal
power ui,l based on the feedback information, i.e., its OSNR at the output of link l,
yi,l , and fixed parameters, μl (t) and yi,l . Note that compared to (8.10) in Sect. 8.2
the term μl appears due to coupled constraint, while compared to (7.62) in Chap. 7)
the term yi,l appears due to partitioning in link stages and interconnection between
link l and link l. Then applying Theorem 8.3 in Sect. 8.2 one can obtain the follow-
ing convergence result.

Corollary 8.10 If for all i ∈ Ml , ai,l in Li,l (8.35) are selected such that (8.36)
is satisfied. Then for each given μl ≥ 0, channel algorithm (8.39) converges to the
inner NE solution ū∗l (μl ).

Link Algorithm The link algorithm is a gradient projection algorithm [24, 81],
developed based on (8.38). On each link l, after every K iterations of the channel al-
gorithm (8.39), the new link price μl is generated at each iteration time k̄, according
to the following link algorithm:
  +

μl (k̄ + 1) = μl (k̄) + η l
ui,l μl (k̄) − P (8.40)
i∈Ml

where η > 0 is a step-size and [z]+ = max{z, 0}. Practically, K is sufficiently large
such that each channel power converges to its solution. Thus given the total power
of all channels on the link l, the link algorithm (8.40) is completely distributed and
can be implemented by individual links using only local information. A proof for
the convergence of this link algorithm is provided in [111].
The link algorithm above is a simple projection algorithm. Theorem 1 in [111]
provides an explicit upper-bound for the step-size η related to the NE solution. It
can be mentioned that an extragradient method can been used on each stage for
the link pricing [112]. The extragradient method is a modified projection algorithm,
and sufficient conditions for its convergence depend on a Lipschitz constant which
is intractable.

Remark 8.11 For any given μl ≥ 0, Theorem 8.3 shows that the channel algorithm
converges to the NE solution which is unique in the sense of inner-ness. Though
uniqueness of a solution is not guaranteed, μ∗l , (u∗l , μ∗l ) is an optimal NE solution–
Lagrange multiplier pair and the algorithm converges to one such pair.
142 8 Games in Network Topologies

8.5 Notes

For mesh and multi-link network topologies we treated several cases in increasing
order of complexity. First we treated simple γ -link topologies and we formulated
a partitioned Nash game composed of link Nash games, where for simplicity no
coupled constraints were considered. In the partitioned Nash game, each link Nash
game is played towards minimizing channel OSNR degradation. Based on such a
partition, the hierarchical decomposition is applicable to each link Nash game.
Next we considered games with coupled utility and constraints in multi-link
topologies, extending the approach in Chap. 7 where only single point-to-point fiber
links were treated. In multi-link topologies constraints propagate along link and
convexity may not be preserved. The general multi-link topology was dealt with by
formulating a partitioned Nash game composed of stage Nash games, where each
stage has a single-link structure, i.e., a γ -link structure. Connections between stages
are realized by exploiting the flexibility of extra adjustments at bifurcation points.
By selecting a starting link, the game on each link can be played sequentially and
the hierarchical decomposition leads to a lower-level Nash game for channels with
no coupled constraints and a higher-level problem for link pricing. Computation of
equilibria is based on a hierarchical algorithm. An alternative partitioned game could
be based on single-sink structure (which has convexity ensured), but the single-link
partition has a more regular and scalable structure. A possible implementation is
described later on in Chap. 10.
In Chaps. 7 and 8 game-theoretic approaches have been used to solve channel
OSNR optimization with link capacity constraints in optical networks. In particu-
lar, while we formulated a game-theoretic framework towards optimizing channel
OSNR, however, we have not taken desired channel OSNR targets into account yet.
In a Nash game in the presence of player selfishness, Nash equilibria may not op-
timize overall system performance. These efficiency issues as well as robustness of
network control algorithms derived from game-theoretic formulations in the pres-
ence of time-delay will be addressed in the following two chapters.
Chapter 9
Nash Equilibria Efficiency and Numerical
Studies

Abstract This chapter provides an alternative constrained OSNR optimization ap-

proach. This framework can be used to investigate the effects of parameters in the
game-theoretic approach, i.e., the efficiency of Nash equilibria. A system optimiza-
tion problem is formulated towards achieving an OSNR target for each channel
while satisfying the link capacity constraint. In the game case we show that OSNR
targets can be achieved and efficiency can be possibly improved by appropriate se-
lection of game parameters.

9.1 Introduction
In previous chapters we showed how a game-theoretic framework can be formulated
towards optimizing channel OSNR, but we have not taken desired channel OSNR
targets into account yet. In a Nash game in the presence of player selfishness, Nash
equilibria may not optimize overall system performance. These efficiency issues
will be addressed next, focusing on a single point-to-point link game. The material
in this chapter is mostly from [102–104]. A combination approach can be found
in [168].
We first consider a system optimization framework towards achieving a target
OSNR level for each channel, while minimizing the interference and hence im-
proving the overall performance. Link capacity constraint is also considered. This
framework is used to investigate the effects of parameters in a game-theoretic ap-
proach.
Efficiency of Nash equilibria has been studied extensively [1, 38, 64, 122, 129,
134]. The efficiency loss as compared to an socially optimal system performance
is quantified by a ratio called “Price of Anarchy” [73]. Work on how to quantify
the efficiency loss under separable and under non-separable costs can be found in
[64] and [122], respectively. Results suggest that, provided a pricing mechanism is
chosen properly, the selfish behavior of players in a Nash game may not degrade
the system performance arbitrarily [64, 134]. Such approaches will be reviewed in
Chap. 12 for routing and path coloring via game approaches in optical networks. In
this chapter, instead of looking from the view point of degree of efficiency (Price
of Anarchy), we study the efficiency of an OSNR game by investigating the effects
of parameters in individual game cost functions. In particular, we use the system

L. Pavel, Game Theory for Control of Optical Networks, 143

Static & Dynamic Game Theory: Foundations & Applications,
DOI 10.1007/978-0-8176-8322-1_9, © Springer Science+Business Media New York 2012
144 9 Nash Equilibria Efficiency and Numerical Studies

optimization framework to measure the efficiency of a game as in Chap. 7. This

can be motivated by the fact that the aggregate cost function in a game-theoretic
formulation is not always strictly convex; hence the optimal solution of the associ-
ated constrained optimization problem is not available. Nevertheless, we show that
a possible cost function in the system optimization has an approximate interpreta-
tion as the game cost function in the game-theoretic formulation. We compare the
numerical results based on this and show the effects of pricing mechanisms.

9.2 A System Optimization Formulation

Consider a single point-to-point WDM link with a set of M = {1, . . . , m} channels

transmitted over the link and two types of constraint: the link capacity constraint as
before

ui − P 0 ≤ 0,
i∈M

where P 0 is the link total power target, and the target OSNR constraint

OSNRi ≥ 
γi , ∀i ∈ M, (9.1)

where γi is the target OSNR for channel i. As before the signal optical power at Tx
ui is bounded in Ωi = [0, umax ] with constant umax > P 0 , for all i ∈ M.
Consider an alternative OSNR optimization setup from the perspective of system
performance, as opposed to a game-theoretic approach as developed in previous
chapters. Specifically consider the following cost:

min C(u)
subject to ui ∈ Ωi , ∀i ∈ M,
 i ≥
OSNR γi , ∀i ∈ M,
uj ≤ P 0 ,
j ∈M

where C(u) is the system cost function, defined as the sum of all individual costs,
Ci (ui ),

C(u) = Ci (ui ),
i∈M

hence it is separable. Each individual cost function Ci (ui ) is a generic cost function
that satisfies the following assumption:

Assumption 9.1 Each Ci (ui ) is strictly convex, continuously differentiable and

lim Ci (ui ) = +∞ (9.2)

ui →0
9.2 A System Optimization Formulation 145

Fig. 9.1 OSNRi vs. ui

This cost function can be defined similarly to the form of (7.3), Ci = Pi −Ui . The
pricing function Pi is a linear function of ui . The utility function Ui is a logarithmic
function of ui , which quantifies approximately the link’s demand or the channel’s
willingness to pay for a certain level of OSNRi based on the relationship between
OSNRi and ui . This relationship is illustrated approximately in Fig. 9.1.
By using the OSNR model (6.16), (9.1) can be rewritten as
ui
 ≥
γi ,
n0i + j ∈M Γi,j uj

or

ui + γi Γi,j )uj ≥ n0i 
(− γi ,
j ∈M

or in vector form
T u ≥ b, (9.3)
where
⎡ ⎤ ⎡ ⎤
1−
γ1 Γ1,1 −
γ1 Γ1,2 ··· −
γ1 Γ1,m n01 
γ1
⎢ − ⎥ ⎢ 0 ⎥
⎢ γ2 Γ2,1 1−
γ2 Γ2,2 ··· −
γ2 Γ2,m ⎥ ⎢ n2 γ2 ⎥
T =⎢ .. .. .. .. ⎥, b=⎢
⎢ .. ⎥
⎥
⎣ . . . . ⎦ ⎣ . ⎦
−
γm Γm,1 −
γm Γm,2 ··· 1−
γm Γm,m n0m 
γm
All T ’s off-diagonal entries −
γi Γi,j are less than zero. By Definition A.9, T is a
Z-matrix.
146 9 Nash Equilibria Efficiency and Numerical Studies

From the total power constraint and ui ≥ 0, ∀i ∈ M, we have ui ≤ P 0 . Recalling

i is bounded in Ωi = [0, umax ] and umax > P , we can deduce that the condi-
that u 0

tions j ∈M uj ≤ P and ui ∈ Ωi , ∀i ∈ M are equivalent to 1T u ≤ P 0 and u ≥ 0,

0

where 1 is the m × 1 all ones vector. Therefore, the constraint set of the system
optimization problem is Ω := {u ∈ Rm | T u ≥ b, 1T u ≤ P 0 and u ≥ 0}. This con-
strained system optimization problem can be formulated as

min C(u)
(9.4)
subject to u ∈ Ω

and denote this problem by OPT(Ω, C). Condition (9.2) in Assumption 9.1 ensures
that the solution to OPT(Ω, C) does not hit ui = 0, ∀i ∈ M. Let 
γ = [
γ1 , . . . , 
γm ]T .
The following result characterizes the unique solution of OPT(Ω, C).

Theorem 9.1 If the following conditions on 

γ hold:
1
γi < 
 , ∀i ∈ M, (9.5)
j ∈M Γi,j

where Γ = [Γi,j ] is the system matrix defined in (6.17) and

1T · T((
γ ) · b(
γ ) ≤ P 0, (9.6)

γ ) = [n01 
with b( γm ]T and T((
γ1 , . . . , n0m  γ ) = T −1 (
γ ), then the constraint set Ω is
non-empty and OPT(Ω, C) has a unique positive solution uopt .

Proof We first show that the constraint set Ω is non-empty. Note that in the link
OSNR model, the system matrix Γ (6.17) is a positive matrix, so if (9.5) is satisfied,
we have

1− γi Γi,i > 
γi Γi,j > 0, ∀i ∈ M,
j ∈M,j =i

or equivalently,

1−
γi Γi,i > |−
γi Γi,j |, ∀i ∈ M,
j ∈M,j =i

which implies that the Z-matrix T has positive diagonal entries and by Definition
A.6, T is strictly diagonally dominant. According to Theorem A.7, each eigenvalue
of T has a positive real part. Then it follows from Theorem A.10 that T is an M-
matrix. So it has the following properties: T u ≥ b > 0 implies u ≥ 0, and T −1 is
non-negative. Thus
u ≥ T −1 b := T(b (9.7)
and then we have 1T u ≥ 1T · T( · b. Note that both T( and b depend on 
γ , i.e.,
T( = T((
γ ) and b = b(
γ ). So

1T · u ≥ 1T · T((
γ ) · b(
γ)
9.2 A System Optimization Formulation 147

Fig. 9.2 Illustration of the

constraint set Ω

u = T(b. Then T (
Let ( u = 1T · T(b. By (9.6), 1T · (
u = b. Also we have 1T · ( u ≤ P 0 . It
follows that
 
(u ∈ u ∈ Rm | T u ≥ b, 1T u ≤ P 0
Thus the above set is non-empty if both (9.5) and (9.6) are satisfied. Since T u ≥
b > 0 implies u ≥ 0, we have proved that if  γ is selected such that (9.5) and (9.6)
are satisfied, the constraint set Ω is non-empty.
Moreover, the constraint set Ω is convex and we have 0 ≤ ui ≤ P0 , ∀i ∈ M.
So Ω is bounded. In addition, it is also closed since it consists of the intersection of
half-spaces. Thus this system optimization problem is a strictly convex optimization
problem on a convex compact constraint set [30], which, following Proposition A.2,
always admits a unique globe minimum, uopt . 

Example 9.2 To illustrate the constraint set Ω and the conditions (9.5) and (9.6)
consider a simple example in Fig. 9.2 where m = 2. From (9.7), we have
   
u1 row1 (T() · b
≥
u2 row2 (T() · b

where rowi (A)

 is defined as the ith row of the matrix A. From Fig. 9.2, it is readily
seen that if 2i=1 rowi (T()b ≤ P 0 , the intersection point Q lies in the set of the total
input power constraint, i.e.,
  
row1 (T() · b, row2 (T() · b ∈ u ∈ R2 | u1 + u2 ≤ P0

Therefore the constraint set Ω is non-empty.

Remark 9.3 Recall that n0 denotes the input noise power at Tx and may include
external noise, such as thermal noise. If the input noise is neglected, n0 includes
only external noise, which is negligible [45]. So b = [n01  γ1 , . . . , n0m 
γm ]T ≈ 0 and
P 0 ≥ 1T · T( · b ≈ 0. This means that the constraint set is non-empty under the first
148 9 Nash Equilibria Efficiency and Numerical Studies

condition (9.5). Thus the OSNR target 

γi can be selected in a distributed way based
on the first condition (9.5).

Remark 9.4 Let us take a close look at the second condition (9.6). Recall that
T = I − diag(γ )Γ , where I is an identity matrix. We know from the proof of The-
orem 9.1 that T is an M-matrix. By Theorem A.11, ρ(diag( γ )Γ ) < 1 and
∞
−1  k k
T −1 = I − diag(
γ )Γ = diag 
γ Γ
k=0

exists, which is positive component-wise. We can rewrite (9.6) as

∞
 k k
1T ·  Γ · diag(
diag γ γ ) · n0 ≤ P 0 (9.8)
k=0

If 
γi increases (given 
γj , j = i), LHS of (9.8) will increase. We can find a maximum
OSNR target  γmax by solving the following equation:

 γmax Γ )−1 · n0 = P 0
γmax · 1T · (I −  (9.9)

Based on the link OSNR model, we know that the performance for each channel
is interference limited. In addition, (9.9) shows that the OSNR targets significantly
affect the capacity of a link: each link decides the OSNR threshold  γmax by us-
ing (9.9). Any new channel with a required OSNR target no more than  γmax will be
admitted to transfer over the link. This idea can be used for links to develop channel
admission control schemes. Similar algorithms exist for wireless networks [65, 66].

9.3 Distributed Algorithm

Since OPT(Ω, C) is a constrained convex optimization problem, there are several

computational methods for solving it [22]. A possible approach is to use a barrier (or
penalty) function to relax the constrained optimization problem. Barrier functions
set a barrier against leaving the feasible region: If the optimal solution occurs at
the boundary of the feasible region, the procedure moves from the interior to the
boundary [22, 143]. By an appropriate choice of barrier functions, the solution of the
relaxed system problem can arbitrarily approximate the one of the original problem
OPT(Ω, C).
To see this, note that, from the proof of Theorem 9.1 that T u ≥ b > 0 implies
u ≥ 0, OPT(Ω, C) can be rewritten succinctly as

min C(u)
(9.10)
subject to Tu ≥ 
b,
9.3 Distributed Algorithm 149

Fig. 9.3 A barrier function

example

where
   
T b
T = and 
b=
−1T −P 0
Now consider a barrier function λi : R → R with the following properties:
(P.1) ∀i ∈ M, λi (x) is non-increasing, continuous and
 yi (u)
lim λi (x) dx → −∞ (9.11)
ui →∞ 
bi

where
yi (u) := rowi (T)u. (9.12)
(P.2) λi (x) attains the value 0 if x > 
bi , where 
bi is defined in (9.10).
Such an example is shown in Fig. 9.3.
With this barrier function λi , which has properties (P.1) and (P.2), construct a
function
   yi (u)
Vp (u) = Ci (ui ) − λi (x) dx (9.13)

i∈M i∈M bi

and define a relaxed system problem:

min Vp (u) (9.14)

u≥0

Based on (P.1), i.e., the non-increasing property of the barrier function, and from
Assumption 9.1, i.e., that Ci (ui ) is strictly convex, it follows that Vp (u) is strictly
convex and hence has a unique internal minimum value such that

∂Vp (u)  
= Ci (ui ) − rowi TT λ y(u) = 0, ∀i ∈ M
∂ui
150 9 Nash Equilibria Efficiency and Numerical Studies

Fig. 9.4 Primal algorithm: block representation

Thus solving the set of above equations we obtain the unique solution ūopt of (9.14):
 
ūi = Ci −1 rowi TT λ y ūopt , ∀i ∈ M
opt

The barrier function λi (·) can be selected such that the unique solution of (9.14) may
arbitrarily closely approximate the optimal solution of OPT(Ω, C). For example, a
barrier function can be defined as [68]
[
bi − x + ε]+
λi (x) = ,
ε2
where [x]+ = max{x, 0}.
For the relaxed system problem a primal algorithm can be defined as
∂Vp (u)  
u̇i (t) = gi (ui , si ) = −ki = −ki Ci ui (t) − si (t) , ∀i ∈ M, (9.15)
∂ui
where the coefficient ki > 0 and si (t) is defined as
 
si (t) = rowi TT λ y u(t) , (9.16)

and λ(·) = [λ1 (·), . . . , λm+1 (·)]T , with each λi (·) satisfying (P.1) and (P.2).
Algorithm (9.15) is a gradient algorithm and can be implemented in a distributed
way. Each channel varies its input power ui gradually as in (9.15), while the link
(network system) calculates the vector s(t) = [s1 (t), . . . , sm (t)]T based on the re-
ceived input powers, OSNR preference and link constraint, and then feeds this up-
dated information back to each channel. This algorithm is represented in Fig. 9.4.
The following theorem states that the unique equilibrium of the algorithm (9.15)
corresponds to the unique solution of (9.14), ūopt . Moreover, the solution is globally
asymptotically stable.

Theorem 9.5 (See [102]) The unique solution ūopt to the relaxed system optimiza-
tion problem (9.14) is globally asymptotically stable for the system (9.15).
∂V (u)
Proof Since ūopt is the unique solution to the equations ∂u p
i
= 0, ∀i ∈ M, it
follows that it is the unique equilibrium point of the system (9.15). Next, from Vp (u)
9.4 Numerical Study of Efficiency in Nash Game 151

being strictly convex it follows that ūopt is the global minimum point of the function
Vp (u). Let C = Vp (ūopt ). Then Vp (u) > C for all u = ūopt , and we construct a
Lyapunov function for the system (9.15):

V (u) = Vp (u) − C

It follows that V (u) = 0 when u = ūopt , and that V (u) > 0 when u = ūopt . That is,
the function V (u) is positive definite with respect to the equilibrium point u = ūopt .
Taking the derivative of V (u) along the trajectory of the system gives
 ∂   
∂
2
V̇ (u) = Vp (u) · u̇i = ki Vp (u)
∂ui ∂ui
i∈M i∈M

Since V̇ (u) = 0 when u = ūopt , and V̇ (u) < 0 when u = ūopt , it follows that V̇ (u)
is negative definite with respect to the equilibrium point u = ūopt . The conclusion
follows by Lyapunov stability theory. 

Note that the unique solution of the relaxed system problem (9.14) may arbitrar-
ily closely approximate the optimal solution of the original system problem (9.10)
with an approximate selection of the barrier function. For details the reader s re-
ferred to [101].

9.4 Numerical Study of Efficiency in Nash Game

Next we show how the system optimization framework can be used to investigate
the effects of parameters for the game-theoretic framework. We motivate such an
approach by first showing that the aggregate cost function in a game-theoretic for-
mulation is not always strictly convex. An individual cost function in the system
optimization formulation has an approximate interpretation a game cost function.
Based on this a numerical study of the effects of parameters in a game setup can be
performed [103, 104].
Recall that for a game G(M, Ω̂i , Ji ) as formulated in Chap. 7, the cost function is
composed of a pricing function and a utility function. The utility function is defined
so as to indicate each channel’s preference for a better OSNR. We are interested to
know how much cost is added or utility is lost due to each player’s selfish behavior
in a Nash game, i.e., we consider the loss of efficiency versus the socially optimal
cost. Among many possible social welfare functions, the aggregate cost function
can be considered as a candidate

J (u) := Ji (u)
i∈M

Note that while the individual costs are convex in each player’s action ui , in gen-
eral the aggregate game cost function is not necessarily convex in u. The following
simple example shows this. For simplicity we omit the penalty term and noise in the
OSNR model.
152 9 Nash Equilibria Efficiency and Numerical Studies

Example 9.6 Consider a Nash game with three players (m = 3) with individual
costs,
ui
Ji (u) = ui − ln  , i = 1, 2, 3
j =i uj

It follows that

∂ 2 J1 u2 + u3
= >0
∂u21 u1
∂ 2 J2 1
=− <0
∂u1 2 (u1 + u3 )2
∂ 2 J3 1
=− <0
∂u1 2 (u1 + u2 )2

Therefore,

∂ 2J u2 + u3 1 1
= − −
∂u12 u1 (u1 + u3 )2 (u1 + u2 )2

∂2J
Note that has not a definite sign, so that J (u) is not always convex with respect
∂u21
to u1 , even though J1 (u) is strictly convex with respect to u1 .

Thus using a standard approach of setting a constrained minimization problem

associated with this aggregate cost function is not immediate. Alternatively, recall
the system optimization problem OPT(Ω, C) and consider a cost function Ci (ui )
for channel i defined as

Ci (ui ) = αi ui − βi ln ui , (9.17)

where αi > 0 and βi > 0. This cost function Ci (ui ) is obviously strictly convex and
continuously differentiable. Moreover Ci (ui ) → +∞ as ui → 0. Comparing to Ji
in the above example it can be seen that Ci (ui ) in OPT(Ω, C) has an interpretation
similar to the one of the cost function Ji (u) in G(M, Ω̂i , Ji ), and moreover Ci is
uncoupled in u for a given set of other power u−i . Based on the relation between
these two formulations, i.e., OPT(Ω, C) and G(M, Ω̂i , Ji ), we can use the central
cost function in OPT(Ω, C) as a welfare function of G(M, Ω̂i , Ji ).
This is shown next, based on the system optimization being used to measure
the efficiency of the NE solution numerically. We compare by simulation the two
approaches: system optimization approach and the game-theoretic approach, re-
spectively. Consider a link with six channels (m = 6) and total power target is
9.4 Numerical Study of Efficiency in Nash Game 153

P 0 = 2.5 mW (3.98 dBm). Then the system matrix Γ is obtained as

⎡ ⎤
7.463 7.378 7.293 7.210 7.127 6.965
⎢ 7.451 7.365 7.281 7.198 7.115 6.953 ⎥
⎢ ⎥
⎢ 7.438 7.353 7.269 7.186 7.103 6.942 ⎥
Γ =⎢
⎢ 7.427 7.342 7.258 7.175 7.093 6.931 ⎥ × 10 .
⎥ −5
⎢ ⎥
⎣ 7.409 7.324 7.240 7.157 7.075 6.914 ⎦
7.387 7.303 7.219 7.136 7.055 6.894

Within the set of six channels, there are two levels of OSNR target, a 26 dB
level desired on the first three channels and a 22 dB OSNR level on the next
three channels. Conditions (9.5) and (9.6) on the target OSNR are satisfied. So
the feasible constraint set is non-empty. The coefficients in (9.17) are selected as
αi = 1, i = 1, . . . , 6, and β = [0.5, 0.51, 0.52, 0.3, 0.31, 0.32]. Recalling the rela-
tionship between channel OSNR and channel input power shown in Fig. 9.1, the
values of βi implicitly indicate channel OSNR preferences. The coefficient ki is
fixed for each channel with ki = 0.01, i = 1, . . . , 6 and we select a barrier function
6
λi (xi ) = 1000 max{0, b̂i − xi } , (9.18)

where xi (u) = rowi (T)u. Notice that λi (ui ) is zero when the constraints are satis-
fied. So there is a penalty with any violation of the constraints.
Let the NE solution of G(M, Ω̂i , Ji ) be denoted by u∗ . Algorithm GA 7.36 (see
Chap. 7) is used, i.e.,
 
1 βi ai /ui (t)
u̇i (t) = −μ αi + 0  − ,
(P − j ∈M uj (t))2 ( OSNR 1
(t) + a i − Γi,i )
i

where the step-size is selected as μ = 0.01.

Remark 9.7 Theorem 7.7 states that GA converges to the NE solution if (7.40) and
(7.41) are satisfied, where umin > 0 is a positive lower bound on each ui . A lower
bound on βi is set by (7.41). Since each channel attempts to select larger βi for
the purpose of higher OSNR, the lower bound does not affect the results of the
efficiency study.

In OPT(Ω, C), instead of using a generic Ci (ui ), (9.17) is used as the cost func-
tion for channel i, based on the primal algorithm (9.15) and the barrier function as
in (9.18). Thus the equilibrium point of (9.15) closely approximates the solution of
OPT(Ω, C). Initial channel powers are selected for both cases as
u(0) = [0.216 0.221 0.226 0.231 0.236 0.241] (mW)

The approximate optimal solution of OPT(Ω, C) is achieved as

uopt = [0.5 0.51 0.52 0.3 0.31 0.32] (mW)

so that system cost with respect to uopt is C(uopt ) = 4.5789.

154 9 Nash Equilibria Efficiency and Numerical Studies

Fig. 9.5 uT and OSNRi in game with αi = 0.001

We first present three cases in which the parameter selection strategy is not used
as a guideline (thus it is possible that the game settles down at an NE solution where
channels do not reach their OSNR targets). In all cases, the user-defined parameters
βi in G(M, Ω̂i , Ji ) are chosen as same as βi in OPT(Ω, C). A fixed αi is used
of 0.001, 1, and 20, respectively. With these pricing mechanisms, the total power
(uT ) vs. iteration and channel OSNR vs. channel number are shown in Figs. 9.5, 9.6
and 9.7 for three cases.
Thus without proper pricing mechanism, OSNR targets may not be achieved for
some channels or all channels, while the link capacity constraint is satisfied in all
cases. Furthermore, we notice that the penalty term

1

P0 − j ∈M uj

in G(M, Ω̂i , Ji ) plays a key role with small αi . In other words, with larger αi (say,
αi = 20 in the third case), total power is smaller than the link capacity constraint.
While with smaller αi in the first two cases, total power approaches the constraint
and higher channel OSNR is possibly achieved.
Channel powers, uopt and u∗ , inthree games versus channel number are shown
in Fig. 9.8. System cost C(u) = i∈M Ci (ui ) is evaluated via uopt and u∗ , re-
spectively and is shown in Table 9.1. Results imply that larger αi degrades system
performance and even violates the system constraints.
Next three other cases are shown in which proper pricing mechanisms are chosen
such that OSNR targets for all channels are achieved. In the game-theoretic frame-
9.4 Numerical Study of Efficiency in Nash Game 155

Fig. 9.6 uT and OSNRi in game with αi = 1

Fig. 9.7 uT and OSNRi in game with αi = 20

work G(M, Ω̂i , Ji ), the parameter selection strategy (7.28) is used such that proper
pricing mechanisms are chosen and OSNR targets for all channels are achieved.
Although the parameter selection strategy acts as a guideline for the selection of
156 9 Nash Equilibria Efficiency and Numerical Studies

Fig. 9.8 Comparison: channel power in games

Table 9.1 System cost

values with different αi C(u∗ ) C(uopt )

αi = 0.001 4.7403 4.5789

αi = 1 4.9282 4.5789
αi = 20 9.7804 4.5789

Table 9.2 Parameters: βi

βi

Game [a] [3.8 4.8 5.8 2.6 3.0 3.5]

Game [b] [5.5 7.0 9.4 4.0 4.5 5.0]
Game [c] [10 12 15 8.4 8.5 8.3]

each βi , it is practically intractable. Parameters αi are set at 1 for all cases and βi
are selected as in Table 9.2 such that different pricing mechanisms are chosen for
G(M, Ω̂i , Ji ).
Since we do not use Monte Carlo method [44] to simulate, we select βi in three
games by using the following rules. Firstly, βi increases for each channel, i.e., Game
[c] has the largest βi compared to Game [a] and Game [b]. Secondly, Game [b] has
the largest ratio of βi to βmin .
The efficiency of these two solutions u∗ and uopt , is compared by evaluating the
system cost C(u). The corresponding system cost values are obtained and shown in
Table 9.3.
9.4 Numerical Study of Efficiency in Nash Game 157

Table 9.3 System cost

values with different βi C(u∗ ) C(uopt )

Game [a] 4.6171 4.5789

Game [b] 4.6216 4.5789
Game [c] 4.6057 4.5789

Fig. 9.9 Comparison: total power

The results in Table 9.3 (compared with Table 9.1) show that the efficiency in
the solution of the Nash game (user optimization) can be improved by a proper
pricing mechanism. It is a well-known that no full efficiency can be obtained from
results in economics [38], transportation [129] and network resource allocation [64].
However, the Nash game solution gets very close to the optimal solution for system
optimization (see Table 9.3).
Figure 9.9 shows total power versus iteration. Channel power and channel OSNR
versus channel number are shown Figs. 9.10 and 9.11, respectively. Both constraints
(link capacity constraint and channel OSNR target) are satisfied in all cases. From
Fig. 9.11 it is seen that among the three cases, channel final OSNR values in Game
[c] get closest to the optimal solution of OPT(Ω, C).
We recall that the parameters βi in the Nash game are upper-bounded by the
condition (7.23) in Theorem 7.4, or in other words, the ratio of βi to βmin is upper
bounded. This condition restricts each channel asking unilaterally for a much higher
OSNR target than others. This phenomenon is also reflected in the selections of βi
in three cases. We take Game [b] for an example. In this case, β3 = 9.4 which is
greatly larger than other βi , indicating that channel 3 asks for a highest OSNR level.
158 9 Nash Equilibria Efficiency and Numerical Studies

Fig. 9.10 Comparison: channel power

Fig. 9.11 Comparison: channel OSNR

Meanwhile, channel 4 has the smallest β4 = 3.9. Thus the largest ratio of βi to βmin
in Game [b] is around 2.41, which is the largest ratio among the three cases (2.23,
2.41, 1.81, in Game [a], [b], [c], respectively). Recall that Game [b] results in the
9.5 Notes 159

largest system cost value C(u∗ ), 4.6216, which implies a loss of efficiency of the
Nash equilibrium in the case when βi deviates from the average.

9.5 Notes
In this chapter we discussed and alternative constrained OSNR optimization from
the perspective of system performance. This system optimization framework can
be used to measure the efficiency of Nash equilibria of the Nash game presented
in Chap. 7. A numerical approach can be used to study the effects of parameters
in individual game cost functions. Results show that the OSNR target in the game-
theoretic framework can be achieved and the efficiency can be possibly improved
by appropriate selection of parameters.
Chapter 10
Simulations and Experimental Studies

Abstract This chapter provides simulation and experimental results for various al-
gorithms studied in previous chapters. The first section describes the physical setup.
This is followed by simulations and experimental results based on implementing
iterative algorithms in Chap. 7 for a Nash game with two, three, and five channels,
respectively, in a point-to-point link topology. The last section presents results for
partitioned Nash game framework in a multi-link topology and a quasi-ring topol-
ogy, based on implementing the hierarchal algorithms studied in Chap. 8. Full use of
the flexibility of channel power adjustment at each optical switch is assumed and the
multi-link is partitioned into stages with single links. Simulation and experimental
results are given for each type of network topology.

10.1 Introduction

In the following we present simulation and experimental results for the various algo-
rithms studied in some of the previous chapters. Simulations are done in MATLAB
while the experimental results are performed on a physical setup described below.
An optical network test system (ONTS) composed of the following optical
devices has been used: stabilized light sources (LS), variable optical attenuators
(VOA), optical spectrum analyzer (OSA), optical Erbium-Doped Fiber Amplifier
(OA), tunable bandpass fiber optic filters, fiber optical couplers and an ASE broad-
band source. Parts of the ONTS are shown in Fig. 10.1.
Any transmitter (Tx) is composed of a light source (of a particular wavelength)
and a variable optical attenuator. Each channel input power is adjustable by setting
the value of the corresponding VOA. An OSA provides accurate and comprehensive
measurement capabilities for spectral analysis, and can be used in ONTS to mea-
sure channel optical powers and OSNRs. An OA amplifies optical signals and noise
simultaneously. Tunable filters are used to adjust the center wavelength of a narrow
passband so that it separates optical signals with different wavelengths. Fiber optical
couplers are used to combine or distribute optical powers from single (or multiple)
input to single (or multiple) output. Couplers are designed bi-directionally and thus
can be used as a coupler or a splitter. Input optical noise in ONTS is obtained by
using the ASE broadband source.

L. Pavel, Game Theory for Control of Optical Networks, 161

Static & Dynamic Game Theory: Foundations & Applications,
DOI 10.1007/978-0-8176-8322-1_10, © Springer Science+Business Media New York 2012
162 10 Simulations and Experimental Studies

Fig. 10.1 ONTS: devices

Results have been implemented in LabVIEW1 is used for the purpose of com-
munication and control. Communication between instruments and computers is re-
alized via GPIB (General Purpose Interface Bus, also referred to as IEEE-488).
LabVIEW first communicates with the light sources to initialize them (e.g., wave-
length selection and light source power setting). This is followed by communication
with VOAs to get and set the attenuation configurations and then with OSAs to
measure the OSNR level and output power of each channel. Distributed optimiza-
tion algorithms are embedded in the control block by using the MathScript Node in
LabVIEW.
The ONTS with a single optical link is setup shown in Fig 10.2.

10.2 Point-to-Point Link Topologies

The first section presents results for implementation of the iterative algorithms in
Chap. 7 for point-to-point links, either simulations in MATLAB or experimental
results on the ONTS physical setup.

1 LabVIEW (short for Laboratory Virtual Instrumentation Engineering Workbench). LabVIEW,

as a platform and development environment for a visual programming language from National
Instruments® .
10.2 Point-to-Point Link Topologies 163

Fig. 10.2 ONTS configuration setup: single link

10.2.1 Simulations Results

Consider multiple channels transmitted over a single optical link with a total power
capacity constraint P 0 = 2.5 mW (3.98 dBm). The associated system matrix Γ for
m = 3 is obtained as
⎡ ⎤
0.7463 0.7378 0.7293
Γ = ⎣ 0.7451 0.7365 0.7281 ⎦ × 10−4
0.7438 0.7353 0.7269

For the iterative algorithms, the channel parameters in cost functions are se-
lected satisfying both the sufficient conditions for the existence of a unique NE
solution, (7.22)–(7.24), and the conditions for stability of the update scheme,
(7.40)–(7.41). Simulations are repeated with the following selected parameters:
α = [0.001, 0.001, 0.001], β = [1, 3, 2] and a = [1, 1, 1].
Two distributed algorithms are implemented in simulation. At each iteration time
(n + 1), ui (n + 1) is obtained via PUA, i.e., via solving (7.33),
1 βi ai
αi +  =
(P 0 − j ∈M,j =i uj (n) − ui (n + 1))
2
( OSNRi (n) − Γi,i + ai )ui (n + 1)
1

A discretized version of GA (7.36) is used to compute ui (n + 1):

 
1 βi ai
ui (n+1) = ui (n)−μ αi + 0  − ,
(P − j ∈M uj (n))2 ( OSNR 1
+ ai − Γi,i )ui (n)
i (n)

where μ = 0.01. In addition, the initial channel power value in GA are set as u(0) =
[0.116 0.121 0.126] (mW).
A Nash game with two channels is considered first. Two channels compete for
the power resources and the game settles down at the NE solution via PUA and GA.
Figures 10.3, 10.4 and 10.5 show the evolutions of channel input power, total power
and OSNR. It can be observed that wide fluctuations are obtained when running
PUA. These fluctuations are largely avoided when using GA. Moreover, during the
iterative process, the total power constraint is violated when using PUA, but not
when using GA.
164 10 Simulations and Experimental Studies

Fig. 10.3 PUA and GA: channel power

Fig. 10.4 PUA and GA: total power

Next we consider a noncooperative game with three channels (m = 3). Input

power, total input power and OSNR vs. iteration time are plotted in Figs. 10.6, 10.7
and 10.8, respectively. Throughout the adjustments, total input power does not ex-
ceed the link capacity constraint, in accordance with the fact that the trajectory lies
in Ω̄ when the initial state is in Ω̄δ . Moreover, total input power reaches the equi-
librium point more quickly than the input powers and is below the power capacity
constraint, P 0 . These facts partly reflect that the extra term gives only a small ef-
fect on the cost rather than distorting the cost, however, it prevents channels from
10.2 Point-to-Point Link Topologies 165

Fig. 10.5 PUA and GA: channel OSNR

Fig. 10.6 GA: channel power

making full use of input power resource. Simulation results showed above prove the
performance of the algorithms and validate the analytic results.

10.2.2 Experimental Results

Consider the ONTS with a single optical link as shown in Fig. 10.2, with one OA.
Multiple channels are transmitted with the following wavelengths: 1533.47 nm,
166 10 Simulations and Experimental Studies

Fig. 10.7 GA: total power

Fig. 10.8 GA: channel OSNR

1535.04 nm, 1537.40 nm, 1555.75 nm and 1558.17 nm. We continue using the no-
tations defined in Chap. 7. Since only one OA is used, i.e., N = 1 in (6.17), diagonal
elements of the system matrix Γ are derived from (6.17), i.e.,

ASEi
Γi,i = ,
P0
10.2 Point-to-Point Link Topologies 167

where P 0 = 1.5 mW is the constant total power target of this link and ASEi is the
ASE noise, defined as
ASEi = 2nsp (Gi − 1)hνi B,
where
nsp amplifier excess noise factor, nsp = 1
h Planck’s constant, h = 6.626
B optical bandwidth, B = 1010
νi optical frequency of channel i
Gi gain of OA at the wavelength of channel i
The two algorithms, PUA and GA, are applied. In PUA, each ui (n + 1) is ob-
tained by solving the following equation:

1 βi ai
αi +  =
(P 0 − j ∈M,j =i uj (n) − ui (n + 1))
2
( OSNRi (n) − Γi,i + ai )ui (n + 1)
1

In GA,
 
1 βi ai
ui (n+1) = ui (n)−μ αi +  − ,
(P 0 − j ∈M uj (n))
2 1
( OSNR + ai − Γi,i )ui (n)
i (n)

where the step-size μ is selected as μ = 0.0025 for m = 2 and μ = 0.0002 for

m = 5. We set αi at 0.01 and ai at 1, respectively, for each channel. The following
values are chosen for the channel (player) specific parameters: βi , β = [10, 20] for
m = 2 and β = [5, 10, 20, 25, 30] for m = 5.
Using the results in Chap. 7, PUA is applied to a Nash game with two channels
(m = 2). GA is applied to two Nash games: one with two channels (m = 2) and one
with five channels (m = 5), respectively.
Firstly, a Nash game with two channels (m = 2) is considered: two channels
compete for the power resources via PUA and GA, respectively. Experimental data
are plotted in MATLAB. Figures 10.9 and 10.10 show the evolutions of channel
input power and total power, respectively.
Experiment results show that fluctuations in PUA are largely avoided in GA. We
note some difference between results of GA and PUA in Fig. 10.9, which does not
exist in simulation results. This is justified as follows: in ONTS, when running PUA,
we solve the first-order condition equation by using the measured OSNR value and
input power such that the new input power is achieved. A calibration for the insertion
(power) loss of equipment has to be included in the control block. However, the
measured data are not accurate even with calibration. Unlike this, when using GA
this issue is not encountered. A snapshot of final results on OSA after PUA settling
down is shown in Fig. 10.11.
Next the Nash game with five channels (m = 5) is considered. The evolutions of
channel input power and total power are shown in Figs. 10.12 and 10.13, respec-
tively.
168 10 Simulations and Experimental Studies

Fig. 10.9 Evolution of input power: m = 2

Fig. 10.10 Evolution of total power: m = 2

Since all channels are under same conditions, for example, same noise profile
and same network price αi , they compete for the power resource via the value of
βi : the larger the βi is, the higher the OSNR and input power values are obtained.
Snapshots of initial states and final results on OSA are shown in Figs. 10.14 and
10.15, respectively.

10.3 Multi-link and Mesh Topologies

In this section, we consider the partitioned Nash game framework to study several
types of network topology and implement the hierarchical algorithm to compute its
solution. The first two types of network topology are the multi-link topology and
the quasi-ring topology, [18]. Both of them are representative for selected paths ex-
tracted from a mesh configuration. A multi-link topology was studied in Chap. 8;
10.3 Multi-link and Mesh Topologies 169

Fig. 10.11 Snapshot on OSA: results of PUA

Fig. 10.12 Evolution of

input power with GA, m = 5

herein we also fully use the flexibility of channel power adjustment at each optical
switch and partition the multi-link structure into stages with single links. A mesh
network topology is studied next. Simulation results are given for each type of net-
work topology.
Some assumptions and parameters defined in simulation for all types of network
topology are presented first. As before, in all topologies, we assume that each link
is with same number of amplified spans and all optical amplifiers deployed along
links have the same gain spectral shape. The dynamic adjustment parameter γl,i is
bounded within [γmin , γmax ] = [0, 10] for all l ∈ L and for all i ∈ M. In partitioned
170 10 Simulations and Experimental Studies

Fig. 10.13 Evolution of total

power with GA, m = 5

Fig. 10.14 Snapshot on OSA: results of GA, m = 5

Nash games, the individual channel cost function Jl,i is with the following parame-
ter selection:

αl,i = 10 × Γli,i ,
βl,i = 1 + 0.1 × i,
al,i = 50 × l × Γli,i , i ∈ Ml , l ∈ L,
10.3 Multi-link and Mesh Topologies 171

Fig. 10.15 Snapshot on OSA: results of GA, m = 5

Fig. 10.16 Three-player game on a three-link topology

where Γli,i is the diagonal element in the link system matrix Γl . Values of Γli,i
are obtained in each individual network topology. Note that the condition (8.36)
is satisfied.

10.3.1 Multi-link Topologies

We implement the partition approach to a simple multi-link topology with three

links and three channels shown in Fig. 10.16. Initially channels 1 and 2 are added
on link 1 and channel 3 is added on link 2.
Constrained game problems in multi-link topologies were studied in Chap. 8 by
introducing a partitioned Nash game with stages. We consider the case in which
channel powers are adjustable at each optical switching node. A partition method is
172 10 Simulations and Experimental Studies

Fig. 10.17 Multi-link case:

channel input power on link 1

used in which each link is a stage. This partition simplifies the partitioned structure
and the convexity condition is naturally satisfied, as we discussed before.
We present MATLAB simulation results for the topology shown in Fig. 10.16 by
applying the iterative hierarchical algorithm (8.39), (8.40). Link total power targets
are P10 = 1.5 mW, P20 = 2.5 mW and P30 = 2.0 mW. Diagonal elements of each
link system matrix Γl , l = 1, 2, 3, are obtained as

Γ11,1 = 1.139 × 10−4 , Γ12,2 = 3.604 × 10−4 ,

Γ21,1 = 6.804 × 10−5 , Γ22,2 = 2.162 × 10−4 , Γ23,3 = 6.839 × 10−4 ,
Γ31,1 = 8.505 × 10−5 , Γ32,2 = 2.703 × 10−4 ,

while the step-size in the link algorithm is η = 0.8.

After partitioning, the stage Nash game on link 1 is played first. Figure 10.17
shows the evolution vs. iteration of channel input power on link 1.
For every 20 iterations, the link adjusts its price via the link algorithm and then
channels readjust their powers. Figure 10.18 show the evolutions of total power and
link price. After the Nash game on link 1 settles down, the stage Nash game on
link 2 starts to play, followed by the game on link 3. These evolutions are shown in
Figs. 10.19, 10.20, 10.21 and 10.22.
The following are the final achieved values for the adjustable parameters:
⎡ ⎤
0 0 0
γ ∗ = ⎣ 3.6429 0.6240 0 ⎦
5.1592 1.7449 0

For channels added directly from Tx, we set the γ value to 0, since channel power is
adjusted at Tx. Thus γ ∗ is feasible with respect to the predefined range [γmin , γmax ].
10.3 Multi-link and Mesh Topologies 173

Fig. 10.18 Multi-link case:

total power and link price on
link 1

Fig. 10.19 Multi-link case:

channel input power on link 2

10.3.2 Quasi-ring Topologies

In multi-link topologies, links are inter-connected in a ladder-nested manner and the

L link Nash games can be automatically played in a precedence or parallel order.
Situations are different in quasi-ring topologies. We take the quasi-ring topology in
Fig. 8.6 for an example. The optical paths of channels 1 and 3 are l1 → l2 → l3 and
l3 → l1 → l2 , respectively. Each of the links is the intermediate or the end of some
channel optical path. We partition this structure into three stages and each stage is
a link. A partitioned Nash game is formulated composed of three-link Nash games.
On each link Nash game, the convexity of link capacity constraints is automatically
satisfied. By breaking the closed loop and selecting one link as the start, link Nash
games can be played sequentially.
174 10 Simulations and Experimental Studies

Fig. 10.20 Multi-link case:

total power and link price on
link 2

Fig. 10.21 Multi-link case:

channel input power on link 3

Let us take a closer look at the simple quasi-ring topology in Fig. 8.6(a) and three
channels whose optical paths are shown in Fig. 8.6(b). We break the closed loop and
select link 3 as the starting link. The unfolded configuration is shown in Fig. 10.23.
The overall recursive process is such that stage Nash games on links are played
sequentially: l3 → l1 → l2 . On link 3, the adjustable parameters for channels 1 and
2 are initially set as γ3,11 and γ 1 , respectively, where the superscript 1 indicates
3,2
the number of iteration of the game among links. The game on link 3 settles down
at u∗3 (μ∗3 ) with the corresponding channel output power p3∗ (μ∗3 ). Sequentially, the
game on link 1 is played with an NE solution, u∗1 (μ∗1 ). Then channel output power
is p1∗ (μ∗1 ). Given p3∗ (μ∗3 ), and the adjustable parameter on link 1, γ1,3∗ , is deter-

mined. The game on link 2 is played after that. The NE solution of this game is
u∗2 (μ∗2 ) and the channel output power is p2∗ (μ∗2 ). Then the adjustable parameters on
10.3 Multi-link and Mesh Topologies 175

Fig. 10.22 Multi-link case:

total power and link price on
link 3

Fig. 10.23 Unfolded quasi-ring topology with the starting link, link 3

∗ , i = 1, 2, are determined. With the given p ∗ (μ∗ ), link 3 determines its

link 2, γ2,i 2 2
∗ = u∗3,i (μ∗3 )
adjustable parameters by γ3,i ∗ (μ∗ ) ,
p2,i i = 1, 2.
2
We present MATLAB simulation results for the topology shown in Fig. 8.6. Link
total power targets are P10 = 1.5 mW, P20 = 2.5 mW and P30 = 2.0 mW. Diagonal
elements of each link system matrix Γl , l = 1, 2, 3, are obtained as

Γ11,1 = 2.166 × 10−4 , Γ12,2 = 6.852 × 10−4 , Γ13,3 = 2.2 × 10−3 ,

Γ21,1 = 3.611 × 10−4 , Γ22,2 = 1.100 × 10−3 ,
Γ31,1 = 2.708 × 10−4 , Γ32,2 = 8.565 × 10−4 , Γ33,3 = 2.7 × 10−3

A step-size of η = 0.1 is used in the link algorithm. The partitioned Nash game is
played as described above. For every 20 iterations, the link adjusts its price via the
link algorithm and then channels readjust their powers. Evolutions in time of chan-
nel input power, total power and link price on each link l are shown in Figs. 10.24–
10.29, respectively.
176 10 Simulations and Experimental Studies

Fig. 10.24 Quasi-ring case:

channel input power on link 3

Fig. 10.25 Quasi-ring case:

total power and link price on
link 3

The adjustable parameters for three links and three channels are obtained as in
the following 3 × 3 matrix:
⎡ ⎤
0 0 0.7208
γ ∗ = ⎣ 7.5983 1.4868 0 ⎦
1.5277 0.6042 0

The overall game settles down since γ ∗ is feasible. Note that a different starting
link can be selected, say, link 1, such that games on links are played sequentially:
l1 → l2 → l3 . Typically we select the starting link where channels are added directly
from Txs.
10.3 Multi-link and Mesh Topologies 177

Fig. 10.26 Quasi-ring case:

channel input power on link 1

Fig. 10.27 Quasi-ring case:

total power and link price on
link 1

10.3.3 Mesh Topologies

We study a mesh network topology as shown in Fig. 10.30(a), where eight channels
are transmitted over six links. The channel routes are shown in Fig. 10.30(b).
It can be seen from Fig. 10.30(b) that there exists a closed loop among links 1,
4, and 6, which is formed by the optical paths of channels 3, 4, 5, and 6. We break
the closed loop and select link 1 as the starting link. This unfolded configuration is
shown in Fig. 10.31.
The overall recursive play process is described as follows. Games on links 1, 6,
and 4 are played in a precedence order: l1 → l6 → l4 . Since the closed loop among
links 1, 4, and 6 is unfolded, on link 1 the adjustable parameters for channels 5
178 10 Simulations and Experimental Studies

Fig. 10.28 Quasi-ring case:

channel input power on link 2

Fig. 10.29 Quasi-ring case:

total power and link price on
link 2

1 and γ 1 , respectively. The games on links 1, 2, and

and 6 are initially set as γ1,5 1,6
5 can be played in a parallel order, while the game on link 3 is played after games
on links 2 and 5 settle down. The last game to play is the one on link 4. After
all the games settle down on u∗l (μ∗l ) with corresponding channel output powers
p∗l (μ∗l ) and adjustable parameters γl∗ , link 1 re-determines its adjustable parameter
γ1∗ according to p4∗ (μ∗4 ) because of the closed loop among links 1, 4, and 6. Again
we note that link 1 is not the only choice for the starting link. Typically we select
the link where channels are added directly from transmitters as the starting link.
A user-defined step-size η = 0.6 is used in the simulation. Link total power
targets are P 0 = [2.0 1.5 2.0 2.5 1.5 1.5] mW. Diagonal elements of each link
system matrix Γl , l = 1, . . . , 6, are obtained as Γ13,3 = 0.0014, Γ14,4 = 0.0027,
Γ15,5 = 0.0027, Γ16,6 = 0.0013, Γ21,1 = 7.212 × 10−4 , Γ22,2 = 0.001, Γ31,1 =
10.3 Multi-link and Mesh Topologies 179

Fig. 10.30 Mesh network topology

Fig. 10.31 Unfolded mesh

topology with the starting
link, link 1

5.409 × 10−4 , Γ32,2 = 8.564 × 10−4 , Γ37,7 = 8.483 × 10−4 , Γ38,8 = 5.34 × 10−4 ,
Γ41,1 = 4.327 × 10−4 , Γ42,2 = 6.852 × 10−4 , Γ43,3 = 0.0011, Γ44,4 = 0.0022, Γ45,5 =
0.0022, Γ46,6 = 0.0011, Γ57,7 = 0.0011, Γ58,8 = 7.12 × 10−4 , Γ63,3 = 0.0018 and
Γ64,4 = 0.0036. We show the evolutions of channel input power, total power and
link price on links 1, 2, and 4 in Figs. 10.32–10.37, respectively. The final adjustable
parameter values for six links and eight channels are obtained as in the 6 × 8 matrix
below:
⎡ ⎤
0 0 0 0 0.7067 1.3548 0 0
⎢ 0 0 0 0 0 0 0 0 ⎥
⎢ ⎥
⎢ 0.6893 0.4308 0 0 0 0 0.6535 1.0415 ⎥
γ∗ =⎢ ⎢ 1.1210 0.7271 0.8486 0.4335
⎥
⎢ 0 0 0 0 ⎥ ⎥
⎣ 0 0 0 0 0 0 0 0 ⎦
0 0 2.5839 1.1778 0 0 0 0
180 10 Simulations and Experimental Studies

Fig. 10.32 Mesh case:

channel input power on link 1

Fig. 10.33 Mesh case: total

power and link price on link 1

10.3.4 Experimental Results

Next experimental results are presented on the same ONTS where this time three
links are setup as shown in Fig. 10.38. Each link is composed of an OA. Total
power targets are P10 = 1.5 mW, P20 = 2.5 mW and P30 = 1.5 mW, respectively.
Channels 1 and 2 are added on link 1 and channel 3 is added on link 2. We use
following wavelengths for each channel: 1535.04 nm on Light source 1 (LS 1),
1537.40 nm on Light source 2 (LS 2) and 1533.47 nm on Light source 3 (LS 3).
Channel 3 is dropped after link 2. This is realized by using a filter which is used to
perform wavelength selection. Channels 1 and 2 are transmitted through two filters,
respectively.
10.3 Multi-link and Mesh Topologies 181

Fig. 10.34 Mesh case:

channel input power on link 2

Fig. 10.35 Mesh case: total

power and link price on link 2

We partition this three-link game into two stage games (K = 2): Stage 1 is com-
posed of links 1 and 2 and stage 2 is link 3. Diagonal elements of each stage system
matrix Γk are derived from stage OSNR mode. Particularly in this configuration,

ASE1,i ASE2,i
Γ1i,i = + , i = 1, 2,
P10 P20
ASE2,3
Γ13,3 = ,
P20
ASE3,i
Γ2i,i = , i = 1, 2,
P30
182 10 Simulations and Experimental Studies

Fig. 10.36 Mesh case:

channel input power on link 4

Fig. 10.37 Mesh case: total

power and link price on link 4

where
ASEl,i = 2nsp (Gl,i − 1)hνi B,
with Gl,i the gain of OA at the wavelength of channel i on link l.
The value of the dynamic adjustment parameter γk,i is in fact the attenuation
value of channel i on stage k, which is physically bounded.
The iterative hierarchical algorithm developed in Chap. 8 is implemented, which
is composed of a channel algorithm and a link algorithm. At each stage k, the fol-
lowing channel algorithm is used:
 
βk,i 1 1 uk,i (n)
uk,i (n + 1) = − − − Γki,i ,
αk,i + μk,r(i) (t) OSNRk,i (n) OSNRk ,i ak,i
10.3 Multi-link and Mesh Topologies 183

Fig. 10.38 ONTS configuration setup: multiple links

Fig. 10.39 Evolution of

input power on stage 1

where stage k is the precedent of stage k and OSNRk ,i and μk,r(i) (t) are invariable
during the channel iteration in stage k. Then after every Nk iterations of the channel
algorithm, the new link price is generated according to the following link algorithm:
 +
μk (t + 1) = μk (t) − ηk Pk0 − Ek uk μ̄k (t) ,

where Pk0 is a modified coupled power constraint on stage k and

 +
μ̄k (t) = μk (t) − ηk Pk0 − Ek uk μk (t)

In the experiment, the game on Stage 1 is played first. Figure 10.39 shows the evo-
lution in iteration time of channel input power on Stage 1.
For every N1 = 10 iteration, link 1 and link 2 adjust their prices simultaneously
(link algorithm) and then channels readjust their powers. Total power and prices
evolutions are shown in Fig. 10.40. After Stage 1 settles down, the game on Stage 2
starts to play. For every N2 = 7 iteration, link 3 adjusts its price. The evolutions are
shown in Figs. 10.41 and 10.42. Snapshots of the values of OSNR and attenuation
184 10 Simulations and Experimental Studies

Fig. 10.40 Evolution of total power and price on stage 1

Fig. 10.41 Evolution of
input power on stage 2

Fig. 10.42 Evolution of total power and price on stage 2

10.3 Multi-link and Mesh Topologies 185

Fig. 10.43 Snapshot: partial evolution of OSNR and attenuation value on stage 2

during the iteration on Stage 2 are shown in Fig. 10.43. The final values of adjustable
parameters, i.e., the attenuation values are
 
∗ 0 0 0
γ =
4.454 4.073 0
 Part III
Robustness, Delay Effects, and Other
Problems
Chapter 11
Robustness and Delay Effects on Network
Games

Abstract This chapter presents results on robustness of network control algo-

rithms derived from game-theoretic formulations in the presence of time delay.
Mesh optical networks are distributed over large surface areas. Any realistic OSNR
model must account for these delays. Sufficiently large time delays may destabi-
lize the closed-loop systems that implement these game-theoretic-inspired control
algorithms. These algorithms need to have been appropriately adjusted so as to en-
sure closed-loop stability in the presence of time delays. We consider delay effects
in network games without constraints, followed by games with constraints. Then
we study a delayed primal–dual algorithm and perform a two time-scale stability
analysis. We conclude by considering robustness and delay effects combined.

11.1 Introduction

In the previous chapters we presented game-theoretic formulations and, based on

these, network control algorithms towards channel OSNR optimization. We dis-
cussed pricing strategies and selection of parameters that ensure optimal conver-
gence to the NE point without time delays. Due to large distances and propagation
delays, any realistic OSNR model must account for these delays. The figure below
shows a typical mesh topology with the control plane being the layer where control
algorithms will be implemented to adjust network parameters (Fig. 11.1). These pa-
rameters are given as reference values to the transport layer (physical layer). Links
between nodes span usually hundreds or thousands of kilometers hence propagation
time delay is non-negligible. Control algorithms need to be appropriately modified
to ensure closed-loop stability in the presence of time delays. In this chapter, we
address this problem.
We first present a multi-link OSNR model with time delay and the corresponding
closed-loop system based on game-theoretic algorithms. Next we consider delay ef-
fects in network games without constraints, followed by a treatment for games with
coupled constraints. We present an analysis for the delayed primal–dual algorithm
and a time-scale stability analysis for point-to-point topologies. We also indicate
how this can be extended to multi-link topologies. The last section is focused on
robustness to a combination of delay effects and uncertainties. Stability conditions

L. Pavel, Game Theory for Control of Optical Networks, 189

Static & Dynamic Game Theory: Foundations & Applications,
DOI 10.1007/978-0-8176-8322-1_11, © Springer Science+Business Media New York 2012
190 11 Robustness and Delay Effects on Network Games

Fig. 11.1 Example general optical network

are presented based on a frequency domain analysis and a Lyapunov–Razumikhin

function analysis. The material in this chapter is based on [145, 149, 150].

11.2 System Model

In this section we consider a multi-link OSNR model with time delay and the corre-
sponding closed-loop system based on game-theoretic algorithms. The OSNR mod-
els developed in Chap. 6 in Lemmas 6.2 and 6.9 (see (6.16), (6.36)) do not account
for time delays. Let us define the notation to express time delays for arbitrary multi-
link topology and for single links. Consider a general mesh network topology with
time delays. Each channel OSNRi , yi value is measured in real-time, and fed back
to its respective source to be processed by the control algorithms at Tx and adjust
the channel power ui .
f
Let us denote by τi,j the forward propagation delay of a signal from its source,
j ∈ M, to an OSNR output i. Similarly let τib denote the backward propagation
delay of a signal from the OSNR output, i, to its associated source, i. Note that a
signal from any channel, j , may interfere at the OSNR output of any other channel
f
i, so two indices are needed for the forward propagation delay, τi,j . However, even
in the mesh configuration, the OSNR output of a channel i is fed back only to the
source of channel i, and not to any other channel. Thus, for the backward propa-
gation delay, τib a single index is needed. Let us denote the combined round-trip
f
delay by τi,j = τi,j + τib . Round-trip propagation delay from a source i to its OSNR
output i, yi , and back to the source i is denoted by τi,i .
With these notations, from Lemmas 6.2 and 6.9, the following time-delayed
OSNR model is obtained.

Lemma 11.1 Consider the ith channel in an arbitrary multi-link optical network,
i ∈ M. Its OSNR value, fed back from Rx to Tx, evaluated at time t is denoted by
11.2 System Model 191

OSNRi or yi and is given as

ui (t − τi,i )
yi (t) =  (11.1)
n0,i + j ∈M Γi,j uj (t − τi,j )

where Γi,j , elements of the full (n × n) system matrix Γ , are defined as (6.36).

Note that (11.1) is the OSNR signal that is measured at Rx and then fed back
to Tx. Thus both the forward and the backward time delays are present, since the
OSNR value is fed back to the transmitter.
For a single-link case, as in Lemma 6.2, time-delay notation can be simplified
such that τ = τ f + τ b , where τ is the round-trip time delay of the link, τ f is the
forward propagation delay, and τ b is the backward propagation delay. Thus, from
Lemma 6.2, we obtain the following lemma.

Lemma 11.2 Consider the ith channel in a point-to-point optical link, i ∈ M. Its
OSNR value, fed back from Rx to Tx, evaluated at time t is denoted by OSNRi or yi
and is given as

ui (t − τ )
yi (t) =  , i∈M (11.2)
n0,i + j ∈M Γi,j uj (t − τ )

where Γ is the (n × n) link matrix with elements Γi,j , given by (6.15).

Next let us revisit the iterative algorithm for channel power. From Chap. 7, recall
that the algorithm (7.62) ensures that the closed-loop system converges to the NE
equilibrium when no time delays are considered. In order to find out under what
conditions this holds in the presence of time delay let us introduce tunable control
parameters, ρi , for each channel i, into the control algorithm (7.62) and convert into
it to continuous-time. Thus, for each channel i we subtract ui (n) from each side
of (7.62). The n time index is normalized with respect to the time period T0 , i.e. a
time period of T0 elapses between n and n + 1 iterations, i.e., tn = nT0 , where tn
is the total elapsed time at the nth algorithm update. We also introduce parameter
ρi , 0 < ρi ≤ 1, multiplying the right-hand side of (7.62), and we approximate the
left-hand side by dudti (t) = ui (n + 1) − ui (n), to obtain
#   *
dui (t) βi 1 1
= ρi − − Γi,i + ai ui (t) (11.3)
dt αi ai yi (t)

where t is the independent continuous-time variable. The above approximation as-

sumes that the algorithm is updated quickly enough that we may work with t rather
than discrete time periods tn . Note that this continuous-time approximation in (11.3)
is relatively simplistic, and may be replaced by other approaches. For example, bi-
linear transformation could be used to create a mapping between continuous-time
and discrete-time domain. Alternatively, one could study the stability of (7.62) and
192 11 Robustness and Delay Effects on Network Games

(11.1) within the discrete-time domain. However, as the majority of time-delay sta-
bility results in literature in continuous-time, this is the approach we take also.
Consider further that (11.3) is modified by substituting ui (t) for ui (t − τi,i ),
which yields
#     *
dui (t) βi 1 1 Γi,i
= ρi − ui (t − τi,i ) + − 1 ui (t) (11.4)
dt αi ai yi (t) ai
This is used so as to simplify the closed-loop system’s structure. We explicitly write
this as the following assumption.

Assumption 11.1 Each channel, i, has knowledge of its own round-trip time de-
lay, τi,i .

Assumption 11.1 is realistic in practice since round-trip time delays, which are
due to signal propagation times, are measurable. Furthermore, the decentralized na-
ture of the control algorithm (11.3) is also preserved since this added information
is local. Because the control algorithm is implemented in software, this substitution
as in (11.4) is allowed. Furthermore, the channel algorithm running at the channel
sources may store the signal powers, u(t), over time. Thus, with τi,i known, it is
used to reference the appropriate past channel power u(t − τi,i ). Hence, (11.4) is a
modified channel algorithm of (7.62) to handle time delays more effectively. Note
that while the control algorithm (11.4) can be modified, the OSNR model (11.1),
which is the physical model of the optical network, cannot be modified.

11.3 Delay Effects in Network Games Without Constraints

In this section we consider games without coupled constraints so that we apply only
the primal algorithm (channel) without the dual algorithm (link price). Let us first
obtain a closed-loop system that incorporates time delays by substituting the time-
delayed OSNR model (11.1) into (11.4). This yields
#      *
dui (t) βi 1 Γi,i
= ρi − n0,i + Γi,j uj (t − τi,j ) + − 1 ui (t) (11.5)
dt αi ai ai
j ∈M

or, after shifting it around the equilibrium point u∗ , (7.11) (see Chap. 7), the equiv-
alent form is
  
dui ρi
=− (ai − Γi,i )ui (t) + Γi,j uj (t − τi,j ) (11.6)
dt ai
j ∈M

This (11.6) is the closed loop that is analyzed next. Specifically, we give sufficient
conditions for stability of the time-delay closed-loop system (11.6). These condi-
tions are given in terms of tunable parameters ρi and ai . The approach is based on
11.3 Delay Effects in Network Games Without Constraints 193

an analysis as in [100], but unlike [100], the system does not have any symmetry in
the system matrix Γ that can be exploited.
First, let us map (11.6) into the form (A.33) (see the appendix) in order to study
its asymptotic stability as in Definition A.32. For convenience, we restate (A.33)
using the state variables and system parameters from (11.6) as
2

m
u̇(t) = A0 u(t) + Ak u(t − τk ), τk ≥ 0 (11.7)
k=1

for m channels in the system. Starting from (11.6), assume ai > Γi,i for all i, let
εi = ρaii (ai − Γi,i ), and define

A0 = diag(−εi )

In order to define Ak one can proceed as follows. Notice in (11.6), for channel i,
there may be up to m delay terms with the form − ρaii Γi,j u(t − τi,j ) for j = 1, . . . , m.
Each time delay, τi,j , may be unique for a total of m2 time delays for all i, j =
1, . . . , m. Thus, to rewrite (11.6) into the form (11.7), each coefficient − ρaii Γi,j ,
associated with the time-delayed state u(t −τi,j ), occupies one element of the m×m
matrix Ak for the associated time-delayed state u(t − τk ). Let k = (i − 1) × m + j
for i, j = 1, . . . , m and let Ak be an (m × m) matrix of zeros except for the (i, j )th
element corresponding to the k index which is equal to − ρaii Γi,j . Furthermore, let
τk = τi,j . With these A0 , Ak , and τk (11.6) is mapped directly to (11.7).
Then by applying Laplace transform to (11.7) we obtain
 2 m2  

m  0
−sτk −st
sI − A0 − Ak e U (s) = u(0) + Ak e dt e−sτk (11.8)
k=1 k=1 −τk

where U (s) is the Laplace transform of u(t). Similarly to (A.34), let

 2

m
Δ(s) = sI − A0 − Ak e−sτk (11.9)
k=1

The roots of the characteristic equation det(Δ(s)) = 0 determine the stability of

(11.7) or (11.8) (see Definition A.32). Consider Δ(s) in an equivalent form
 2

m
−1
Δ(s) = (sI − A0 ) I + (sI − A0 ) −Ak e−sτk
k=1

or
   
1
Δ(s) = (sI − A0 ) I + diag F (s)
s + εi
194 11 Robustness and Delay Effects on Network Games

m2 −sτk .
where F (s) = k=1 −Ak e Finally, we write

Δ(s) = (sI − A0 ) I + L(s) (11.10)

where L(s) = diag( s+ε

1
i
)F (s). Thus Δ(s) in (11.10) is expressed in a unity feed-
back form with loop transfer function L(s).
This loop (m × m) transfer function, L(s), has the (i, j )th element

Γi,j e−τi,j s
Li,j (s) = ρi Γi,i
(11.11)
ai s + ρi (1 −
ai )

Stability of (11.8) is equivalent to Δ−1 ∈ H∞ (see Definition A.32), where H∞

denotes the Hardy space of all complex-valued functions, analytic and bounded in
the open RHP, Re[s] > 0. Since εi > 0 by assumption, A0 is stable, so by (11.10),
it follows that Δ−1 ∈ H∞ if (I + L)−1 ∈ H∞ , where L is as above. Based on this
the following result, modified from [100] for εi = 0, gives sufficient conditions for
stability of (11.8).

Proposition 11.3 Suppose

(i) F (0) − A0 has all its eigenvalues in the open RHP, and
(ii) for every nonzero ω and every μ̄ ∈ (0, 1], I + μ̄L(j ω) is invertible.
Then (I + L)−1 ∈ H∞ .

We now state and prove a stability result (Theorem 11.6) for the closed-loop
system resulting from games with no coupled constraints; part (i) gives a time-delay
dependent stability condition based on the tuning parameters ρi ; part (ii) provides a
conservative stability condition independent of time delay. Before that we give two
useful lemmas.

Lemma 11.4 (See [145]) Let Li,i (j ω) be defined as

Γi,i e−τi,i j ω
Li,i (j ω) = ρi Γi,i
(11.12)
ai j ω + ρi (1 −
ai )

Furthermore, define ω = ω∗ as the frequency value that minimizes |μ̄Li,i (j ω) −

(−1)| over all frequencies for all μ̄ ∈ (0, 1]. Then,
      
1 − μ̄Li,i (j ω) ≤ μ̄Li,i j ω∗ − (−1) ∀ω ∈ 0, ω∗ , ∀μ̄ ∈ (0, 1] (11.13)

or equivalently, ∀ω ∈ [0, ω∗ ], ∀μ̄ ∈ (0, 1]

Γi,i μ̄ρi   
1− 0 ≤ μ̄Li,i j ω∗ − (−1) (11.14)
ai ω2 + ρ 2 (1 − Γ /a )2
i i,i i
11.3 Delay Effects in Network Games Without Constraints 195

Proof of Lemma 11.4 is immediate, based on the triangle inequality and the fact
that the magnitude of Li,i (j ω) is monotone with respect to ω.

Lemma 11.5 (See [145]) Let Li,i (j ω) be defined as in (11.12) and ω̃ = π

2τi,i . Then
ω̃ is guaranteed to satisfy (11.13), i.e. ω̃ ≤ ω∗ , if

0.68244
≤ ρi
(1 − Γ i,i
ai )τi,i

Theorem 11.6 (i) If the system design parameters ai are selected such that
 
Γi,j < ai < Γi,i + Γi,j , ∀i (11.15)
j =i j

Γi,i < ai , ∀i (11.16)

then the closed-loop time-delay system, (11.4), (11.1) is stable if the tuning param-
eters ρi are selected such that
0.68244
≤ ρi , ∀i (11.17)
(1 − Γ i,i
ai )τi,i
π
0 < ρi < 0  , ∀i (11.18)
Γ
2τi,i (1 − ai,ii ) Γi,i
2 /(a −
i j Γi,j )2−1

(ii) The closed-loop time-delayed system, (11.4), (11.1) is stable for any time delays
if the parameters ai are selected such that

Γi,i + Γi,j < ai , ∀i (11.19)
j

Proof We prove that (I + L(s))−1 ∈ H∞ by applying Proposition 11.3 to L(s). To

show condition (i), we apply Gershgorin’s theorem to F (0) + diag(εi ). Note, based
on the left side of (11.15), that we have
Γi,i Γi,i  Γi,j
ρi + ρi − ρi > ρi , ∀i
ai ai ai
j =i

or equivalently, j =i Fi,j (0) < |Fi,i (0) + εi |, and condition (i) follows. Next, we
show that condition (ii) is satisfied for L(j ω), (11.11) for s = j ω. We apply Gersh-
gorin’s theorem to show that no Gershgorin disc of μ̄L(j ω) contains the value −1
for any μ̄ ∈ (0, 1]. This is ensured if for all μ̄ ∈ (0, 1]

μ̄ρi  Γi,j  
0 < μ̄Li,i (j ω) + 1 (11.20)
ω2 + ρi2 (1 − Γi,i /ai )2 ai
j =i
196 11 Robustness and Delay Effects on Network Games

∀i, ∀ω ≥ 0 where Li,i (j ω) is defined in (11.12). The RHS of (11.20) is the distance
from the critical point, −1, to the Nyquist curve of μ̄Li,i (j ω). It can be shown that
if
1  Γi,j  
Γi,i
< μ̄Li,i (j ω) + 1, ∀i, ∀ω ≥ 0 (11.21)
(1 − )
ai
ai
j =i

holds, see (11.21), then (11.20) is satisfied.

(i) Let ω∗ be the value of ω that minimizes the RHS of (11.21). Then if the follow-
ing holds:
1  Γi,j   
Γi,i
< μ̄Li,i j ω∗ + 1, ∀i (11.22)
(1 − ) ai
ai j =i

it follows that (11.21), hence (11.20) holds, so −1 ∈/ eig(μ̄L(j ω)). We rewrite

the RHS of (11.22) into a more useful form based on Lemmas 11.4 and 11.5.
From Lemmas 11.4 and 11.5, ω̃ = 2τπi satisfies (11.14), as long as (11.17) is
satisfied. Next, (11.15), (11.17), and (11.18) guarantee that

1  Γi,j μ̄ρi Γi,i

Γi,i
<1− 0 (11.23)
(1 − ai ) j =i
ai ai 2 ) + ρ 2 (1 − Γ /a )2
π 2 /(4τi,i i,i i
i

Thus by (11.23) and (11.14) it follows that (11.22) and (11.21) hold, and hence
−1 ∈/ eig(μ̄L(j ω)). To show (11.23), note that μ̄ = 1 gives the smallest value
on the RHS of (11.23). After some manipulation, this can be rewritten as
   
Γi,i 2 Γ2 π2
ρi2 1 − i,i − 1 < (11.24)
ai (ai − j Γi,j )2 2
4τi,i

By (11.15) the third term on the LHS of (11.24) is positive and (11.24) can be
rearranged as in (11.18). Then (11.24) holds along with (11.23) and therefore
−1 ∈/ eig(μ̄Lii (j ω)). By Proposition 11.3, the τ -dependent stability is proved.
(ii) We return to (11.21). On the RHS, we have |μ̄Li,i (j ω) + 1| ≥ 1 − |Li,i (0)|
based on 0 < μ̄ ≤ 1 and the monotonicity of |Li,i (j ω)| with respect to ω. Thus,
|μ̄Li,i (j ω) + 1| ≥ 1 − Γi,i /(ai − Γi,i ) and from condition (11.19) it follows that
(11.21) holds, which implies (11.20) holds, and −1 ∈ / eig(μ̄L(j ω)).


Corollary 11.7 Both (11.17) and (11.18) can only be satisfied simultaneously in
the range
Γi,i 
+ Γi,j ≤ ai (11.25)
δ
j
0
where δ = ( 2(0.68244)
π
)2 + 1 = 2.5096.
11.4 Delay Effects in Network Games with Constraints 197

11.4 Delay Effects in Network Games with Constraints

In this section we consider the effect of time delay in network games with coupled
constraints. We consider a single link for simplicity and both channel and link al-
gorithm as in Chap. 7. For multi-link topologies extension of the results in Chap. 8
can be similarly to incorporate time delay and the reader is referred to [148]. We
present stability conditions for single links with time delays via primal–dual control
algorithms based on [149]. We consider primal–dual algorithms (or channel-link al-
gorithms) as developed for game with coupled constraints in Chap. 7. Compared to
the analysis in the previous section, the new component of pricing algorithm (link
algorithm) ensures that channel powers are below the threshold for nonlinear effects.
The added dynamics of the pricing or link algorithm produces a nonlinear closed-
loop system that requires a more advanced stability analysis than that presented
in the previous section. We present a singular perturbation approach [69] (see the
appendix for a review). This approach provides the means to analyze stability of
systems that operate on two separate time-scales.
First we formulate the continuous-time closed-loop system that incorporates time
delay. We follow with a Lyapunov–Razumikhin analysis combined with a singular
perturbation approach applied to this closed-loop system. The main stability result
is stated as Theorem 11.10. Material is based on [149] and [146], where detailed
proofs can be found. An approach based on Lyapunov–Krasovskii stability analysis
can be found in [150].

11.4.1 Delayed Primal–Dual Algorithm

In this section, we derive the closed-loop expression for the channel-price or primal–
dual algorithms (7.62) and (7.61) applied to the time-delayed OSNR model (11.2).
We then apply a coordinate shift that simplifies the foregoing stability analysis in
the following section.
Let us first briefly review the primal–dual algorithms developed in Chaps. 7
and 8. Recall that the channel algorithm (7.62) is located at the transmitters (Tx)
and the price algorithm or link algorithm (7.61) is located at the receivers (Rx) at
the output of the link. The control algorithm of the ith channel adjusts the channel
powers, ui , towards OSNR optimization using its OSNR value and channel price
μ as feedback. The link algorithm measures the total sum of the channel powers
and computes a channel price, μ, based on the threshold for nonlinear effects. The
channel price is fed back to the channel algorithms at Tx.
Let us consider the closed-loop system obtained by using the channel and price
algorithms and the OSNR model (11.2). Recall, from Chaps. 7 and 8 that the channel
algorithm operates on a faster time-scale reflected by the fact that its index, n, is
related to the index of the link algorithm, k̄, via n = K k̄, where K is large, e.g.,
K = 100. In continuous-time, we denote the “fast” time variable by t and the “slow”
time variable by tˆ. The two time-scales are related by tˆ = εt, where ε = K1 is small,
198 11 Robustness and Delay Effects on Network Games

and we will see later that the stability of the closed-loop system is ensured if ε is
sufficiently small. The time delays on the two time-scales are related similarly by
τ̂ = ετ . A smaller ε value means a more effective time-scale decoupling between
the channel and the price (link) algorithm.
Let us begin with the continuous-time channel algorithm (11.3) derived in
Chap. 7, where αi = μ(tˆ ) = μ(εt) is the channel price, rewritten below as (11.26)
#   *
dui (t) βi 1 1
= ρi − − Γi,i + ai ui (t) (11.26)
dt μ(εt) ai yi (t)
Now, substituting the time-delayed model (11.2) into (11.26) and including the
backward propagation delay, τ̂ b = ετ b , into the channel price, μ(tˆ ) = μ(εt), yields
#
dui (t) βi
= ρi
dt μ(ε(t − τ b ))
   *
1 n0,i + j ∈M Γi,j uj (t − τ )
− − Γi,i + ai ui (t) (11.27)
ai ui (t − τ )
Note that (11.27) and (11.26) are on the “fast” time-scale, t. For the single-link case
here, τ = τi,i , since there is only one round-trip time delay. The price is adjusted on
the “slow” time-scale; let us rewrite (11.27) in this slow time-scale, tˆ, so that both
the channel and link/price dynamics are on the same time-scale. Using tˆ = ε t, from
(11.27) we obtain
#
d ûi (tˆ ) βi
ε = ρi
d tˆ μ(tˆ − τ̂ b )
   *
1 n0,i + j Γi,j ûj (tˆ − τ̂ )
− − Γi,i + ai ûi (tˆ ) (11.28)
ai ûi (tˆ − τ̂ )

where we used ddttˆ = ε and û denotes u on the slow time-scale tˆ. This form is used
to apply standard singular perturbation theory (see Sect. A.9, (A.19)). If we divide
both sides of (11.28) by ε, then the 1ε term on the RHS is a large gain. Thus, the ε
term embodies the fast dynamics of the channel algorithm in the slow time-scale.
Let us move now to the price or link algorithm (7.61) and convert it to a
tˆ )
continuous-time version. Using the same approximation dμ( d tˆ
= μ(k̄ + 1) − μ(k̄),
we obtain in the slow time-scale, tˆ
 
μ̇(tˆ ) = η ûj (tˆ ) − P0 (11.29)
j =1

Finally let us incorporate the forward time delay that acts on the input powers to get
 

μ̇(tˆ ) = η ûj tˆ − τ̂ f − P0 (11.30)
j =1

As before one can modify (11.28) to eliminate ûi (tˆ − τ̂ ) in the denominator by
design (see also (11.4) for a similar technique). If we keep a record of past power
11.4 Delay Effects in Network Games with Constraints 199

input, ûi (tˆ − τ̂ ), we can then appropriately replace ûi (tˆ ) with ûi (tˆ − τ̂ ) to obtain
#   *
βi 1
ε û˙ i (tˆ ) = ρi − n 0,i + Γ˜i,j ûj (tˆ − τ̂ ) (11.31)
μ(tˆ − τ̂ b ) ai j

where Γ˜i,j is Γi,j modified on the diagonal (scaling by ai ).

Thus the closed-loop time-delay system is represented by (11.31) and (11.30).
Let us proceed by shifting (11.30) and (11.31) about the equilibrium point (u∗ , μ∗ ).
In vector form this gives

ẋ(tˆ ) = η1row z tˆ − τ̂ f (11.32)
# *
x(tˆ − τ̂ b )
ε ż(tˆ ) = ρ −β ∗ − Γ˜a z(tˆ − τ̂ ) (11.33)
μ (x(tˆ − τ̂ b ) + μ∗ )

where z and x denote in vector form the shifted variables, zi = ûi − u∗i and x =
Γ˜
μ − μ∗ . In the foregoing Γ˜a is a matrix with elements ai,ji , β is a column matrix with
elements βi , 1row is a row vector with all elements equal to 1, ρ = diag(ρi ). Note
that Γ˜a is invertible because of the diagonal dominance condition (see Chap. 7).
Now we can see that (11.32) and (11.33) is written in a standard singular pertur-
bation form. Let
 
−x
˜
h(x) = Γa β−1
(11.34)
μ∗ (x + μ∗ )
be the isolated root of (11.33) and use the coordinate shift
 
ẑ(tˆ ) = z tˆ − τ̂ f − h x(tˆ ) (11.35)

into (11.32). This yields


ẋ(tˆ ) = η1row ẑ(tˆ ) + h x(tˆ ) (11.36)

Next, taking the derivative of (11.35) with respect to time, tˆ, yields

˙ tˆ ) = ż tˆ − τ̂ f − dh(x) ẋ(tˆ )
ẑ(
dx
and using (11.33) it follows that
# *
x(tˆ − τ̂ )  dh(x)
˙ tˆ ) = ρ −β
ε ẑ( − Γ˜a z tˆ − τ̂ − τ̂ f − ε ẋ(tˆ )
μ∗ (x(tˆ − τ̂ ) + μ∗ ) dx

On the right-hand side, using (11.34) and then again (11.35) yields

dh(x)
ε ẑ˙ = −ρ Γ˜a ẑ(tˆ − τ̂ ) − ε ẋ(tˆ ) (11.37)
dx
200 11 Robustness and Delay Effects on Network Games

Finally we rewrite (11.36) and (11.37) as


ẋ(tˆ ) = f ẑ(tˆ ) + h x(tˆ ) (11.38)
 
˙ tˆ ) = g ẑ(tˆ − τ̂ ) − ε dh(x) f ẑ(tˆ ) + h x(tˆ )
ε ẑ( (11.39)
dx
where
 
f ẑ(tˆ ) + h x(tˆ ) := η1row ẑ(tˆ ) + h x(tˆ ) (11.40)

g ẑ(tˆ − τ̂ ) := −ρ Γ˜a ẑ(tˆ − τ̂ ) (11.41)

Thus, the coordinate shifted system to be analyzed is given by (11.38) and (11.39),
where x is on the fast and ẑ is on the slow time-scale. Let us introduce the reduced
system and the boundary-layer system (see the appendix). These are simpler to an-
alyze and can be used to study the full system (11.38) and (11.39).
To obtain the reduced system, we set ẑ = 0 in (11.38), that is, we assume that the
fast dynamics converge instantly to steady-state value. This yields

˙ tˆ ) = f h x̂(tˆ )
x̂( (11.42)

where we denoted its solution by x̂(tˆ ), which is distinct from x(tˆ ), solution of
(11.38). Let us use the same notation

ẑ tˆ(t) = (ẑ ◦ tˆ )(t) := z̃(t) (11.43)

where z̃ = ẑ ◦ tˆ, and recalling τ̂ = ετ we get ẑ(tˆ − τ̂ ) = z̃(t − τ ). Towards obtaining

the boundary-layer system, we rewrite (11.39) back on the fast time-scale, t, to
explicitly account for its fast dynamics. Thus, we substitute (11.43), tˆ = εt and
τ̂ = ετ into (11.39) to get

d z̃(t)  dh(x(εt)) 
= g z̃(t − τ ) − ε f z̃(t) + h x(εt) (11.44)
dt dx

Finally, we set ε = 0, that is we assume that tˆ and x(tˆ ) are “frozen” (since they
evolve on the slow time-scale). Denoting the solution by ž(t), we get

d ž(t) 
= g ž(t − τ ) (11.45)
dt
We can write (11.42) and (11.45) in their explicit forms by using (11.40), (11.41)
 
˙x̂(tˆ ) = η1row Γ˜ −1 β −x̂
(11.46)
a
μ∗ (x̂ + μ∗ )
d ž
= −ρ Γ˜a ž(t − τ ) (11.47)
dt
11.4 Delay Effects in Network Games with Constraints 201

Thus we can see that the reduced system (11.46) is a nonlinear, scalar system with
no time delay. On the other hand, the boundary-layer system (11.47) is a linear time-
delay system. Since the two systems are decoupled, they are much simpler to study
than the full system (11.38) and (11.39). The full system (11.38), (11.39) or (11.32),
(11.33), is studied based on these simplified subsystems and a composite Lyapunov
function.

11.4.2 Time-Scale Stability Analysis: Point-to-Point Topologies

In this section, we present conditions for stability of the closed-loop system (11.32),
(11.33) in the presence of time delays, based on the reduced and boundary-layer
systems (11.46) and (11.47). We apply a Lyapunov–Razumikhin stability analysis
(see Sect. A.11.2). This Lyapunov–Razumikhin stability analysis does not rely on
functionals and the stability conditions are simple in form, but relatively conserva-
tive.
The following lemmas are used to prove the main result, Theorem 11.10.

Lemma 11.8 The boundary-layer system (11.47) is exponentially stable if

σ (ρ Γ˜a + Γ˜aT ρ)
τ< 0 (11.48)
2 σ̄ ((ρ Γ˜a )2 ((ρ Γ˜a )2 )T )

where Γ˜a is defined as in (11.33), ρ = diag(ρi ).

Proof Since the boundary-layer system (11.47) is a linear, time-delay system, we

can use Razumikhin theory. From Theorem A.35, given a system of the form

d ž
= A1 ž(t − τ ) (11.49)
dt
we ensure asymptotic stability if we satisfy the LMI (A.40), in fact ensuring expo-
nential stability.
For the boundary-layer system, (11.47), A1 = −ρ Γ˜a . The LMI (A.40) then be-
comes
⎛ 1 ⎞
− τ [Pρ Γ˜a + Γ˜aT ρP ] + (α̂0 + α̂1 )P 0 −Pρ Γ˜a ρ Γ˜a
⎝ 0 −α̂0 P 0 ⎠<0 (11.50)
˜ T ˜
−Γa ρ Γa ρP
T 0 −α̂1 P

We rewrite (11.50) using the Schur complement, we set Q = P −1 , and after some
manipulation one can obtain
1  1 T
− ρ Γ˜a Q + QΓ˜aT ρ + (α̂0 + α̂1 )Q + (ρ Γ˜a )2 Q (ρ Γ˜a )2 < 0 (11.51)
τ α̂1
202 11 Robustness and Delay Effects on Network Games

Notice that the second and third terms of (11.51) are positive definite. In addition,
notice that there always exists a Q to satisfy (11.51) given a sufficiently small τ .
Thus, we know that a solution exists, the only question remaining is the bound on τ
that gives such a result.
We can simplify (11.51) by a priori choosing Q = vI , where v is a constant
multiplier, and I is the identity matrix. Rearranging (11.51), and setting α̂0 = 0
yields
 
1 
˜ ˜
τ α̂1 I + (ρ Γa ) (ρ Γa )
2 2 T
< ρ Γ˜a + Γ˜aT ρ (11.52)
α̂1
Following rather lengthy manipulations leads to (11.48) as a sufficient condition for
(11.52). 

Lemma 11.9 The reduced system (11.46) is exponentially stable if

1
ai > (Γi,j + Γj,i ) (11.53)
2
j =i

The following result gives a stability result based on Lyapunov–Razumikhin the-

orem (see the appendix, Sect. A.11.2).

Theorem 11.10 For the singularly perturbed system (11.32) and (11.33), there ex-
ists ε ∗ > 0 such that, for 0 < ε < ε ∗ , the origin is exponentially stable if (11.48) and
(11.53) are satisfied.

Proof We use the singular perturbation approach reviewed in the appendix,

Sect. A.9, but modified to handle time delays (see Sect. A.11.2). We prove that the
system (11.38) and (11.39) is exponentially stable via a Lyapunov function based
on the reduced and boundary-layer systems, (11.46) and (11.47), respectively.
Consider the reduced system (11.46). If (11.53) holds, it can be shown that the
Lyapunov function V (x̂) = 12 x̂ 2 applied to (11.46) satisfies
∂V  η1row Γ˜ −1 β
f h(x̂) ≤ − ∗ ∗ a x̂22 ≤ −k∗ x̂22 (11.54)
∂ x̂ μ (μ + r1 )

for x̂ ≤ r1 , where r1 > −μ∗ , and k∗ > 0 is a constant. By (11.53), 1row Γ˜a−1 β > 0.
Note that we can make r1 arbitrarily large, but (11.54) is satisfied locally for x̂.
Consider the boundary-layer system (11.47) and select the Lyapunov function
W (ž) = žT ž. By (11.48) in Lemma 11.8, this is exponential stable and it can be
shown that
∂W   ) )2
g ž(t − τ ) = 2žT −ρ Γ˜a ž(t − τ ) ≤ −k∗2 )ž(t))2
∂ ž
) ) (11.55)
) ∂W )
) ) = 2ž2
) ∂ ž )
2

for k∗2 sufficiently small.

11.4 Delay Effects in Network Games with Constraints 203

Moreover, the following relations hold:

   √
f ẑ + h(x) − f h(x)  = η1row ẑ2 ≤ η mẑ2 (11.56)
  ˜ −1
f h(x)  ≤ η1row Γa β x2 (11.57)
μ∗ (μ∗ + r2 )
) )
) ∂h(x) ) ) −1 ) 1
) ) )˜ )
) ∂x ) ≤ Γa β 2 (μ∗ + r )2 ≤ k2 (11.58)
2 2
) ) ) ) x2
)h(x)) ≤ )Γ˜ −1 β ) ≤ k3 x2 (11.59)
2 a 2 μ∗ (μ∗ + r )
2

for x ≥ r2 , where r2 > −μ∗ , and k2 > 0 and k3 > 0 are constants. By (11.53),
1row Γ˜a−1 β > 0. Thus, we have the overall range, r2 ≤ x ≤ r1 . Note that r2 can be
selected arbitrarily close to −μ∗ . Thus, the inequalities (11.56)–(11.59) are valid
locally over x.
We analyze the stability of the full closed-loop system (11.38), (11.39) via a com-
posite Lyapunov function obtained from these two above. This composite Lyapunov
function is defined as χ(x, ẑ) = V (x) + W (ẑ) = 12 x 2 + ẑT ẑ, hence is quadratic.
Thus, we immediately satisfy the first condition in Theorem A.33.
Next we use the composite Lyapunov function χ(x, ẑ) for (11.38) and (11.39),
and exploit the Lyapunov inequalities (11.54), (11.55) and Lipschitz properties
(11.56)–(11.58). The rest of the proof follows as in Theorem 11.4 [69]. We take
the time-derivative of χ(x, ẑ) along the trajectory of the system (11.38) and (11.39),

∂V  ∂W ∂h  1 ∂W 
χ̇ = f ẑ + h(x) − f ẑ + h(x) + g ẑ(tˆ − τ̂ ) (11.60)
∂x ∂ ẑ ∂x ε ∂ ẑ

where the general functions f , g and h, are defined in (11.40), (11.41), and (11.34),
respectively. Next, we use the norm inequalities (11.54), (11.55), and (11.56)–
(11.58), applied to each term on the RHS of (11.60). For the first term of (11.60),
we apply (11.56), (11.57), and (11.54) to get

∂V 
f ẑ + h(x) ≤ L3 x2 ẑ2 − c3 x22 (11.61)
∂x

where L3 and c3 are constants. Using (11.55) the third term of (11.60) can be rewrit-
ten as

1 ∂W  −1
g ẑ(tˆ − τ̂ ) ≤ k∗2 ẑ22 (11.62)
ε ∂ ẑ ε

Finally, we apply (11.55)–(11.58) to the second term of (11.60) to get

∂W ∂h 
f ẑ + h(x) ≤ b4 ẑ22 + L4 x2 ẑ2 (11.63)
∂ ẑ ∂x
204 11 Robustness and Delay Effects on Network Games

where b4 and L4 are constants. By substituting (11.61)–(11.63) back into (11.60),

we get
⎛ ⎞
 T d̃5  
x2 −
⎠ x2
d̃
χ̇ ≤ − ⎝ 1 2
(11.64)
ẑ2 ẑ2
− d̃25 (d̃3 /ε) − d̃4

where d̃1 = c3 , d̃3 = k, d̃4 = b4 , and d̃5 = L3 + L4 . The di matrix on the RHS of
(11.64) is positive definite if

4d̃1 d̃3
ε< = ε∗
4d̃1 d̃4 + d̃52

Thus, (11.64) satisfies the second condition in Theorem A.33. The system (11.38)
and (11.39) is exponentially stable. 

The time-delay bound (11.48) is very simple in form and offers a clear intu-
ition for selecting the design parameters ai and ρi . This time-delay bound (11.48)
is inversely proportional to the control gain, ρi and is proportional to the design pa-
rameters ai . The trade-off for stability is a slower convergence time for the control
algorithms.

11.5 Robustness and Delay Effects Combined

This section extends the results to include uncertainties and multiple time delays,
based on [147] and [148]. We first present a perturbed OSNR model with time de-
lays, where we apply a norm-bounded uncertainty framework to the system gains,
the input powers and the transmitter noise. This model is presented as (11.68). We
then augment the single-link analysis from the previous section to include both ad-
ditive uncertainty and input multiplicative uncertainties.

11.5.1 OSNR Model with Time Delays and Uncertainties

Until now the OSNR models have assumed that the system gains, Γi,j , are constant
(see Lemmas 11.1 and 11.2). In fact due to slow parameter drift over long periods of
time and changes in amplified spontaneous emission (ASE), they are time-varying.
The slow parameter drift occurs due to the aging of network hardware. Uncertainties
f
exist in time delays τi,j and τib for i, j ∈ M due to external environmental effects
on the fibers and system reconfigurations. Fiber lengths may expand or contract due
to environmental effects, or the network may be reconfigured without remodeling
the network and redesigning all of the control algorithms. Finally, uncertainty can
exist in the transmitter noise at the sources due to slow parameter drift. These un-
certainties are captured in the following OSNR model.
11.5 Robustness and Delay Effects Combined 205

Slow parameter drift in optical networks over long periods of time may be mod-
eled as additive uncertainty in the Γ matrix. Additive uncertainty accurately models
the independent changes that occur to each Γi,j element over time. Let the uncer-
tainty set for the perturbed system ΓΔ ∈ ΩΓ be defined as
 
ΩΓ = Γ + ΔΓ | ΔΓi,row 2 ≤ q̂i , ∀i (11.65)

where Γ is a matrix with elements Γi,j , ΔΓ is a matrix with elements ΔΓi,j ,

ΔΓi,row is an (m × m) matrix with the ith row equal to the ith row of ΔΓ and
the remaining terms equal to zero, and q̂i is an independent uncertainty bound.
Due to system reconfigurations, or external environmental effects on the link
lengths, time-delay values in the input signals may change over time. These changes
result in the channel powers varying from their predicted values. The actual channel
powers and the predicted channel powers are proportional to each other. Thus, we
may model the signal uncertainty as an input multiplicative uncertainty. We define
the uncertain input, uΔj ∈ Ωj , where
 
Ωj = uj + δj uj | δj 2 ≤ q̄j , ∀j (11.66)

where q̄j is the independent uncertainty for input channel j . Note q̄j < 1.
The transmitter noise may drift over time as the hardware ages. Assume that
the signal powers change proportionally, modeled via a proportional additive uncer-
tainty and let us define the uncertain transmitter noise, mΔ,0,i ∈ Ωni , where
 
Ωni = n0,i + δni n0,i | δni 2 ≤ qni , ∀i (11.67)

with qni being the independent uncertainty bound on the transmitter noise for chan-
nel i. Note that qni < 1 since the noise power m0,i is positive.
Then the OSNR model that captures multiple time delays and uncertainties
(11.65), (11.66), and (11.67) follows immediately from Lemma 11.1. This perturbed
OSNR model for the ith channel in a multi-link network is given as
uΔi (t − τi,i )
yΔi (t) =  (11.68)
nΔ,0,i + j ∈M ΓΔi,j uΔj (t − τi,j )
i

where ΓΔ ∈ ΩΓ , uΔj ∈ Ωj , and mΔ,0,i ∈ Ωni .

The special case of a single link with only additive system uncertainty, i.e.
q̄j = qni = 0 will also be considered. For this situation, define the perturbed sys-
tem matrix, ΓΔ ∈ ΩS , where
 
ΩS = Γ + ΔΓ | ΔΓ 2 ≤ q̂ (11.69)

where Γ is a matrix with elements Γi,j defined in (6.15), ΔΓ is a matrix with ele-
ments ΔΓi,j , and q̂ is an independent uncertainty bound, distinct from q̂i defined in
(11.65). The uncertainty set (11.69) is similar to the uncertainty set (11.65), except
the uncertainty bounds in (11.65) apply to each channel output separately, while
(11.65) has one independent bound for all output.
206 11 Robustness and Delay Effects on Network Games

11.5.2 Single-Link Analysis

Consider the continuous-time control algorithms developed based on the link algo-
rithm (7.61) and the channel algorithm (7.62). The following analysis applies to a
single link with time delays and uncertainties as modeled in (11.68). Conditions that
ensure the stability of the closed-loop system are presented.
Using a similar procedure to rewrite (7.61) and (7.62) in their continuous-time
forms yields
 
μ̇(tˆ ) = η uin,j − P0 (11.70)
j =1
#   *
d ûi (tˆ ) βi 1 1
ε = ρi − − Γi,i + ai ûi (tˆ − τ̂ ) (11.71)
dt ˆ μin,i ai yin,i (tˆ )

where uin,j is the signal power transmitted to the link algorithm, μin,i is the channel
price transmitted to the source i, and yin,i is the OSNR signal transmitted back to the
source i. The “in” subscript is used to abstract various signals as input to the control
algorithms without the details of the time delays or uncertainties. This allows us to
substitute for uin,j , μin,i and yin,i appropriately later in the analysis. We use this
notation throughout the remainder of this chapter.
Consider (11.68) for the single-link case, i.e.,

ûΔi (tˆ − τ̂ )
yΔi (tˆ ) =  (11.72)
nΔ,0,i + ˆ
j ∈M ΓΔi,j ûΔj (t − τ̂ )
i

where ûΔj ∈ Ωj , mΔ,0,i ∈ Ωni , and ΓΔ ∈ ΩS , where Ωj , Ωni and ΩS are defined
in (11.66), (11.67), and (11.69), respectively. Note that we use ΓΔ ∈ ΩS for the
single-link case rather than ΓΔ ∈ ΩΓ , where ΩΓ is defined in (11.65).
Substitute (11.72) into (11.71), where yin,i = yΔi (tˆ ) and μin,i = μ(tˆ − τ̂ b ) to
obtain the closed-loop uncertain system with time delays,
#  
βi Γi,i
ε û˙ i (tˆ ) = ρi + − 1 ûi (tˆ − τ̂ )
μ(tˆ − τ̂ b ) ai
   *
1 ûi (tˆ − τ̂ )
− nΔ,0,i + ΓΔi,j ûΔj (tˆ − τ̂ ) (11.73)
ai ûΔi (tˆ − τ̂ )
j ∈Mi

In (11.70), let uin,j = (1 + δj )ûj (tˆ − τ̂ f ) to get

 

μ̇(tˆ ) = η (1 + δj )ûj tˆ − τ̂ f − P0 (11.74)
j ∈M

Thus, (11.73) and (11.74) represent the uncertain closed-loop system with time de-
lay for a single link. Next, we use the fact that ûΔj = ûj + δj ûj = (1 + δj )ûj since
11.5 Robustness and Delay Effects Combined 207

ûj is scalar and make the following substitution, which we state as the following
assumption.

Assumption 11.2 In the denominator of the last term of (11.73), we replace the
uncertainty (1 + δi ) in ûΔi with the worst case situation (1 − q̄i ).

This Assumption 11.2 can be justified as follows: as ai increases in (11.73), the

magnitude of the last term approaches zero. A very large ai essentially eliminates
the OSNR feedback, which results in a stable, but suboptimal open-loop system
response. The smaller the ai value, the larger the OSNR feedback, and the less stable
the close-loop system. However, we are still interested in minimizing the ai values,
since the transient response becomes faster. Thus, replacing (1 + δi ) by (1 − q̄i )
in the denominator of the last term in (11.73), where (1 + δi ) is proportional to
ai , produces a worst case stability condition. Note that this substitution replaces a
system with nonlinear uncertainties with a system with linear uncertainties. Thus,
(11.73) becomes
#  *
βi nΔ,0,i
ε û˙ i (tˆ ) = ρi − − Γ¯Δi,j (1 + δj )ûj (tˆ − τ̂ ) (11.75)
μ(tˆ − τ̂ b ) (1 − q̄i )ai
j ∈Mi

where Γ¯Δi,j = Γ¯i,j + ΔΓi,j /(1 − q̄i )ai , such that

 1
, i=j
Γ¯i,j = 1−Γq̄i,ji
(11.76)
(1−q̄i )ai , i = j

Rewrite (11.75) and (11.74) using x = μ − μ∗ and zi = ûi − u∗i , where μ∗ and u∗i
are the equilibrium points of (11.75) and (11.74)
 
ẋ = η1row I + diag(δj ) z tˆ − τ̂ f (11.77)
# *
−x(tˆ − τ̂ b ) 
ε ż = ρ β ∗ − Γ¯Δ I + diag(δj ) z(tˆ − τ̂ ) (11.78)
μ (x(tˆ − τ̂ b ) + μ∗ )

where z is a vector with elements zi , ρ = diag(ρi ), β is a column vector with ele-

ments βi , and 1row is a row vector of 1’s. Equations (11.77) and (11.78) represent the
perturbed, closed-loop system with time delay shifted about the equilibrium points
(μ, u∗ ). Note that mΔ,0,i is no longer present.
Next, we rewrite (11.77) and (11.78), using the coordinate shift,
 
ẑ1 (tˆ ) = z tˆ − τ̂ f − h1 x(tˆ ) (11.79)

where the function h1 (x(tˆ )) is the isolated root of the RHS of (11.78),

 −1 −x(tˆ )
h1 x(tˆ ) = I + diag(δi ) Γ¯Δ−1 β (11.80)
μ∗ (x(tˆ ) + μ∗ )
208 11 Robustness and Delay Effects on Network Games

and I + diag(δi ) is invertible, Γ¯Δ = Γ¯ + diag( (1−1q̄i )ai )ΔΓ , Γ¯ is a matrix with
elements defined in (11.76). For now, we assume that Γ¯Δ is invertible. In fact, we
impose this condition later in Lemma 11.9. Proceeding as in the previous section
we can obtain the reduced and the boundary-layers systems as
−x̂1 (tˆ )
x̂˙1 (tˆ ) = η1row Γ¯Δ−1 β (11.81)
1 (tˆ ) + μ )
μ∗ (x̂ ∗

d ž1 
= −ρ Γ¯Δ I + diag(δi ) ž1 (t − τ ) (11.82)
dt
The reduced system (11.81) is scalar and nonlinear with no time delay. Notice that
the input multiplicative uncertainty terms have canceled out. The boundary-layer
system (11.82) is linear with multiple time delays. The two systems are decoupled
from each other.
The following lemma presents the LMI that ensures exponential stability for the
boundary-layer system (11.82). A detailed proof can be found in [147].

Lemma 11.11 The boundary-layer system (11.82) is exponentially stable if there

exist symmetric, positive definite matrices P and S1 such that
 
H̄1 H̄2
<0 (11.83)
H̄2T H̄3

where λ̄0 > 0, ki > 0, k1 > 0 with k̄i < k1 for all i,

Mn2 −P A20
H̄1 =
−(AT0 )2 P −S1 + λ̄0 (G∗ )T G∗

P E∗ 1
τ (G )
∗ T −P A0 E ∗
H̄2 =
0 −AT0 (G∗ )T λ̄0 (G∗ )T D ∗
⎛ ⎞
−I (D ∗ )T 0
⎜ ∗ ⎟
H̄3 = ⎝ D −I −G∗ E ∗ ⎠
0 −(E ∗ )T (G∗ )T −λ̄0 (I − (D ∗ )T D ∗ )

A0 = −ρ Γ¯ , E2 = −ρ diag( (1−1q̄i )ai ) and

1  7 7 
Mn2 = P A0 + AT0 P + S1 , E ∗ = q̂E2 k1 A0 diag( q̄i k̄i )
τ
7  √   √
q̂ q̄i q̂ √
∗
G = I diag , D ∗ = 0 k1 diag( q̄i )
k1 k̄i 0 0

One can remark that if we set q̂ = 0 and q̄i = 0 (no uncertainty), and we select
S1 = α̂P , where α̂ is a scalar variable, we can recover the LMI used to derive the
time-delay bound (11.48).
11.6 Notes 209

The following lemma ensures the stability of the reduced system (11.81) via the
adjustment of the channel parameter ai .

Lemma 11.12 The uncertain reduced system (11.81) is exponentially stable if

1
ai > (Γi,j + Γj,i ) + q̂, ∀i (11.84)
2
j =i

One can note that the bound on the channel uncertainty term (1 − q̄i ) is not
present. Only the bound on the system matrix uncertainty q̂ is present. If we elimi-
nate the uncertainty, i.e. q̂ = 0, we exactly recover (11.53).
We now state the main stability theorem for the single-link case based on Lem-
mas 11.11 and 11.12.

Theorem 11.13 Consider the uncertain, singularly perturbed system (11.77) and
(11.78), with uncertainty sets defined in (11.65), (11.66), and (11.67). There exists
an ε ∗ > 0 such that for 0 < ε < ε ∗ the origin is asymptotically stable if (11.83) and
(11.84) hold.

The proof of asymptotic stability can be done using a composite Lyapunov func-
tional based on the reduced and boundary-layer system functionals. For details the
reader is referred to [147] and [144].

11.6 Notes
In this chapter we presented stability conditions for the closed-loop system in the
presence of time delays based on frequency domain and on Lyapunov–Razumikhin
analysis. The time-delay bound (11.48) is very simple in form. A Lyapunov–
Krasovskii analysis which utilizes Lyapunov functionals could also be applied. Lya-
punov functionals significantly complicate the system analysis, and they produce
more complicated stability conditions. However, the Lypaunov–Krasovskii analy-
sis produces less conservative stability conditions than the Lyapunov–Razumikhin
analysis. The reader is referred to [150] for such an approach.
Chapter 12
Games for Routing and Path Coloring

Abstract This chapter provides and overview of routing and path coloring prob-
lems in all-optical networks as noncooperative games. We focus on oblivious pay-
ment functions, that is, functions that charge a player according to its own strategy
only. We review results on the relation between such games and online routing and
path coloring. In particular, these results show that the Price of Anarchy of such
games is lower-bounded by, and in several cases precisely equal to, the competitive
ratio of appropriate modifications of the First Fit algorithm.

12.1 Introduction
In this chapter we present other types of game formulations in optical networks. One
of the problems that can be considered in a game-theory framework has to do with
the problem of routing a number of communication requests in WDM all-optical
networks. Specifically, we focus on routing and wavelength assignment as a class of
problems. Recall that communication requests are carried out by assigning a path in
the network (routing) as well as a transmission wavelength (or color). Wavelength
division multiplexing (WDM) allows several requests to be routed through the same
link(s) of the network, and carry them out simultaneously by assigning a different
wavelength to each request [165, 167].
Given an optical network topology and a set of communication requests another
set of questions arise in addition to optimizing channel performance (e.g. OSNR) as
treated until now. In case when the specific routing of the requests is not given, the
routing and path coloring Routing and Path Coloring (RPC) problem asks for both a
routing and a wavelength (color) assignment minimizing the number of colors. See
Fig. 12.1 for a request routed over two different paths. This RPC problem is also
called the RWA assignment problem [125, 126]. On the other hand, if the routing
of the requests is given, the Path Coloring (PC) or wavelength assignment (WA)
problem asks for the minimum number of colors (wavelengths) required such that
requests sharing a common link are assigned different colors. Other optimization
questions can be stated by introducing additional parameters and constraints [165].
A large number of results have been concentrated on the complexity and approx-
imability questions for these optimization problems [39, 51, 99, 166]. A survey of
early results can be found in [52] and references therein.

L. Pavel, Game Theory for Control of Optical Networks, 211

Static & Dynamic Game Theory: Foundations & Applications,
DOI 10.1007/978-0-8176-8322-1_12, © Springer Science+Business Media New York 2012
212 12 Games for Routing and Path Coloring

Fig. 12.1 Network layers diagram

A recent research direction in this context considers game-theoretic criteria

[73, 130, 160, 162]. As illustrated at various points in this monograph, one can
formulate a noncooperative game between independent players, herein as a model
of an optimization problem that arises in the context of routing. These players have
their own objectives and they do not necessarily have to follow a centralized pro-
tocol, or they can manipulate this protocol (e.g. by providing false information) for
their own goals.
In the rest of this chapter we discuss approaches to the PC (WA) and RPC (RWA)
problems in all-optical networks modeled as noncooperative games. In the previous
chapters each channel was a player in a game and the games considered were in
the class of continuous action games with coupled utilities and coupled constraints.
Herein we formulate games where each routing request (pair of source-destination
nodes) is a player, and a strategy of a player consists of a path from the source to the
destination and a wavelength/frequency (color), hence in the class of finite strategy
games (see Chap. 4). Such an approach is presented in [53]. Other approaches are
those in which the ISPs act as brokers or players between the TSP and the end user
[162].
12.2 Game-Theoretic Model 213

Since each communication request is considered a player, an individual cost or

payment function is associated as follows: each player is charged a cost depending
on the (routing and color) choices of all players (including her own choices).
As in a standard game setup, given a set of choices for all players we say that
the game is in equilibrium if no player can decrease her own cost by changing her
choices, hence we talk of a Nash equilibrium.
Although Nash has shown that each noncooperative game has a mixed Nash
equilibrium, the existence of a pure one is an open question for many games, as
mentioned in Chap. 4. Moreover, due to the selfish behavior of the players, such a
pure equilibrium does not necessarily optimize a global objective goal, i.e., it is not
socially optimal. Such a social goal is also known as the social welfare cost that
we mentioned in Chap. 9. For the problems studied in this chapter, given a set of
requests, the social cost can be defined as the number of colors needed for (routing
and) coloring these requests.
The efficiency of Nash equilibria is measured by the Price of Anarchy which is
defined as the ratio of the social cost of the worst Nash equilibrium over the optimal
centralized solution [73], and reflects the loss in the global performance due to lack
of coordination between players.
The chapter is organized as follows: In the next section we describe the for-
mal model and the notation used, following [28, 86]. Then we examine the relation
between the solutions obtained by online and offline algorithms for PC and RPC
and the Nash equilibria for the corresponding noncooperative game noncooperative
games. We review some results for selfish PC and RPC, respectively, based on pay-
ment functions that yield Nash equilibria, as well as upper and lower bounds for the
Price of Anarchy in both cases.

12.2 Game-Theoretic Model

Consider an optical network modeled as an undirected graph G = (Ng , Eg ), where
nodes in Ng represent sites (OXC/OADM), |Ng | = N and undirected edges in Eg
are bidirectional optical fiber links between the sites (see also Fig. 12.2). Recall
from Chap. 6 that communication is realized by using different wavelengths (colors)
along the optical fiber lines (edges).
Assume that a set of communication requests R is given where each request r is
a source-destination pair of nodes of G, i.e., r = (x, y), x, y ∈ Ng .
A point-to-point communication requires to establish a uniquely colored path be-
tween the two nodes, whose color (wavelength) is different from the colors (wave-
lengths) of all other paths sharing one of it sedges (no color collisions). This is a
version of the graph coloring problem [99].
We consider that a set of colors (wavelengths) {λ1 , . . . , λk } is represented as in-
tegers in the interval C = [1, . . . , k], where k is the maximum number of available
colors. If we were to relate this to the number of channels m used in previous chap-
ters we would have m ≤ k. Without loss of generality assume that the number of
colors to be no more than |R|, i.e., k ≤ |R|.
214 12 Games for Routing and Path Coloring

Fig. 12.2 Network diagram

Fig. 12.3 Network graph diagram (1)

A solution to the request is a set of |R| paths of G, denoted by Pi , i = 1, . . . , |R|,

and a set of wavelengths (colors), denoted by λi , i = 1, . . . , |R|, each wavelength
associated with a path (see Figs. 12.2, 12.3 and 12.4). By a path between two nodes
x, y we understand a set of consecutive edges that begin in node x and end in node y.
If Pi ∩ Pj = for i = j then λi = λj . This represents the assumption that if any two
requests are routed through edge-intersecting paths, i.e., they use the same optical
link at some point along their paths, then they must use different wavelengths (col-
ors) (no color collisions).
In the case when the actual routing of requests in R is given in advance (pre-
determined), this means we can simply consider that a set of paths P is given instead
of R and only the wavelengths (colors) have to be decided. Therefore, an instance
of the RPC problem is denoted by (G, R), while an instance of the PC problem is
12.2 Game-Theoretic Model 215

Fig. 12.4 Network graph diagram (2)

denoted by (G, P ), and in this last case the players only have to choose a color for
their paths.
Consider a selfish or noncooperative RPC (PC) game on G where each player i
issues a request ri (a path, respectively), hence we identify a player with a request.
A strategy σi for player i is a pair σi = (Pi , λi ), where Pi is a simple path connect-
ing the endpoints of ri and λi is a color assigned to Pi . In the case of PC game this
strategy is just λi . Let Si denote all possible strategies of player i. Note that for each
player these possible strategies depend on the topology of graph G and the number
of colors allowed.
Under the assumption that the number of colors is restricted to be no more than
|R|, this means that there is a finite number of strategies for each player. Thus a
game with |R| players is defined by a finite set of strategies Si , i = 1, . . . , |R|.
Towards formulating the game, for each player i we define a payoff function
(utility function) Ui , Ui : S1 × · · · × S|R| → N , function that herein maps S1 × · · · ×
S|R| to the integers. Such a payoff (utility) is typically associated with a cost or
payment function Ji , Ji : S1 × · · · × S|R| → N , and here we consider that this done
via the relationship Ui = −Ji , i = 1, . . . , |R|.
The elements of S1 × · · · × S|R| are called action combinations or states [28].
Note that unlike the setup in the previous chapter, e.g., Chap. 7, herein the strategies
are finite as in Chap. 3 and mapping is to the integers (unlike a continuous action
set Ωi for games in Chap. 7).
From now on, for simplicity we consider games where all players have the same
cost function or payment function, J . Also, herein the graph plays an important role
so instead of the set of players we use the graph G in the game notation (see below).
216 12 Games for Routing and Path Coloring

Let S-RPC and S-PC denote the class of Selfish-RPC and class of Selfish-PC
games (pre-determined routing), respectively. Then by G(G, R, J ) we denote a
game in S-RPC with input graph G, set of requests R, and payment function J .
Similarly, by G(G, P , J ) we denote a game in S-PC with input graph G, set of
routed requests P , and payment function J .
For a game G(G, R, J ) (and similarly for a game G(G, P , J )), let a pure-strategy
profile, or simply strategy profile, be defined as σ = {σ1 , . . . , σ|R| }, one strategy for
each player. Let S denote the set of all possible strategy profiles.

Definition 12.1 A (pure) strategy profile σ ∗ ∈ S is a pure-strategy Nash Equilib-

rium (NE) if for each player i we have
 
J σ1∗ , . . . , σi∗ , . . . , σ|R|
∗
≤ J σ1∗ , . . . , σi , . . . , σ|R|
∗

for any strategy σi ∈ Si .

A game may not have pure-strategy Nash equilibria but if we extend the game
to include as mixed-strategies all possible distributions on Si , then a mixed-strategy
Nash equilibrium (NE) is always guaranteed to exist [95] (see Theorem 3.15 in
Chap. 3). In order to look at the efficiency (performance) of an NE, let NE(G) denote
the set of all NEs, NE(G) ⊂ S.

Definition 12.2 The social cost Js (σ ) of a strategy profile σ ∈ S is defined as the

number of colors used for (routing and) coloring, if no color collisions appear; oth-
erwise Js (σ ) = ∞.

Let Js∗ denote the socially optimal cost, that is,

Js∗ = min Js (σ )
σ ∈S
where S is the set of possible strategy profiles. Note that Js∗ coincides with the cost
of an optimal solution of the corresponding RPC (PC) problem.
Now consider an NE point, σ ∗ ∈ NE(G), where NE(G) denotes the set of all NEs.

Definition 12.3 The Price of Anarchy (PoA) of a game G(G, R, J ) (or G(G, P , J ))
is defined as the worst-case number of colors used in a NE (social cost) divided by
Js∗ , that is,
maxσ ∗ ∈NE(G ) Js (σ ∗ )
Price of Anarchy (PoA) =
Js∗
The Price of Stability (PoS) of a game G(G, R, J ) (or G(G, P , J )) is defined as the
best-case number of colors used in a NE (social cost) over Js∗ , that is,
minσ ∗ ∈NE(G ) Js (σ ∗ )
Price of Stability (PoS) =
Js∗

The assumption of being oblivious collision-free is typically used as defined next

with respect to the cost or payment function.
12.3 Cost Functions and PoA 217

Definition 12.4 Consider a (routing and) path coloring game G(G, R, J ) (or
G(G, P , J )). A cost or payment function J is called oblivious collision-free if:
(a) it guarantees that in a Nash Equilibrium (NE) no color collisions occur, and
(b) it charges a player (who does not collide with other players) according to the
player’s own strategy only.

Note that condition (a) can be satisfied if one charges a very large amount to play-
ers that use the same color and share links of the network. For simplicity the condi-
tion that guarantees collision-free Nash Equilibria is omitted from the descriptions
of the payment functions, since functions we consider here are oblivious collision-
free.
Let us observe that for any instance of S-RPC with oblivious collision-free pay-
ment function, if σ ∗ is a NE, then the following holds:

Js σ ∗ ≤ |R|
Similarly for any instance of S-PC with oblivious collision-free payment function,
if σ ∗ is a NE, then the following hold:

Js σ ∗ ≤ |P |
Thus
|R|
PoA ≤
Js∗
and, respectively,
|P |
PoA ≤
Js∗
in the S-PC case.

12.3 Cost Functions and PoA

As in a typical game setup, the question we are interested in is whether Nash equi-
libria exist for such selfish RPC (PC) games, and if so, what is their performance
(efficiency). In this section we present an approach based on [28, 42] in which self-
ish or noncooperative players can configure their communications so as to minimize
the cost paid for the service.
As in the previous section we model an optical network as an undirected graph
G = (Ng , Eg ) where nodes in Ng represent sites, and undirected edges in Eg repre-
sent bidirectional optical fiber links between the sites. During a series of time steps,
at each of which only one player can move to a better configuration, a Nash equilib-
rium is eventually reached. In such a setting, we are interested in suitable payment
functions covering the network costs that actually induce Nash equilibria (NE) with
the best possible efficiency (lowest Price of Anarchy PoA). We focus on the classi-
cal scenario of minimizing the total number of used wavelengths (see social cost Js
defined above), that is, the approach of optimizing the optical spectrum.
218 12 Games for Routing and Path Coloring

Definition 12.5 A game G(G, R, J ) is said to be convergent if, starting from a

generic configuration, and at each stage letting a player to move to a better configu-
ration, a Nash equilibrium (NE) is always reached.

Notice that, conversely, a non-convergent game might admit a Nash equilibrium.

As before recall that for a r = (x, y) request (player) the utility function is
U = −J (x, y), where J is the individual cost function (payment function), which
is assumed the same for all players/requests. In what follows this cost or payment
function J will be associated with a pricing function f that is introduced below. Let
λr or λ(x, y) denote the color (wavelength) used by the request r = (x, y).
The following assumption is made to model the scenario of minimizing Js , i.e.,
the total number of colors used: for each color (wavelength) λr there exists an as-
sociated (positive) cost (pricing), i.e., there exists a non-decreasing pricing function
f : C → R+ associating a (positive) cost to every color λr or. This setup of using the
pricing function f models in fact the increasing cost incurred by a network provider
when implementing a routing that uses up to a given wavelength.
Specifically, we consider four possible (cost) payment functions [28]:
1. J1 (x, y) = f (λ(x, y)) or J1 (r) = f (λr )
This means that for a request r = (x, y) the amount charged (payment func-
tion) is the price of the color that request uses to communicate, price that goes
according to f .
2. J2 (x, y) = maxe∈R(x,y) max(a,b)∈R|e∈R(a,b) f (λ(a, b))
This means that a request r = (x, y) pays for the maximum price of all colors
used by any other request sharing an edge with the path used by (x, y).
3. J3 (x, y) = e∈R(x,y) max(a,b)∈R|e∈R(a,b) f (λ(a, b))
This means that a request r = (x, y) pays for the sum over all the edges be-
longing to the used path of the maximum price of the colors through the edge.
 f (λ(a,b))
4. J4 (x, y) = e∈R(x,y) max(a,b)∈R|e∈R(a,b) |(a,b)∈R|e∈R(a,b)|
This means that a request r = (x, y) pays for the sum over all the edges be-
longing to the used path of the maximum price of the colors through the edge,
divided by the number of request sharing the edge.
These natural cost functions are used as building blocks for the definition of
the players’ cost or payment functions. The combination of this introduced pricing
functions f with the strategies of maximization or of summation leads to a number
of possible cost or payment functions J .
In all these cases, since the pricing function f is non-decreasing, players have an
incentive to choose small colors so as to possibly minimize the overall number of
used colors. Unfortunately, the results of [26–28] show that these cost or payment
functions either are not convergent or yield the worst possible Price of Anarchy
(PoA), i.e., they converge to an NE equilibrium in which each player uses a different
wavelength (color).
In fact only two of these cost (payment) functions guarantee convergence to a
pure NE (see results below). These are the cases where each player pays for its
own color hence J1 , and where it pays for the maximum color used by any other
12.3 Cost Functions and PoA 219

overlapping player J2 , respectively. As for performance, the PoA is as high as |R|

even for rings. More precisely, the following theorems hold (for proofs see [28]).

Theorem 12.6 (See [28]) The game G(G, R, J1 ) converges to a Nash equilibrium
(NE) in at most |R|2 moves.

This result shows that a NE is reached in polynomial time. However, the payment
function J1 is unable in general to guarantee good performance. As shown by the
following theorem, there exist graphs G for which the Price of Anarchy can be very
high.

Theorem 12.7 (See [28]) For any pricing function f , there exist G and R such that
the Price of Anarchy of game G(G, R, J1 ) is PoA = |R|.

While negative, this result can be justified as follows. Given a routing, finding a
minimum assignment of wavelengths to satisfy the paths in the routing is equiva-
lent to finding a minimum coloring on the graph induced by the routing. It has been
shown that the problem of coloring a graph G = (Ng , Eg ) cannot be approximated
within |Ng |1/7 [23], and since an NE achieved in polynomial time is an approxi-
mated solution to this graph coloring, this implies that the Price of Anarchy cannot
be better than |R|1/7 .
For the second type of cost function J2 the following two results hold.

Theorem 12.8 (See [28]) The game G(G, R, J2 ) converges to a Nash equilibrium
(NE) in a finite number of moves.

Theorem 12.9 (See [28]) For any pricing function f , there exist G and R such that
the Price of Anarchy of game G(G, R, J2 ) is PoA = |R|.

For the last two types of cost function, J3 and J4 , it can be shown that the asso-
ciated games do not even converge in general [28].
Thus the next logical step is to restrict the graph topology, by considering chains
(point-to-point topologies), trees, or rings, or alternatively restricting the communi-
cation patterns. Indeed the following improved results can be shown.

Theorem 12.10 (See [42]) There exist cost or payment functions inducing conver-
gent games with a Price of Anarchy (PoA) of 25.72 in chains, 51.44 in rings and
O(log(|R|)) in trees, all converging in |R|2 steps.

Finally, it is possible to improve the above result for rings and chains by forcing
the agents to simulate the behavior of the online algorithm proposed by Slusarek
[140]. In particular, the following theorem holds.

Theorem 12.11 (See [42]) There exist cost or payment functions inducing conver-
gent games with a Price of Anarchy (PoA) of 6 in rings and 3 in chains.
220 12 Games for Routing and Path Coloring

Another aspect that can be considered is such a RPC game formulation is related
to the different information levels of local knowledge that players may have for
computing their payments [28]. In the complete level of information each player
knows all other players’ routing and coloring strategies. In the intermediate level
of information each player only knows which colors are used on any edge of the
network, while in the minimal level of information each player knows which colors
are used only on edges along paths that the player can choose. For the complete level
it can be shown that the PoA is 1 in chains and 2 in rings, under cost or payment
functions specifically constructed according to the corresponding algorithms [28].
The existence of Nash equilibria and the complexity of recognizing and comput-
ing a Nash equilibrium (NE) for selfish RPC under several payment functions are
considered also in [53]. These results indicate that recognizing a Nash equilibrium
can be done in polynomial time, when each player pays for its own color, when it
pays for the maximum color used by any other overlapping player and when it pays
for the most loaded edge that it uses. On the other hand, when the player pays for
all the different colors appearing along its path, recognizing a Nash equilibrium is
NP-complete.

12.4 Solution to Online S-RPC as Nash Equilibria

Consider now an online version of RPC problem whereby requests arrive as an

ordered sequence R = {r1 , r2 , . . . , r|R| }. Such an online instance of RPC is denoted
by (G, R). Similarly, an instance of online PC is denoted by (G, P), where P is
a sequence of paths ordered by arrival time. In this section we review results that
explore the relation of Nash equilibria (NE) for S-PC and S-RPC games and various
oblivious collision-free payment functions, to the solutions obtained by online and
offline algorithms for (G, R).
Upon arrival of a request ri , ri ∈ R an online algorithm decides a path Pi and a
color assignment λi to ri so that no color collisions appear on any edge of paths that
are already colored (that is, corresponding to requests rj with j < i).
We assume that the algorithm has no knowledge of requests that are going to
appear later (that is, requests rj with j > i). The objective is to minimize the number
of colors used.
The simplest online algorithm for PC is the First-Fit algorithm or greedy algo-
rithm [126]. This algorithm colors each request ri with the smallest available color,
provided that no color collisions occur. For online RPC an appropriate version is the
following version of First-Fit: a path and color for request ri is chose in such a way
that no color collisions occur and the color assigned to ri is the minimum possible.
In the context of a setup where a cost or payment function is specified also, the
following useful generalization of First-Fit for RPC can be defined. Consider a cost
function J which specifies a cost for each path and wavelength (color) assignment
(P, λ) to a request ri , taking into account the path and color assignment to previous
requests rj , j < i. Then, First-Fit with criterion J (or FF(J ) for short) assigns to
12.4 Solution to Online S-RPC as Nash Equilibria 221

each request ri the path Pi and color λi that minimize J (ri , P, λ), assuming that
ties are broken arbitrarily. Notice that here we have added the two arguments P, λ
in addition to r in the notation of the payment function to indicate this dependency.
For example, the standard First-Fit for RPC described above can be seen as
FF(J ), where J (ri , P, λ) = λ if P does not overlap with any path of color λ, oth-
erwise J (r, P, λ) = ∞. A similar generalization of First-Fit can be defined for PC
[86]. Note that in the above descriptions, formally for each payment function J ,
the path-color (or just wavelength (color)) assignment for the previous requests rj ,
j < i, should also appear as argument of function J .
The relation between selfish routing and coloring (S-RPC) games and the corre-
sponding online (centralized) RPC problems is shown in the following two results.

Lemma 12.12 (See [86]) Consider a game G(G, R, J ) in S-RPC (S-PC) where J is
an oblivious collision-free payment function. For any ordering R of R, an execution
of FF(J ) algorithm on (G, R) gives a strategy profile which is a Nash Equilibrium
(NE) for G(G, R, J ).

Proof Consider the path-color assignment obtained by an execution of FF(J ) on

(G, R). By definition, a request ri cannot be assigned a path-color combination of
lower cost unilaterally, otherwise FF(J ) would have chosen that path-color combi-
nation for ri (see above description of First-Fit algorithm). This can be also justified
as follows: if such a different assignment were possible then it does not cause color
collisions with respect to the path-color assignment of all other requests. Therefore,
it certainly does not cause any color collision with respect to previous requests rj ,
j < i; hence, upon arrival of ri , FF(J ) would have chosen this lower cost assign-
ment. 

Lemma 12.13 (See [28]) Consider a game G(G, R, J ) in S-RPC (S-PC) where J
is collision-free and non-decreasing on the players’ color (hence also oblivious).
For every strategy profile σ ∗ that is a Nash Equilibrium for G(G, R, J ), there is
an ordering R of R such that there is an execution of FF(J ) algorithm on (G, R)
yielding the same path-color assignment to R as σ ∗ .

This result gives the explicit relationship of an NE to an online solution via FF.
Moreover, any (routing and) coloring solution of RPC (PC) can be converted to a
Nash Equilibrium (NE) for the corresponding game in S-RPC (S-PC, respectively),
with at most the same number of colors, via the so called Nash conversion algorithm
[86]. This leads to the following result.

Theorem 12.14 (See [86])

1. The Price of Anarchy (PoA) for any game G(G, R, J ) (G(G, P , J ), respectively)
in S-RPC (S-PC), where J is oblivious collision-free, is at least as large as the
competitive ratio of FF(J ) for RPC (PC, respectively).
2. The Price of Anarchy (PoA) for any game G(G, R, J ) (G(G, P , J ), respectively)
in S-RPC (S-PC), where J is oblivious collision-free and is a non-decreasing
222 12 Games for Routing and Path Coloring

function of the players’ color, is equal to the competitive ratio of First-Fit for
RPC (PC, respectively).
3. The Price of Stability (PoS) for any game G(G, R, J ) (G(G, P , J ), respectively)
in S-RPC (S-PC), where J is oblivious collision-free and is a non-decreasing
function of the players’ color, is equal to 1.

For the full proof, see [86]. Here we give a short summary.

Proof For 1, by Lemma 12.12, each execution of FF(J ) leads to a path-color (wave-
length) assignment which is a Nash equilibrium (NE) for a game G(G, R, J ). Hence
the social cost Js of that NE is equal to the number of colors used by FF(J ). Di-
viding by Js∗ we get the claim. For 2, let σ ∗ be an NE of the highest social cost.
By Lemma 12.13, it follows that there is an execution of FF(J ) on the correspond-
ing RPC (PC) instance that requires the same number of colors as σ ∗ . Dividing by
Js∗ we find that the competitive ratio of FF(J ) is at least as large as the Price of
Anarchy. Claim 2 follows by combining this with 1. Part 3 can be shown by consid-
ering the optimal coloring and converting it to an NE by using the Nash conversion
algorithm. 

Some specific cost functions can be defined based on which some refinements of
the PoA results be obtained [86].
Specifically consider the cost or payment function J (r, P, λ) = λ · N +
length(P), where N is the number of nodes in the ring. Under this function a player
r always selects the smallest possible color even if it requires to follow the longest
one of its two possible alternative paths.

Theorem 12.15 (See [86]) The cost function J (r, P, λ) = λ · N + length(P) used
|R|
for S-RPC games in rings leads to a Price of Anarchy (PoA) equal to 2J ∗ + 1, where
s
R is the given set of requests.

Even if this function does not achieve a low PoA, its PoA is half of that of any
payment function that charges according to the value of the color only (see results
above from [26, 28]).
Next, consider the cost or payment function J (r, P, λ) = length(P) · |R| + λ,
where R is the given set of requests. Under this function a player r always selects
the shortest one of its two possible alternative paths even if it requires to take a larger
color. For this case the following result holds.

Theorem 12.16 (See [86]) The cost function J (r, P, λ) = length(P) · |R| + λ used
for S-RPC games in rings leads to a Price of Anarchy (PoA) such that FF chain + 1 ≤
P oA ≤ 5.06 log N + 10, where N is the number of nodes in the ring.

As seen above, this second payment function (which favors shortest path)
achieves a PoA which does not depend on the number of requests but only on the
number of nodes of the ring (logarithmically). It is still an open question whether
12.5 Notes 223

this upper bound can be further improved. Also worth mentioning is the fact that all
these functions require only local color information, namely to know which colors
are used along edges that can be used by a player (minimal level of information
according to the classification in [28] and [27]).

12.5 Notes

We presented an overview of a class of games for routing and path coloring prob-
lems in all-optical networks. We focused on games where each routing request (pair
of source-destination nodes) is a player, and a strategy of a player consists of a path
from the source to the destination and a wavelength/frequency (color), hence in the
class of finite strategy games. We reviewed results on the relation between such
games and online routing and path coloring. Other approaches are those in which
the ISPs act as brokers or players between the TSP and the end user [162].
Chapter 13
Summary and Conclusions

The goal of this monograph has been to study optical networks as examples of com-
munication networks and to understand how control algorithms can be designed
for optical networks from a game theoretic perspective. From a game-theory and
control perspective, there are a multitude of problems to be tackled in optical net-
works and the field is still in its infancy. An important problem is how to control
channel transmission performance in optical networks while incorporating physical
layer impairments. Of particular interest have been the OSNR optimization prob-
lems with link capacity constraints and/or channel OSNR targets. Game-theoretic
approaches have been introduced to study such problems.
An optical network can be considered as a dynamic multiuser environment, in
which signal of a channel is regarded as interfering for all others on the same fiber
link. Power control is a key issue in designing an interference-limited multiuser
communication network system. A major portion of this monograph has been fo-
cused on this first type of problems by using game-theoretic approaches. Another
part of the book has discussed game-theoretic approaches for other challenging
problems such as routing and wavelength assignment (path coloring) (RWA) or
(RPC) in optical networks [99, 125, 165].
Throughout the book our focus has been on analysis of dynamic systems arising
from game formulations with non-separable player utilities and with coupled as well
as propagated (modified) constraints, in the class of continuous strategy games. It
turns out that this is the case for a large class of games in optical networks, where
the utility of each player is affected by decisions of other players, and action sets
are no longer orthogonal (Chaps. 7 and 8).
The monograph has been organized as follows. Chapter 2 has provided an
overview of basic concepts in game theory, mostly focused on noncooperative
(Nash) game theory. Chapter 3 focused on matrix games, while Chap. 4 on games
with continuous action sets and cost functions. Chapter 5 presented some relatively
new theoretical results for continuous Nash games with coupled constraints, i.e.,
coupled action sets. Chapter 6 presented an overview of basic background on trans-
mission in optical networks and on general topologies. The concepts of OSNR and
link power capacity constraint in optical networks have been introduced. In Chap. 7

L. Pavel, Game Theory for Control of Optical Networks, 225

Static & Dynamic Game Theory: Foundations & Applications,
DOI 10.1007/978-0-8176-8322-1_13, © Springer Science+Business Media New York 2012
226 13 Summary and Conclusions

a basic formulation of a game framework towards solving the OSNR optimization

problem in optical networks has been presented, restricted to single point-to-point
optical links, as the simplest network topology. A Nash game played among chan-
nels has been employed towards maximizing OSNR firstly without coupled link
capacity constraint. Then for incorporating the coupled power link capacity con-
straint, two approaches have been considered—an indirect and a direct one, based
on the Lagrangian extension and decomposition theoretical results in Chap. 5.
Approaches on how to deal with games in more complicated network topolo-
gies have been studied in Chap. 8. In network configurations, coupled constraints
are propagated along fiber links and constraint functions become complicated from
the end-to-end point of view. The physical system constraint in optical networks
provides an elimination approach to reduce constraints to a simple structure (less-
coupled constraints). Specifically a partitioned game approach was introduced. The
multi-link structure can be partitioned into stages where each stage has a single-
sink structure (with either a single link or multiple links) and channel powers on
each stage are adjustable. A partitioned Nash game is composed of ladder-nested
stage Nash games. Instead of maximization of channel OSNR from Tx to Rx (end
to end), we considered minimization of channel OSNR degradation between stages.
We showed that the partition directly led to the convexity of link capacity constraints
on each stage. For the case when channel powers are adjustable at each optical
switch each link is a stage after partition. The convexity of constraints propagated
along links is automatically satisfied and Lagrangian extension and decomposition
results can be applied directly. Chapter 9 has addressed the efficiency of Nash game
and methods to improve the efficiency by tuning the parameters in cost functions.
Extensive simulations in MATLAB and practical experimental implementations in
an optical network testbed system (ONTS) have been presented in Chap. 10.
Stability properties of game and pricing dynamics have been investigated under
the assumption of time-scale separation using Lyapunov theory and singular pertur-
bation approach, and considering time-delay effects (Chap. 11). As communication
networks grow in geographical size and complexity, the effects of time delay be-
come more pronounced and cannot be ignored. Chapter 11 has addressed the impact
of time delay in optical network control algorithms derived from game-theoretic for-
mulations. In Chap. 12 other game formulations have been addressed, specifically
for routing and the wavelength-assignment (path coloring) RWA or RPC problem,
these ones being in the class of finite strategy games.
The material presented in this monograph generates several possible directions
for future work. Some open questions are related to asynchronism. Algorithms in
Chaps. 7 and 8 are two-timescale algorithms. The channel and link algorithms con-
verge to optimal solutions for the synchronous updating of channels’ power and
links’ price, respectively. An asynchronous setting of algorithms can better resem-
ble the reality of large-scale networks, in the sense that updates at channels and links
in complex network topologies do not occur synchronously.
Another issue is related to dynamics. Optical networks are operated in a dy-
namic environment, where network reconfiguration (channels add/drop) is being
performed while other existing channels are still in service. The inherent dynamics
13 Summary and Conclusions 227

of a fiber link composed of cascaded dynamical optical elements makes the opti-
cal network dynamics an important aspect both at the data layer and the physical
layer. Such a problem has been addressed in [114]. In mesh network topologies,
channel routes can be changed by optical switches and closed loop structures can
be formed, for example, quasi-ring structures in Chap. 8. Strictly speaking, time
delay and asynchronism are aspects of dynamics. The material presented in this
monograph considers the network after reconfiguration. In other words, system pa-
rameters are stationary between any updates. From the dynamical point of view,
study of the scalability of algorithms is an open direction.
Finally I would like to end on the same note mentioned in the introduction: I be-
lieve there is a lot of scope for fundamental research in optical networks from a
control- and game-theoretic perspective. Hopefully this book will provide the moti-
vation for many other developments and much research in this area.
Appendix A
Supplementary Material

A.1 Notations
We present below some notations used throughout the monograph.
Let a = [ai ] and b = [bi ] be n-dimensional vectors. We write a ≥ b if all ai ≥ bi
and a > b if all ai > bi . We use diag(a) to denote a diagonal matrix whose diag-
onal entries are elements of vector a = [ai ]. A superscript T denotes the transpose
operation.
For a twice continuously differentiable function f : Rm → R, its gradient at u,
denoted by ∇f (u), is defined as
⎡ ⎤
∂f (u)
⎢ ∂u1 ⎥
∇f (u) = ⎣ · · · ⎦ , (A.1)
∂f (u)
∂um

while the Hessian of function f , denoted by ∇ 2 f , is given as

⎡ ∂ 2 f (u) ∂ 2 f (u) ∂ 2 f (u)
⎤
2 ∂u ∂u · · · ∂u ∂u
⎢ 2∂u1 1 2 1 m
⎥
⎢ ∂ f (u) ∂ 2 f (u) ∂ 2 f (u) ⎥
⎢ · · · ∂u2 ∂um ⎥
∇ 2 f (u) = ⎢ ∂u22 ⎥
∂u2 ∂u1
⎢ . (A.2)
⎢ .. .. .. .. ⎥
⎣ . . . ⎥ ⎦
∂ 2 f (u) ∂ 2 f (u) ∂ 2 f (u)
∂um ∂u1 ∂um ∂u2 ··· ∂u2m

For a vector-valued function f : Rm → Rm , with components f1 , . . . , fm , which

are twice continuously differentiable, its pseudo-gradient at u, denoted by ∇f(u) is
defined as a column vector of the first-order partial derivatives of fi (u) with respect
to ui , ∂f∂u
i (u)
i
,
⎡ ⎤
∂f1 (u)
⎢ ∂u1 ⎥
∇f(u) := ⎣ · · · ⎦ , (A.3)
∂fm (u)
∂um

L. Pavel, Game Theory for Control of Optical Networks, 229

Static & Dynamic Game Theory: Foundations & Applications,
DOI 10.1007/978-0-8176-8322-1, © Springer Science+Business Media New York 2012
230 A Supplementary Material

with elements from the diagonal of the Jacobian matrix of f. The Jacobian of ∇f(u)
2
with respect to u is denoted by ∇ f(u),
⎡ ∂ 2 f (u) ∂ 2 f (u) ∂ 2 f1 (u)
⎤
1
2 ∂u
1
∂u · · · ∂u ∂u
⎢ 2∂u1 1 2 1 m
⎥
⎢ ∂ f2 (u) ∂ 2 f2 (u) ∂ 2 f2 (u) ⎥
⎢ · · · ∂u2 ∂um ⎥
∇ f(u) := ⎢ ⎥
2 ∂u2 ∂u1 ∂u22
⎢ . (A.4)
⎢ .. .. .. .. ⎥ ⎥
⎣ . . . ⎦
∂ 2 fm (u) ∂ 2 fm (u) ∂ 2 fm (u)
∂um ∂u1 ∂um ∂u2 ··· ∂u2m

For a two-argument continuously differentiable function f : Rm × Rm → R, with

arguments (u; x), its gradient is defined as
 T
∇f (u; x) = ∇uT f (u; x) ∇xT f (u; x) (A.5)

where ∇u f (u; x) and ∇x f (u; x) are the gradients with respect to the first argument
u = [u1 , . . . , um ] and the second argument x = [x1 , . . . , xm ], respectively, as defined
in (A.1).

A.2 Standard Optimization Review

We review basic concepts in optimization, mainly drawn from [24]. Covered topics
include unconstrained and constrained optimization problems, Lagrange multiplier
and the duality approach.
A mathematical optimization problem has the form

min J0 (u)
(A.6)
subject to u ∈ Ω,

where u = [u1 , . . . , um ]T is the variable, Ω ⊆ Rm is the general constraint set. The

problem (A.6) is called an unconstrained optimization problem if ΩRm . The objec-
tive function J0 : Ω → R is the cost function.
A vector satisfying the constraints in (A.6) is called a feasible vector for this
problem. A feasible vector uopt is a local minimum of J0 (u) if there exists an ε > 0
such that
 ) )
J0 uopt ≤ J0 (u), ∀u ∈ Ω with )u − uopt ) < ε
where  ·  denotes the Euclidean norm. A feasible vector uopt is called a global
minimum of J0 (u), or a solution of the problem (A.6) if

J0 uopt ≤ J0 (u), ∀u ∈ Ω

and uopt is called strict if the inequality above is strict for u = uopt .
The following is a standard result (combining Proposition 1.1.1 and Proposi-
tion 1.1.2 in [24]) regarding the solution of (A.6).
A.2 Standard Optimization Review 231

Proposition A.1 (See [24]) Let uopt be a local minimum of J0 : Ω → R. Assume

that J0 is continuously differentiable in an open set S ⊆ Ω containing uopt . Then

∇J0 uopt = 0 (first-order necessary condition) (A.7)

If in addition J0 (u) is twice continuously differentiable within S, then


∇ 2 J0 uopt is positive semidefinite (second-order necessary condition) (A.8)

If in addition S is convex and J0 (u) is a convex function over S, then (A.7) is a

necessary and sufficient condition for uopt ∈ S to be a global minimum of J0 (u)
over S.

The problem (A.6) is called a constrained optimization problem if Ω is a strict

subset of Rm , Ω ⊂ Rm . Throughout this section, it is assumed that Ω is convex.
The following result follows directly from Proposition 2.1.1 in [24].

Proposition A.2 If Ω ⊂ Rm is a convex and compact set and J0 : Ω → R is a

strictly convex function, then the problem (A.6) admits a unique global minimum.

For constrained optimization problems, a specific structure of Ω ⊂ R con-

structed by inequalities is taken into account in this monograph. That is,

min J0 (u)
(A.9)
subject to gr (u) ≤ 0, r = 1, . . . , R,

where constraints g1 (u), . . . , gR (u) are real-valued continuously differentiable

functions defined from Rm to R. In a compact form the general constraint can
be written as
  
Ω u ∈ Rm  gr (u) ≤ 0, r = 1, . . . , R
We denote this optimization problem (A.9) by OPT(Ω, J0 ).
For any feasible vector u, the set of active inequality constraints is denoted by
  
A(u) = r  gr (u) = 0, r = 1, . . . , R (A.10)

If r ∈
/ A(u), the constraint gr (u) is inactive at u. A feasible vector u is said to be
regular if the active inequality constraint gradients ∇gr (u), r ∈ A(u), are linear
independent.
The Lagrangian function L : Rm+R → R for problem (A.9) is defined as


R
L(u, μ) = J0 (u) + μr gr (u), (A.11)
r=1

where μr , r = 1, . . . , R, are scalars. The following proposition (Proposition 3.3.1

in [24]) states necessary conditions for solving OPT(Ω, J0 ) in terms of the La-
grangian function defined in (A.11).
232 A Supplementary Material

Proposition A.3 (Karush–Kuhn–Tucker (KKT) Necessary Condition) Let uopt be

a local minimum of the problem (A.9). Assume that uopt is regular. Then there exists
a unique vector μ∗ = (μ∗1 , . . . , μ∗R ), called a Lagrange multiplier vector, such that

∇u L uopt , μ∗ = 0 (A.12)
μ∗r ≥ 0, ∀r = 1, . . . , R (A.13)

μ∗r = 0, ∀r ∈
/ A uopt (A.14)

Condition (A.14) in Proposition A.3 is called the complementary slackness con-

dition.
The next result (Proposition 3.3.4 in [24]) reviews general sufficient conditions,
in terms of the Lagrangian function (A.11), for the following constrained optimiza-
tion problem:

min J0 (u)
subject to gr (u) ≤ 0, r = 1, . . . , R, (A.15)
u ∈ U,

where U ⊆ Rm . The conditions are general since differentiability and convexity of

J0 and gr , r = 1, . . . , R, are not required. Meanwhile, U may be a strict subset of
Rm , U ⊂ Rm .

Proposition A.4 (General Sufficiency Condition) Consider the problem (A.15). Let
uopt be a feasible vector, together with a vector μ∗ = [μ∗1 , . . . , μ∗R ]T that satisfies

μ∗r ≥ 0, ∀r = 1, . . . , R

μ∗r = 0, ∀r ∈
/ A uopt

and assume uopt minimizes the Lagrangian function L(u, μ∗ ) (A.11) over u ∈ U ,
denoted as

uopt ∈ arg min L u, μ∗ (A.16)
u∈U

Then uopt is a global minimum of the problem (A.15).

Remark A.5 If in addition, J0 and gr , r = 1, . . . , R, are also convex and U = Rm ,

then the Lagrangian function L(u, μ) is convex with respect to u. Therefore, by
Proposition A.1, (A.16) is equivalent to the first-order necessary condition (A.12) in
Proposition A.3. Thus conditions (A.12)–(A.14) are also sufficient.
A.3 Diagonally Dominant and M-matrices 233

A.3 Diagonally Dominant and M-matrices

Definition A.6 (See [94]) Let A := [aij ] be an m × m matrix. The matrix A is said
to be diagonally dominant if


m
|aii | ≥ |aij |, ∀i = 1, . . . , m
j =1,j =i

It is said to be strictly diagonally dominant if


m
|aii | > |aij |, ∀i = 1, . . . , m
j =1,j =i

Some useful results are shown in the following theorem, adapted from [59].

Theorem A.7 (See [59], p. 349) Let the m × m matrix A = [aij ] be strictly diago-
nally dominant. Then A is invertible and
(a) If all main diagonal entries of A are positive, then all the eigenvalues of A have
positive real part.
(b) If A is Hermitian and all main diagonal entries of A are positive, then all the
eigenvalues of A are real and positive.

Lemma A.8 Let A be an m × m real matrix with all main diagonal entries positive.
Then A is positive definite if A and AT are both strictly diagonally dominant.

Proof If A and AT are both strictly diagonally dominant, then it follows that


m 
m
aii > |aij | and aii > |aj i |
j =1,j =i j =1,j =i

Thus for matrix As := A + AT ,


m
 
m
2aii > |aij | + |aj i | ≥ |aij + aj i |
j =1,j =i j =1,j =i

Thus As is also strictly diagonally dominant. From Theorem A.7, it follows that As
is positive definite. Therefore A is positive definite in the sense that the symmetric
part, 12 As , is positive definite. 

Definition A.9 A square matrix is called a Z-matrix if all off-diagonal entries are
less than or equal to zero.

Definition A.10 (See [43], Theorem 5.1) Let the m × m matrix A = [aij ] be a Z-
matrix. A is called an M-matrix if it satisfies any one of the following conditions.
234 A Supplementary Material

1. There exists an m × 1 vector v with non-negative entries such that Av > 0.

2. Every real eigenvalue of A is positive.
3. There exist a non-negative m × m matrix B and a number λ > ρ(B), where ρ(B)
is the spectral radius of B, such that A = λI − B.
4. The real part of any eigenvalue of A is positive.
5. A is non-singular and the inverse of A is non-negative.
6. Av ≥ 0 implies v ≥ 0, where v is an m × 1 vector.

Theorem A.11 Let the m × m matrix A = [aij ] be an M-matrix. Then

1. there exist a non-negative m × m matrix B and a number λ > ρ(B), where ρ(B)
is the spectral radius of B, such that A = λI − B; 
2. specifically when λ = 1, A = I − B, ρ(B) < 1, and (I − B)−1 = ∞ k
k=0 B exists
and is positive component-wise.

Proof The proof for the first part can be found in [43] where Theorem
 5.1kis proved.
In the case when λ = 1, ρ(B) < 1 leads to the convergence of ∞ k=0 B . Further-
more,
∞
 
B k (I − B) = I + B + B 2 + · · · (I − B) = I
k=0
∞
Thus it follows that (I − B)−1 = k=0 B
k. 

A.4 Maximum Theorem

Definition A.12 Let S be a (nonempty) subset of Rn . The set of all nonempty sub-
sets of S is called the power set of S, denoted by P(S).

Definition A.13 Let Θ and S be subsets of Rl and Rn , respectively. A correspon-

dence Φ from Θ to S is a map that associates with each element θ ∈ Θ a (nonempty)
subset Φ(θ ) ⊂ S.

Definition A.14 Let Φ be a correspondence from Θ to S by Φ : Θ → P(S). Φ is

said to be upper-semi-continuous at a point θ ∈ Θ if for all open sets V such that
Φ(θ ) ⊂ V, there exists an open set U containing θ , such that θ ∈ U ∩ Θ implies
Φ(θ ) ⊂ V. Φ is upper-semi-continuous if it is at any point of Θ.

Example A.15 The correspondence Φ is defined by

#
{0}, if x = 0
Φ(x) :=
[−1, +1], if x = 0

and is upper-semi-continuous at 0 (Fig. A.1).

A.5 Fixed-Point Theorems 235

Fig. A.1 Example:

upper-semi-continuous

Remark A.16 Every upper-semi-continuous single valued correspondence is a con-

tinuous function.

Theorem A.17 (Berge’s Maximum Theorem) Let f : X × Y → R be a continuous

function and Φ : X → Y be a nonempty, compact-valued, continuous correspon-
dence. Then
1. f ∗ : X → R with
 
f ∗ (x) := max f (x, y) : y ∈ Φ(x)

is a continuous function;
2. Φ ∗ : X → Y with
   
Φ ∗ := arg max f (x, y) : y ∈ Φ(x) = y ∈ Φ(x) : f (x, y) = f ∗ (x)

is a compact-valued, upper-semi-continuous correspondence.

A.5 Fixed-Point Theorems

We introduce the two fundamental fixed-point theorems used in existence proofs.

Theorem A.18 (See [20], Brouwer’s Fixed-point Theorem) If S is a compact and

convex subset of Rn and f is a continuous function mapping S into itself, then there
exists at least one x ∈ S such that f (x) = x.

Theorem A.19 (See [20], Kakutani Fixed-point Theorem) Let S be a compact and
convex subset of Rn , and let f be an upper-semi-continuous correspondence which
assigns to each x ∈ S a closed and convex subset of S. Then there exists some x ∈ S
such that x ∈ f (x).
236 A Supplementary Material

A.6 Projection Theorem

Projection method is widely used in developing iterative algorithms solving con-

strained optimization problems. Each time when an update jumps outside the feasi-
ble set X , the algorithm can project it back to the set X .
The projection of a vector x onto a nonempty, closed, and convex set X is defined
with respect to the Euclidean norm, denoted by [·]+ , i.e.,

[x]+ := arg minz − x

z∈X

The properties of the projection are presented in the following theorem.

Theorem A.20 (See [25], Projection Theorem) Let X be a nonempty, closed, and
convex subset of Rn .
1. For every x ∈ Rn , there exists a unique z ∈ X that minimizes z − x over all
z ∈ X and is denoted by [x]+ .
2. Given some x ∈ Rn , a vector x ∗ ∈ X is equal to [x]+ if and only if
T 
z − x ∗ x − x ∗ ≤ 0, ∀z ∈ X

3. The mapping f : Rn → X defined by f (x) = [x]+ is continuous and nonexpan-

sive, that is,
) + )
)[x] − [y]+ ) ≤ x − y, ∀x, y ∈ Rn

A.7 Lipschitz Continuity

Lipschitz continuity is a smoothness condition for functions which is stronger than

regular continuity. For a function f (x) : D → Rn , where D is a compact subset of
Rm , the Lipschitz condition is defined as
) )
)f (x) − f (y)) ≤ Lx − y, (A.17)

where  ·  denotes Euclidean norm. The positive constant L is called a Lipschitz

constant.

Definition A.21 A function f (x) is said to be Lipschitz continuous if there exists a

constant L ≥ 0 such that for all x, y ∈ D, the Lipschitz condition (A.17) is satisfied.
The function f (x) is called locally Lipschitz continuous if each point of D has
a neighborhood D0 such that f (x) satisfies the Lipschitz condition for all points in
D0 with some Lipschitz constant L0 .
A locally Lipschitz continuous function f (x) on D is Lipschitz continuous on
every compact (closed and bounded) set W ⊆ D.
A.8 Variational Inequalities 237

The Lipschitz property is weaker than continuous differentiability, as stated in

the next proposition.
∂f
Proposition A.22 (Lemma 3.2, [69]) If a function f (x) and ∂x (x) are continuous
on D, then f (x) is locally Lipschitz continuous on D.

A.8 Variational Inequalities

Variational inequalities can be used to study equilibrium problems. Given a subset
X ⊂ Rn and a function f : Rn → Rn , the variational inequality problem associated
with X is to find a vector x ∗ ∈ X such that
T 
x − x ∗ f x ∗ ≥ 0, ∀x ∈ X

A short notation of this problem is by VI(X, f ).

A necessary and sufficient condition for a vector x ∗ to be a solution of VI(X, f )
is given below.

Proposition A.23 (Proposition 5.1, [25], p. 267) Let γ be a positive scalar and let
G be a symmetric positive definite matrix. A vector x ∗ is a solution of VI(X, f ) if
and only if
 ∗ +
x − γ G−1 f x ∗ G = x ∗ ,
where [·]+
G is the projection on X with respect to norm xG = (x Gx)
T 1/2 .

The existence of a solution is ensured by the following proposition.

Proposition A.24 (Proposition 5.2, [25], p. 268) Suppose that X is compact and
that f : Rn → Rn is continuous. Then there exists a solution to VI(X, f ).

A.9 Singular Perturbation Theory

Singular perturbation theory [69] is used to study the stability of systems that have
dynamics on two different time-scales. Consider a general nonlinear system without
time-delays of the form

ẋ(tˆ) = f x(tˆ), z(tˆ)
 (A.18)
ε ż(tˆ) = g x(tˆ), z(tˆ)

where tˆ is the independent time variable, x ∈ n , z ∈ p , ε > 0 is a real scalar, and

f and g are locally Lipschitz around the origin with f (0, 0) = 0 and g(0, 0) = 0.
The ε term is considered to be “small”, i.e. take the limit as ε → 0. We define by
238 A Supplementary Material

ˆ
ν̇ = dν(
d tˆ
t)
. The system (A.18) is defined on two different time-scales, one fast, and
one slow.
We describe (A.18) in further detail as follows. We denote by tˆ the “slow” time-
scale, and we denote by t the “fast” time-scale. The two time-scales are related to
each other by tˆ = εt. A nonlinear system that is in singular perturbation form, as
presented in (A.18), may be time-decoupled into two subsystems that are easier to
analyze than the full nonlinear system. One subsystem is defined on the slow time-
scale, and the other subsystem is defined on the fast time-scale. The system on the
slow time-scale is called the reduced system, while the system on the fast time-scale
is called the boundary-layer system. A Lyapunov stability analysis may be applied
to the reduced and boundary-layer systems separately. The Lyapunov functions of
the reduced and boundary-layer systems may be combined into a composite Lya-
punov function that applies to the full, singularly perturbed system. This composite
Lyapunov function can be used to prove the stability of the full system.
Next we show how to obtain the reduced system and the boundary-layer system
of (A.18) as in [69]. Assume there is at least one isolated root of 0 = g(x, z) and
let z = h(x) be one such root that vanishes at x = 0. For convenience, we define the
coordinate shift ẑ = z − h(x). Thus, the coordinate shifted system becomes

ẋ = f x, ẑ + h(x) (A.19)
 ∂h 
ε ẑ˙ = g x, ẑ + h(x) − ε f x, ẑ + h(x) (A.20)
∂x

where x and ẑ are functions of the slow time variable tˆ. Note that the ε term appears
in the second equation because it is written with respect to the slow time variable, tˆ.
If we rewrite this equation as a function of the fast time variable t, the ε term disap-
pears.
The reduced model is derived from (A.19) by setting ẑ = 0, which is equivalent
˙ converge instantly. This produces the following
to assuming the fast dynamics, ε ẑ,
reduced system:

ẋ = f x, h(x)
Notice that this system is on the slow time-scale, tˆ.
The boundary-layer system is derived from (A.20). We rewrite (A.20) with re-
spect to the fast time-scale, t, as follows. Let t = εtˆ . Note that setting ε = 0 freezes
the tˆ and x(tˆ) variables with respect to the fast time-scale t so that tˆ and x appear as
constants. Alternatively, we can think of the variables on the t time-scale converg-
ing instantaneously, since t = εtˆ . We introduce the notation tˆ(t) := εt for the scaling
by ε. Thus, we define z̃(t) = ẑ(tˆ(t)) = (ẑ ◦ tˆ)(t), where ◦ denotes composition and
y = ẑ ◦ tˆ. Thus, when we set ε = 0 in (A.20), and rewrite it with respect to the t
time-scale, we obtain

d z̃(t) 
= g x(tˆ), z̃(t) + h x(tˆ)
dt
A.9 Singular Perturbation Theory 239

where d z̃
dt = ε ddẑtˆ . Thus, we define the reduced and boundary-layer systems as fol-
lows.

Definition A.25 Consider the system (A.18). Let the reduced system be defined as

ẋ = f x(tˆ), h x(tˆ)

and the boundary system be defined as

d z̃(t) 
= g x(tˆ), z̃(t) + h x(tˆ)
dt
where h(x) is the isolated root of the RHS of (A.18) that vanishes at x = 0, t and
tˆ are the fast and slow time-scales in the system, respectively, z̃ = ẑ ◦ tˆ and ẑ =
z − h(x) is the coordinate shift.

We define exponential stability as follows [69].

Definition A.26 Consider the general autonomous system

v̇ = f (t, v) (A.21)

where f : [0, ∞) × D → n is piecewise continuous in time t and locally Lips-

chitz in v on [0, ∞) × D and D ⊂ n is a domain that contains the origin v = 0.
The equilibrium point v = 0 of (A.21) is exponentially stable if there exist positive
constants c1 , c2 , and c3 such that

v(t) ≤ c2 v(t0 )e−c3 (t−t0 ) , ∀v(t0 ) < c1 , ∀t ≥ t0 (A.22)

and globally exponentially stable if (A.22) is satisfied for any initial state v(t0 ).

We now introduce the formal singular perturbation theorem for analyzing the
stability of (A.18). The following result is Theorem 11.4 simplified from [69] and
restated as Theorem A.27. Assume that (A.18) satisfies all necessary assumptions
outlined below.

Theorem A.27 Consider the singularly perturbed system


ẋ = f x(tˆ), z(tˆ) (A.23)

ε ż = g x(tˆ), z(tˆ) (A.24)

where x ∈ n , z ∈ p , tˆ ∈  and ε is a real scalar.

Assume the following are satisfied for all

(tˆ, x, ε) ∈ [0, ∞) × Br × [0, ε0 ]

where Br = {x ∈ n | x ≤ r}:

240 A Supplementary Material

– f (0, 0) = 0 and g(0, 0) = 0.

– The equation 0 = g(x, z) has an isolated root z = h(x) such that h(0) = 0.
– The functions f ,g,h and their partial derivatives up to the second order are
bounded for z − h(x) ∈ Bρ .
– The origin of the reduced system ẋ = f (x(tˆ), h(x(tˆ))) is exponentially stable.
– The origin of the boundary-layer system d z̃(t) ˆ ˆ
dt = g(x(t ), z̃(t) + h(x(t ))) is expo-
nentially stable, uniformly in (tˆ, x), i.e., there exist constants c1 , c2 , and c3 such
that z̃(t) ≤ c1 z̃(0)e−c2 t , ∀z̃(0) < c3 , ∀(tˆ, x), ∀t ≥ 0.
Then there exists ε ∗ > 0 such that for all ε < ε ∗ , the origin of (A.23)–(A.24) is
exponentially stable.

The high level proof of Theorem A.27 proceeds as follows. First, a Lyapunov
function for each of the reduced and boundary-layer systems is found. Next, the
two functions are summed together to produce a composite Lyapunov function that
is applied to the full system, (A.23)–(A.24). Using the properties of the Lyapunov
functions, as well as bounds on f , g, and h around the origin, exponential stability
is proved. The details of the proof of Theorem A.27 are found in [69].

A.10 Time-Delay Stability Results

A.11 Time-Delay Stability Theory

In this section, we review results on stability of time-delay systems. We first de-
fine the continuous norm, the retarded functional differential equation, and the def-
initions of stability from [55]. We then present the Lyapunov–Razumikhin theo-
rems [55]. These theorems do not rely on functionals, but on typical Lyapunov
functions, and as such they produce simpler, more conservative stability criteria.
Lyapunov–Krasovskii theory relies on functional equations which give less conser-
vative stability criteria at the expense of both analytical and computational complex-
ity and it is not covered here.

A.11.1 Definitions

We define C([−τ, 0], n ) as the set of all continuous functions mapping [−τ, 0] to
n . Let C = C([−τ, 0], n ). The general form of a retarded functional differential
equation is defined as
dx
= f (t, xt ) (A.25)
dt
where xt ∈ C is the function mapping [t − τ, t] to n , and f :  × C → n .
The following norm is used in the definition of stability for time-delay systems.
A.11 Time-Delay Stability Theory 241

Definition A.28 The continuous norm is defined as

) )
xt c = max )x(t + θ ))2
−τ ≤θ≤0

where xt ∈ C is the function mapping [t − τ, t] to n .

Note that x(t)2 ≤ xt c . We now define stability for (A.25).

Definition A.29 For the system (A.25), the origin is:

– stable if for any t0 ∈  and any ε̂ > 0, there exists a δ̂ = δ̂(t0 , ε̂) > 0 such that
xt0 c < δ̂ implies x(t) < ε̂ for t ≥ t0 .
– asymptotically stable if it is stable and for any t0 ∈  and any ε̂ > 0, there exists
a δ̂a = δ̂a (t0 , ε̂) > 0 such that xt0 c < δ̂a implies limt→0 x(t) = 0.
– uniformly stable if it is stable and δ̂(t0 , ε̂) can be chosen independently of t0 .
– uniformly asymptotically stable if it is uniformly stable and there exists a δ̂a > 0
such that for any ηa > 0, there exists a T = T (δ̂a , ηa ) such that xt0 c < δ̂ implies
x(t) < ηa for t ≥ t0 + T and t0 ∈ .

We define Ch as the space of continuous functions, φ, mapping [−τ, 0] to n

such that φc < h, where h is a positive real number. Thus, Ch is the open ball of
C of radius h. We use the following definition of exponential stability.

Definition A.30 The origin of (A.25) is exponentially stable if there exist positive
real numbers h, a, and b such that for every initial condition φ ∈ Ch the solution
xt (φ) of (A.25) exists for all t ≥ 0 and furthermore satisfies
) )
)xt (φ)) ≤ ae−bt φc
c

The above results apply to general retarded functional differential equations of

the form (A.25). We next specialize stability criteria for linear time-delay systems.
Consider the system of the form

ẋ(t) = a0 x(t) + a1 x(t − τ ) (A.26)

where x(t), τ, a0 , a1 ∈ . The initial condition for (A.26) is defined as the function
φ : [−τ, 0] →  such that

x(t) = φ(t) ∀t ∈ [−τ, 0] (A.27)

The Laplace transform of (A.26) with initial conditions φ exists, and may be written
as follows
#  0 *
1
X(s) = φ(0) + a1 e−s(v+τ ) φ(v) dv (A.28)
Δ(s) −τ
where
Δ(s) = s − a0 − a1 e−τ s (A.29)
242 A Supplementary Material

is the characteristic quasipolynomial of (A.26). Let Φ(t) denote the inverse Laplace
transform of 1/Δ(s). We call Φ(t) the fundamental solution of (A.26). Notice that
with the initial condition φ(0) = 1, and φ(v) = 0 for all v ∈ [−τ, 0), then Φ(t) is
the solution of (A.26). The growth of the general solution of (A.26) is related to the
exponential growth of the fundamental solution Φ(t), which is determined by the
poles of the system (A.28), i.e., the solutions of the characteristic equation

Δ(s) = 0 (A.30)

In general, (A.30) has an infinite number of solutions. Denote by Re[s] the real com-
ponent of the complex value s. The following proposition from [55] gives conditions
for stability of (A.26).

Proposition A.31 For any α ∈ , there are a finite number of poles with real parts
greater than α. Let si for i = 1, 2, . . . be the poles of the system (A.26), i.e., the
solutions of (A.30), and let
α0 = max Re[si ] (A.31)
i
Then for any α > α0 , there exists a L > 0 such that the solution of (A.26) with the
initial condition (A.27) satisfies the inequality
 
x(t) ≤ Leαt φc (A.32)

By Proposition A.31, for the solution of (A.26) to approach zero as t → ∞ for

any initial condition, it is sufficient that the poles of the system have negative real
parts. This stability result is similar to the stability result for systems without time-
delay. In fact, this condition is both necessary and sufficient [55].
The above discussion for (A.26) may be generalized to linear systems of the
following form:


K
ẋ(t) = A0 x(t) + Ak x(t − τk ), τk ≥ 0 (A.33)
k=1

where A0 , Ak ∈ n×n , and τk ∈  for k = 1, . . . , K. The stability of (A.33) is deter-

mined by the characteristic quasipolynomial
 
p s; e−τ1 s , . . . , e−τK s = det Δ(s) (A.34)

where

K
Δ(s) = sI − A0 − Ak e−τk s
k=1

We say (A.33) is asymptotically stable if and only if p(s; e−τ1 s , . . . , e−τK s ) has
no root in the closed RHP, or C̄+ . This definition of stability is stated formally as
follows [55].
A.11 Time-Delay Stability Theory 243

Definition A.32 The system (A.33) is said to be asymptotically stable if and only
if its characteristic quasipolynomial (A.34) satisfies

p s; e−τ1 s , . . . , e−τK s = 0, ∀s ∈ C̄+ (A.35)

The following section presents the Lyapunov–Razumikhin stability theorems

which provide alternative stability criteria for linear time-delay systems.

A.11.2 Razumikhin Stability Theory

We review the Lyapunov–Razumikhin theorem and a set of stability theorems for

linear systems with time-delays [55]. This theorem does not depend on Lyapunov
functionals, but on the traditional Lyapunov functions used in systems without de-
lays.
Consider the linear time-invariant system of the form

dx(t)
= A0 x(t) + A1 x(t − τ ) (A.36)
dt
where A0 and A1 are (n × n) real matrices.
The following theorem, known as the restricted Lyapunov–Razumikhin theorem
[55] (Proposition 5.1, p. 149) gives conditions for asymptotic stability.

Theorem A.33 The time-delay system (A.36) with maximum time-delay τ is asymp-
totically stable if there exists a quadratic Lyapunov function V : n →  such that
for some ψ1 > 0, ψ2 > 0 it satisfies

ψ1 x22 ≤ V (x) ≤ ψ2 x22

dV (x)
where the time derivative along the system trajectory dt satisfies

dV (x) ) )2
≤ −ψ1 )x(t))2 (A.37)
dt
if ∀p > 1, ∃t, ∀ξ ∈ [−τ, 0] such that
 
V x(t + ξ ) ≤ pV x(t)

Notice that Theorem A.33 closely resembles the Lyapunov stability theo-
rem [69]. The first condition is the same as in the Lyapunov stability theorem.
The second condition requires that the time derivative of the Lyapunov function
be non-positive which is also similar to the Lyapunov stability theorem. The third
requirement is an extra condition that only appears for time-delayed systems. We
244 A Supplementary Material

explain this extra condition as follows. The negation of the third condition of Theo-
rem A.33 is
 
∃p > 1, ∀t, ∃ξ ∈ [−τ, 0] V x(t + ξ ) > pV x(t) (A.38)

Thus, for some p > 1, at every instant of time, t, the value V (x(t)) is less than a
past instant. Intuitively, this means the state trajectory is approaching its equilibrium
point, and hence, the system is stable. Thus, we only need to ensure the first two
conditions of Theorem A.33 are satisfied if the third condition holds. If the third
condition does not hold, then the system is implicitly stable. An interesting note is
that the Lyapunov function and its time derivative are compared to the square of the
norm of the state, which implies exponential stability.
Theorem A.33 presents a general set of stability conditions that do not exploit the
structure of (A.36), i.e., A0 and A1 do not appear in the stability criteria. In addition,
it is not obvious how to pick a Lyapunov function to satisfy the stability criteria in
Theorem A.33. We show that in the foregoing work, the Lyapunov function V =
x T P x may be used.
The remainder of this section is dedicated to delay-independent and delay-
dependent stability criteria specific to the structure of the linear system (A.36). We
first outline time-delay-independent stability conditions, which provide the simplest
and most conservative stability conditions (Proposition 5.3 in [55]). These are pro-
vided below as Proposition A.34.

Proposition A.34 The system (A.36) is asymptotically stable if there exist a scalar
α̂ > 0 and a real symmetric matrix P such that

P A0 + AT0 P + α̂P P A1
<0 (A.39)
AT1 P −α̂P

If A0 = 0 then Proposition A.34 cannot be directly applied since this does not
produces a feasible solution. Thus, more general time-delay-dependent conditions
need to be applied. Time-delay-dependent stability conditions produce less con-
servative stability criteria. The following theorem presents a set of time-delay-
dependent conditions to ensure asymptotic stability of (A.36) (Corollary 5.8 in [55]).

Theorem A.35 The system (A.36) is asymptotically stable if there exist a real sym-
metric matrix P and real scalars α̂0 > 0, α̂1 > 0 such that
⎛ ⎞
M −P A1 A0 −P A21
⎜ ⎟
⎜ −AT AT P −α̂0 P 0 ⎟
⎝ 0 1 ⎠<0 (A.40)
−(A21 )T P 0 −α̂1 P

where
1 
M= P (A0 + A1 ) + (A0 + A1 )T P + (α̂0 + α̂1 )P (A.41)
τ
A.11 Time-Delay Stability Theory 245

An alternative approach based on Lyapunov–Krasovskii stability theory is less

conservative than the Lyapunov–Razumikhin approach reviewed above. The trade-
off in using Lyapunov–Krasovskii theory is in its analytical and computational com-
plexity. This theory depends on Lyapunov functionals, which are structurally more
complicated than traditional Lyapunov functions.
Appendix B
List of Notations

B.1 Notations and Symbols

Notations for signals and noise
Location Notation

Transmitter ui —signal power of channel i

u = [u1 · · · um ]T —vector signal power
n0i —noise power of channel i
Receiver n0 = [n01 · · · n0m ]T —vector noise power
pi —signal power of channel i
p = [p1 · · · pm ]T —vector signal power
ni —noise power of channel i
Link l n = [n1 · · · nm ]T —vector noise power
y = [y1 · · · ym ]T —vector OSNR
ui,l —signal power of channel i at the input
nin
i,l —noise power of channel i at the input
pi,l —signal power of channel i at the output
nout
i,l —noise power of channel i at the output
ul = [u1,l · · · um,l ]T —vector signal power at input of link l
pl = [p1,l · · · pm,l ]T —vector signal power at output of link l
ū—vector signal power at input of all links
ū = [uT1 · · · uTL ]T
yl = [y1,l · · · ym,l ]T —vector OSNR
Span s on Link l pi,l,s —signal power of channel i at the output
ni,l,s —noise power of channel i at the output

L. Pavel, Game Theory for Control of Optical Networks, 247

Static & Dynamic Game Theory: Foundations & Applications,
DOI 10.1007/978-0-8176-8322-1, © Springer Science+Business Media New York 2012
248 B List of Notations

Notations for parameters

L Set of links
M Set of channels
Ri Route of channel i
Ml Set of channels transmitted on link l
l
P Total power target on link l
Ll,s Fiber loss coefficient at span s on link l
Gi,l,s Gain coefficient of channel i at span s on link l
ASEi,l,s ASE noise power on channel i at span s on link l
hi,l,s Span transmission for channel i at span s on link l
Ti,i Link transmission for channel i on link l
Gi,l Gain value for channel i on the spectral shape of the optical
amplifier on link l
G = [Gi,s ] δl,s Filter loss of the optical amplifier at span s on link l
Γ Network system matrix
Γl Link system matrix

B.2 List of Symbols

H∞ Hardy space of all complex-valued functions of a complex variable s which
are analytic and bounded in the open RHP.
C̄+ closed RHP of the complex plane.
tˆ slow time-scale continuous-time variable.
ν̇ = dν ˆ
dt dot notation represents the derivative with respect to the slow time-scale, t .
λmin (A), λmax (A) maximum and minimum eigenvalues, respectively, of matrix A.
σ̄ (A) largest singular value of a matrix A.
σ (A) smallest singular value of a matrix A.
C = C([−τ, 0], n ) set of all continuous functions mapping [−τ, 0] to n .
xt ∈ C set of values x(q) for all q ∈ [t − τ, t].
Cĥ space of continuous functions, φ, mapping [−τ, 0] to n such that φc < ĥ,
where ĥ is a positive real number.
OSNRi OSNR for the ith channel.
G,k,i or G,i channel dependent gain of the OA in the kth span on the th link for
the ith channel.
L,k wavelength independent loss coefficient for the kth span on the th link.
ASE,k,i ASE noise power introduced by the OA in the kth span on the th link for
the ith channel.
h,k,i optical span transmission, or the total power transmitted by the kth span on
the th link for the ith channel.
T,i link transmission, or the total transmitted power across all spans for the th
link and the ith channel.
λi optical wavelength corresponding to channel i.
νi optical frequency corresponding to channel i.
B.2 List of Symbols 249

u−i vector of elements ui with the ith entry deleted.

umax largest possible input power, i.e., ui ≤ umax for all i.
Ji Nash game cost function for channel i.
Ui utility function for the Nash game or channel i.
αi channel cost associated with the input power ui .
βi utility weight associated with Ui .
ai a channel dependent design parameter for channel i.
Γ˜ Γ matrix such that the diagonal elements are replaced with diag(ai ).
μ scalar channel price computed by the link algorithm.
K number of iterations of the time step, n, before the link algorithm is updated.
k̄ iterative time step on the slow time-scale of the link algorithm.
f
τi,j forward propagation delay experienced by the signal from source j to OSNR
output i.
τib backward propagation delay experienced by the signal from OSNR output i to
the channel source i.
τi,j time-delay from source j to OSNR output i to the source i.
f
τj forward propagation delay from source j to the single OSNR output of the
single-sink network configuration.
τ f forward propagation delay from source to OSNR output in the single-link con-
figuration.
τ b backward propagation delay from the OSNR output to the source in the single-
link configuration.
τ round trip time-delay, from source to OSNR output and back, in the single-link
configuration.
ρi tunable control parameter for the ith channel algorithm.
eig(A) set of eigenvalues of the matrix A.
Re[s] real component of the complex value s.
Im[s] imaginary component of the complex value s.
s complex domain variable.
1row a row vector with all elements equal to 1.
t continuous-time variable on the “fast” time-scale of the channel algorithm.
ε relates tˆ to t by tˆ = εt, where ε = K1 .
ûi composition ui ◦ t.
β column matrix with elements βi .
zi shifted ui variable, zi = ûi − u∗i .
x shifted channel price parameter, μ, i.e. x = μ − μ∗ .
Γ˜
Γ˜a a matrix with elements ai,ji .
ρ diagonal matrix with elements ρi , i.e. ρ = diag(ρi ).
ẑ(tˆ) coordinate shift defined as ẑ(tˆ) = z(tˆ − τ̂ f ) − h(x(tˆ)).
x̂(tˆ) solution of the reduced system (11.46).
z composition defined as z = (ŷ ◦ tˆ).
τ̂ time-delay in the slow time-scale which relates to τ via ε as τ̂ = ετ .
ž solution of the boundary-layer system (11.47).
A1 boundary-layer system matrix, A1 = −ρ Γ˜a .
250 B List of Notations

ζ represents a characteristic polynomial.

ΩΓ uncertainty set for the system matrix, Γ .
ΓΔ perturbed system of the nominal system matrix Γ , i.e., ΓΔ ∈ ΩΔ .
ΔΓ additive uncertainty matrix for the nominal system matrix Γ .
ΔΓi,row an (n × n) matrix with all elements equal to zero except the ith row of ΔΓ .
q̂i independent uncertainty bound on the 2-norm of ΔΓi,row .
Ωj uncertainty set for the input uj .
uΔj uncertain input signal, i.e., uΔj ∈ Ωj .
δj input multiplicative uncertainty on the input signal uj .
q̄j independent uncertainty bound for the input uncertainty, i.e., δj 2 ≤ q̄j .
Ωni uncertainty set for the transmitter noise n0,i .
nΔ,0,i uncertain transmitter noise, i.e., nΔ,0,i ∈ Ωni .
δni transmitter noise uncertainty.
References

1. Acemoglu, D., Ozaglar, A.: Costs of competition in general networks. In: Proceedings of the
44th IEEE Conference on Decision and Control, December, pp. 5324–5329 (2005)
2. Agrawal, G.P.: Fiber-Optic Communication Systems, 3rd edn. Wiley, New York (2002)
3. Alpcan, T.: Noncooperative games for control of networked systems. Ph.D. thesis, University
of Illinois at Urbana–Champaign, Illinois, USA (2006)
4. Alpcan, T., Basar, T.: A game-theoretic framework for congestion control in general topology
networks. In: Proceedings of the 41st IEEE Conference on Decision and Control, December,
pp. 1218–1224 (2002)
5. Alpcan, T., Basar, T.: Global stability analysis of an end-to-end congestion control. In: Pro-
ceedings of the 42nd IEEE Conference on Decision and Control, December, pp. 1092–1097
(2003)
6. Alpcan, T., Basar, T.: A hybrid systems model for power control in multicell wireless data
networks. Perform. Eval. 57(4), 477–495 (2004)
7. Alpcan, T., Basar, T.: Distributed algorithms for Nash equilibria of flow control games. Adv.
Dyn. Games: Appl. Econ. Finance Optim. Stoch. Control Ann. Dynam. Games 7, 473–498
(2005)
8. Alpcan, T., Basar, T., Srikant, R., Altman, E.: CDMA uplink power control as a noncoopera-
tive game. In: Proceedings of the 40th IEEE Conference on Decision and Control, December,
pp. 197–202 (2001)
9. Alpcan, T., Fan, X., Basar, T., Arcak, M., Wen, J.T.: Power control for multicell CDMA
wireless networks: a team optimization approach. Wirel. Netw. 14(5), 647–657 (2008)
10. Altman, E., Altman, Z.: S-modular games and power control in wireless networks. IEEE
Trans. Autom. Control 48(5), 839–842 (2003)
11. Altman, E., Basar, T.: Multiuser rate-based flow control. IEEE Trans. Commun. 46(7), 940–
949 (1998)
12. Altman, E., Basar, T., Jimenez, T., Shimkin, N.: Routing into two parallel links: game-
theoretic distributed algorithms. J. Parallel Distrib. Comput. 61(9), 1367–1381 (2001)
13. Altman, E., Basar, T., Srikant, R.: Nash equilibria for combined flow control and routing
in networks: asymptotic behavior for a large number of users. IEEE Trans. Autom. Control
47(6), 917–930 (2002)
14. Altman, E., Boulogne, T., El-Azouziand, R., Jimenez, T., Wynter, L.: A survey on networking
games in telecommunications. Comput. Oper. Res. 33(2), 286–311 (2006)
15. Altman, E., Solan, E.: Constrained games: the impact of the attitude to adversary’s con-
straints. IEEE Trans. Autom. Control 54(10), 2435–2440 (2009)
16. Arrow, K.J., Debreu, G.: Existence of an equilibrium for a competitive economy. Economet-
rica 22(3), 265–290 (1954)

L. Pavel, Game Theory for Control of Optical Networks, 251

Static & Dynamic Game Theory: Foundations & Applications,
DOI 10.1007/978-0-8176-8322-1, © Springer Science+Business Media New York 2012
252 References

17. EI Azouzi, R., Altman, E.: Constrained traffic equilibrium in routing. IEEE Transactions on
Automatic Control 48(9), 1656–1660 (2003)
18. Bala, K., Brackett, C.: Cycles in wavelength routed optical networks. IEEE J. Lightwave
Technol. 14(7), 1585–1594 (1996)
19. Banerjee, D., Mukherjee, B.: A practical approach for routing and wavelength assignment
in large wavelength-routed optical networks. IEEE J. Sel. Areas Commun. 14(5), 903–908
(1996)
20. Basar, T., Olsder, G.J.: In: Dynamic Noncooperative Game Theory. SIAM Series Classics in
Applied Mathematics, 2nd edn. (1999)
21. Basar, T., Srikant, R.: Revenue-maximizing pricing and capacity expansion in a many-users
regime. In: Proceedings of the 21st IEEE Conference on Computer Communications (INFO-
COM), June 2002, pp. 294–301 (2002)
22. Bazarra, M.S., Sherali, H.D., Shetty, C.M.: In: Nonlinear programming: theory and algo-
rithms, 3rd edn. Wiley-Interscience, New York (2006)
23. Bellare, M., Goldreich, O., Sudan, M.: Free bits, PCPs and non-approximability—towards
tight results. SIAM J. Comput. 27, 804–915 (1998)
24. Bertsekas, D.P.: Nonlinear Programming, 2nd edn. Athena Scientific, Nashua (1999)
25. Bertsekas, D.P., Tsitsiklis, J.N.: Parallel and Distributed Computation: Numerical Methods.
Prentice Hall, New York (1989)
26. Bilo, V., Flammini, M., Moscardelli, L.: On Nash equilibria in non-cooperative all-optical
networks. In: Diekert, V., Durand, B. (eds.) STACS 2005. LNCS, vol. 3404, pp. 448–459.
Springer, Berlin (2005)
27. Bilo, V., Flammini, M., Moscardelli, L.: On Nash Equilibria in Non-cooperative All-Optical
Networks. Optical Switching and Networking (2010)
28. Bilo, V., Moscardelli, L.: The price of anarchy in all-optical networks. In: S’ykora, R.O. (ed.)
SIROCCO 2004. LNCS, vol. 3104, pp. 13–22. Springer, Heidelberg (2004)
29. Borella, M., Jue, J., Banerjee, D., Ramamurthy, B., Mukherjee, B.: Optical components for
WDM lightwave networks. Proc. IEEE 85(8), 1274–1307 (1997)
30. Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge
(2004)
31. Brown, G.W.: Activity Analysis of Production and Allocation. Wiley, New York (1951)
32. Charnes, A., Eisner, M., Kortanek, K.O.: On weakly balanced games and duality theory. Cah.
Cent. Etud. Rech. Oper. 12, 7–21 (1970)
33. Chraplyvy, A.: Limitations on lightwave communications imposed by optical-fiber nonlin-
earities. IEEE/OSA J. Lightwave Technol. 8(10), 1548–1557 (1990)
34. Chraplyvy, A., Nagel, J., Tkach, R.: Equalization in amplified WDM lightwave transmission
systems. IEEE Photonics Technol. Lett. 4(8), 920–922 (1992)
35. de Meyer, B.: Repeated games, duality and the central limit theorem. Math. Oper. Res. 21(1),
237–251 (1996)
36. de Meyer, B., Marino, A.: Duality and optimal strategies in the finitely repeated zero-sum
games with incomplete information on both sides. In: Cahiers de la Maison des Sciences
Economiques, pp. 1–7 (2005)
37. Debreu, G.: A social equilibrium existence theorem. Proc. Natl. Acad. Sci. USA 38(10),
886–893 (1952)
38. Dubey, P.: Inefficiency of Nash equilibria. Math. Oper. Res. 11(1), 1–8 (1986)
39. Erlebach, T., Jansen, K., Kaklamanis, C., Persiano, P.: Optimal wavelength routing on di-
rected fiber trees. Theor. Comput. Sci. 221(1–2), 119–137 (1999)
40. Facchinei, F., Fischer, A., Piccialli, V.: On generalized Nash games and variational inequali-
ties. Oper. Res. Lett. 35(2), 159–164 (2007)
41. Falomari, D., Mandayam, N., Goodman, D.: A new framework for power control in wireless
data networks: games utility and pricing. In: Proceedings of the Allerton Conference on
Communication, Control, and Computing, pp. 546–555 (1998)
42. Fanelli, A., et al.: Game theoretical issues in optical networks. In: Proceedings of ÊICTON
2006, Sep. (2006)
References 253

43. Fiedler, M.: Special Matrices and Their Applications in Numerical Mathematics. Martinus
Nijhoff Publishers, Leiden (1986)
44. Fishman, G.S.: Monte Carlo: Concepts, Algorithms, and Applications. Springer, Berlin
(1996)
45. Forghieri, F., Tkach, R.W., Favin, D.L.: Simple model of optical amplifier chains to evaluate
penalties in WDM systems. IEEE/OSA J. Lightwave Technol. 16(9), 1570–1576 (1998)
46. Foschini, G., Miljanic, Z.: A simple distributed autonomous power control algorithm and its
convergence. IEEE Trans. Veh. Technol. 42(4), 641–646 (1993)
47. Friedman, D.: Evolutionary games in economics. Econometrica 59, 637–666 (1991)
48. Fudenberg, D., Kreps, D.M.: Learning mixed equilibria. Games Econ. Behav. 5, 320–367
(1993)
49. Fudenberg, D., Levine, D.K.: Theory of Learning in Games. MIT Press, Cambridge (1998)
50. Fudenberg, D., Tirole, J.: Game Theory, 2nd edn. MIT Press, Cambridge (1991)
51. Garey, M.R., Johnson, D.S., Miller, G.L., Papadimitriou, C.H.: The complexity of coloring
circular arcs and chords. SIAM J. Algebr. Discrete Methods 1(2), 216–227 (1980)
52. Gargano, L., Vaccaro, U.: Routing in all-optical networks: algorithmic and graph theoretic
problems. In: Numbers, Information and Complexity, pp. 555–578. Kluwer Academic, Dor-
drecht (2000)
53. Georgakopoulos, G.F., Kavvadias, D.J., Sioutis, L.G.: Nash equilibria in all-optical networks.
Discrete Math. 309(13), 4332–4342 (2009)
54. Greenwald, A., Friedman, E.J., Shenker, S.: Learning in network contexts: experimental re-
sults from simulations. Games Econ. Behav. 35, 80–123 (2001)
55. Gu, K., Kharitonov, V.L., Chen, J.: Stability of Time-Delay Systems. Birkhäuser, Boston
(2003)
56. Harker, P.T.: Generalized Nash games and quasi-variational inequalities. Eur. J. Oper. Res.
54(1), 81–94 (1991)
57. Harsanyi, J.C.: Games with randomly disturbed payoffs: a new rationale for mixed-strategy
equilibrium points. Int. J. Game Theory 2, 1–23 (1973)
58. Hofbauer, J., Sigmund, K.: Evolutionary Games and Population Dynamics. Cambridge Uni-
versity Press, Cambridge (1998)
59. Horn, R.A., Johnson, C.R.: Matrix Analysis. Cambridge University Press, Cambridge (1999)
60. Huang, M., Malhame, R.P., Caines, P.E.: Nash certainty equivalence in large population
stochastic dynamic games: connections with the physics of interacting particle systems. In:
Proceedings of the 45th IEEE Conference on Decision and Control, December, pp. 4921–
4926 (2006)
61. Ji, H., Huang, C.: Non-cooperative uplink power control in cellular radio systems. Wirel.
Netw. 4(3), 233–240 (1998)
62. Jia, X., Du, D., Hu, X., Lee, M., Gu, J.: Optimization of wavelength assignment for QoS
multicast in WDM networks. IEEE Trans. Commun. 49(2), 341–350 (2001)
63. Johari, R., Mannor, S., Tsitsiklis, J.N.: Efficiency loss in a resource allocation game: a single
link in elastic supply. In: Proceedings of the 43rd IEEE Conference on Decision and Control,
December, pp. 4679–4683 (2004)
64. Johari, R., Tsitsiklis, J.: Efficiency loss in a network resource allocation game. Math. Oper.
Res. 29(3), 407–435 (2004)
65. Kandukuri, S., Boyd, S.: Simultaneous rate and power control in multirate multimedia
CDMA systems. In: Proceedings of the 6th IEEE International Symposium on Spread Spec-
trum Techniques and Applications, September, pp. 570–574 (2000)
66. Kandukuri, S., Boyd, S.: Optimal power control in interference-limited fading wireless chan-
nels with outage-probability specifications. IEEE Trans. Wirel. Commun. 1(1), 46–55 (2002)
67. Kelly, F.: Charging and rate control for elastic traffic. Eur. Trans. Telecommun. 8, 33–37
(1997)
68. Kelly, F.P., Maulloo, A.K., Tan, D.: Rate control for communication networks: shadow
prices, proportional fairness and stability. J. Oper. Res. Soc. 49(3), 237–252 (1998)
69. Khalil, H.K.: Nonlinear Systems, 3rd edn. Prentice Hall, New York (2002)
254 References

70. Kinderlehrer, D.: An Introduction to Variational Inequalities and Their Applications. Aca-
demic Press, San Diego (1980)
71. Korpelevich, G.M.: The extragradient method for finding saddle points and other problems.
Èkon. Mat. Metody 12, 747–756 (1976) [translated as Matecon.]
72. Koskie, S., Gajic, Z.: A Nash game algorithm for SIR-based power control in 3G wireless
CDMA networks. IEEE/ACM Trans. Netw. 13(5), 1017–1026 (2005)
73. Koutsoupias, E., Papadimitriou, C.H.: Worst-case equilibria. In: Proceedings of the 16th An-
nual Symposium STACS, pp. 404–413 (1999)
74. Krasovskii, N.N., Subbotin, A.I.: Game-Theoretical Control Problems. Springer, Berlin
(1988)
75. Krishna, V., Sjostrom, T.: On the convergence of fictitious play. Math. Oper. Res. 23, 479–
511 (1998)
76. Kunniyur, S., Srikant, R.: End-to-end congestion control schemes: utility functions, random
losses and ECN marks. IEEE/ACM Trans. Netw. 11(5), 689–702 (2003)
77. Laplante, P.A.: Comprehensive Dictionary of Electrical Engineering. Springer, Berlin (1999)
78. Libman, L., Orda, A.: The designer’s perspective to atomic noncooperative networks.
IEEE/ACM Trans. Netw. 7(6), 875–884 (1999)
79. Loja, A., et al.: Inter-domain routing in multiprovider optical networks: game theory and
simulations. In: Proceedings of Next Gen. Internet Networks, May, pp. 157–164 (2005)
80. Low, S., Paganini, F., Doyle, J.: Internet congestion control. IEEE Control Syst. Mag. 22(1),
28–43 (2002)
81. Low, S.H., Lapsley, D.E.: Optimization flow control-I: basic algorithm and convergence.
IEEE/ACM Trans. Netw. 7(6), 861–874 (1999)
82. Luo, Z., Pang, J.: Analysis of iterative waterfilling algorithm for multiuser power control in
digital subscriber lines. EURASIP J. Appl. Signal Process. 2006, 1–10 (2006)
83. Marbach, P., Berry, R.: Downlink resource allocation and pricing for wireless networks. In:
Proceedings of the 21st IEEE Conference on Computer Communications (INFOCOM), June,
pp. 1470–1479 (2002)
84. Marden, J.R., Arslan, G., Shamma, J.S.: Joint strategy fictitious play with inertia for potential
games. In: Proceedings of the 44th IEEE Conference on Decision and Control, December,
pp. 6692–6697 (2005)
85. Mecozzi, A.: On the optimization of the gain distribution of transmission lines with unequal
amplifier spacing. IEEE Photonics Technol. Lett. 10(7), 1033–1035 (1998)
86. Milis, I., Pagourtzis, A., Potika, K.: Selfish routing and path coloring in all-optical networks.
In: Proceedings of CAAN2007, pp. 71–84 (2007)
87. Mitra, D.: An asynchronous distributed algorithm for power control in cellular radio systems.
In: Proceedings of the 4th WINLAB Workshop on Third Generation Wireless Information
Networks, November, pp. 249–257 (1993)
88. Mokhtar, A., Azizoglu, M.: Adaptive wavelength routing in all-optical networks. IEEE/ACM
Trans. Netw. 6(2), 197–206 (1998)
89. Monderer, D., Shapley, L.S.: Potential games. Games Econ. Behav. 14(1), 124–143 (1996)
90. Morgan, J., Romaniello, M.: Generalized quasi-variational inequalities and duality. J. In-
equal. Pure Appl. Math. 4(2), Article 28, 1–7 (2003)
91. Morgan, J., Romaniello, M.: Generalized quasi-variational inequalities: duality under pertur-
bations. J. Math. Anal. Appl. 324(2), 773–784 (2006)
92. Mukherjee, B.: WDM optical communication networks: progress and challenges. IEEE J.
Sel. Areas Commun. 18(10), 1810–1824 (2000)
93. Myerson, R.: Game Theory: Analysis of Conflict. Harvard University Press, Cambridge
(1991)
94. Mync, H.: Nonnegative Matrices. Wiley, New York (1988)
95. Nash, J.: Equilibrium points in n-person games. Proc. Natl. Acad. Sci. USA 36(1), 48–49
(1950)
96. Nash, J.: Non-cooperative games. Ann. Math. 54(2), 286–295 (1951)
References 255

97. Nisan, N., Roughgarden, T., Tardos, E., Vazirani, V. (eds.): Algorithmic Game Theory. Cam-
bridge University Press, Cambridge (2007)
98. Owen, G.: Game Theory, 3rd edn. Academic Press, San Diego (1995)
99. Ozdaglar, A., Bertsekas, D.: Routing and wavelength assignment in optical networks.
IEEE/ACM Trans. Netw. 11(2), 259–272 (2003)
100. Paganini, F., Wang, Z., Doyle, J.C., Low, S.H.: Congestion control for high performance,
stability, and fairness in general networks. IEEE/ACM Trans. Netw. 13(1), 43–55 (2005)
101. Pan, Y.: A game theoretical approach to constrained OSNR optimization problems in optical
networks. Ph.D. Thesis, University of Toronto, Toronto, Canada (2009)
102. Pan, Y., Alpcan, T., Pavel, L.: A distributed optimization approach to constrained OSNR
problem. In: Proceedings of the 17th IFAC World Congress, July (2008)
103. Pan, Y., Alpcan, T., Pavel, L.: Effects of parameters on Nash games with OSNR target. In:
Proceedings of the 2nd International Workshop on Game Theory in Communication Net-
works (GameComm), October (2008)
104. Pan, Y., Alpcan, T., Pavel, L.: A system performance approach to OSNR optimization in
optical networks. IEEE Trans. Commun. 58(4), 1193–1200 (2010)
105. Pan, Y., Pavel, L.: OSNR optimization in optical networks: extension for capacity constraints.
In: Proceedings of the 2005 American Control Conference, June, pp. 2379–2385 (2005)
106. Pan, Y., Pavel, L.: Games with coupled propagated constraints in optical networks: the multi-
link case. In: Proceedings of the 46th IEEE Conference on Decision and Control, December,
pp. 3443–3449 (2007)
107. Pan, Y., Pavel, L.: Global convergence of an iterative gradient algorithm for the Nash equilib-
rium in an extended OSNR game. In: Proceedings of the 26th IEEE Conference on Computer
Communications (INFOCOM), May, pp. 206–212 (2007)
108. Pan, Y., Pavel, L.: Iterative algorithms for Nash equilibrium of an extended OSNR optimiza-
tion game. In: Proceedings of the 6th International Conference on Networking (ICN), April
(2007)
109. Pan, Y., Pavel, L.: OSNR optimization with link capacity constraints in WDM networks:
a cross layer game approach. In: Proceedings of the 4th IEEE Conference on Broadband
Communications, Networks and Systems (BroadNets), September (2007)
110. Pan, Y., Pavel, L.: A Nash game approach for OSNR optimization with capacity constraint
in optical links. IEEE Trans. Commun. 56(11), 1919–1928 (2008)
111. Pan, Y., Pavel, L.: Games with coupled propagated constraints in general topology optical
networks. In: Proceedings of the 1st International Conference on Game Theory in Networks
(GameNets), May, pp. 549–558 (2009)
112. Pan, Y., Pavel, L.: Games with coupled propagated constraints in optical networks with multi-
link topologies. Automatica 45(4), 871–880 (2009)
113. Parrilo, P.A.: Polynomial games and sum of squares optimization. In: Proceedings of the 45th
IEEE Conference on Decision and Control, December, pp. 2855–2860 (2006)
114. Pavel, L.: Dynamics and stability in optical communication networks: a system theory frame-
work. Automatica 40(8), 1361–1370 (2004)
115. Pavel, L.: Power control for OSNR optimization in optical networks: a noncooperative game
approach. In: Proceedings of the 43rd IEEE Conference on Decision and Control, December,
pp. 3033–3038 (2004)
116. Pavel, L.: An extension of duality and hierarchical decomposition to a game-theoretic frame-
work. In: Proceedings of the 44th IEEE Conference on Decision and Control, December,
pp. 5317–5323 (2005)
117. Pavel, L.: A nested noncooperative game formulation for OSNR optimization in distributed
optical links. In: Proceedings of the 44th IEEE Conference on Decision and Control, Decem-
ber, pp. 6958–6965 (2005)
118. Pavel, L.: A noncooperative game approach to OSNR optimization in optical networks. IEEE
Trans. Autom. Control 51(5), 848–852 (2006)
119. Pavel, L.: OSNR optimization in optical networks: modeling and distributed algorithms via
a central cost approach. IEEE J. Sel. Areas Commun. 24(4), 54–65 (2006)
256 References

120. Pavel, L.: An extension of duality to a game-theoretic framework. Automatica 43(2), 226–
237 (2007)
121. Pavel, L.: A nested noncooperative OSNR game in distributed WDM optical links. IEEE
Trans. Commun. 55(6), 1220–1230 (2007)
122. Perakis, G.: The price of anarchy when costs are non-separable and asymmetric. In: Proceed-
ings of the 10th International Integer Programming and Combinatorial Optimization (IPCO)
Conference, June, pp. 46–58 (2004)
123. Perakis, G.: The “price of anarchy” under nonlinear and asymmetric costs. Math. Oper. Res.
32(3), 614–628 (2007)
124. Ramamurthy, B., Datta, D., Feng, H., Heritage, J., Mukherjee, B.: Impact of transmission
impairments on the teletraffic performance of wavelength-routed optical networks. IEEE J.
Lightwave Technol. 17(10), 1713–1723 (1999)
125. Ramaswami, R., Sivarajan, K.: Routing and wavelength assignment in all-optical networks.
IEEE/ACM Trans. Netw. 3(5), 489–500 (1995)
126. Ramaswami, R., Sivarajan, K.N.: Optical Networks: A Practical Perspective, 2nd edn. Aca-
demic Press, San Diego (2002)
127. Robinson, J.: An iterative method of solving a game. Ann. Math. 54, 296–301 (1951)
128. Rosen, J.B.: Existence and uniqueness of equilibrium points for concave n-person games.
Econometrica 33(3), 520–534 (1965)
129. Roughgarden, T.: Selfish Routing and the Price of Anarchy. MIT Press, Cambridge (2005)
130. Roughgarden, T., Tardos, E.: How bad is selfish routing? J. ACM 49(2), 236–259 (2002)
131. Rouskas, G., Perros, H.: A tutorial on optical networks. In: Advanced Lectures on Network-
ing, vol. 2497, pp. 155–193. Springer, Berlin (2002)
132. Sandholm, W.H.: Population Games and Evolutionary Dynamics. Cambridge University
Press, Cambridge (2009)
133. Saraydar, C., Mandayam, N.B., Goodman, D.J.: Pricing and power control in a multicell
wireless data network. IEEE J. Sel. Areas Commun. 19(10), 1883–1892 (2001)
134. Saraydar, C., Mandayam, N.B., Goodman, D.J.: Efficient power control via pricing in wire-
less data networks. IEEE Trans. Commun. 50(2), 291–303 (2002)
135. Scutari, G., Barbarossa, S., Palomar, D.P.: Potential games: a framework for vector power
control problems with coupled constraints. In: Proceedings of IEEE International Conference
on Acoustics, Speech and Signal Processing (ICASSP) (2006)
136. Selten, R.: Re-examination of the perfectness concept for equilibrium points in extensive
games. Int. J. Game Theory 4, 25–55 (1975)
137. Shamma, J.S., Arslan, G.: Dynamic fictitious play, dynamic gradient play and distributed
convergence to Nash equilibria. IEEE Trans. Autom. Control 50(1), 312–326 (2005)
138. Shandbag, U., et al.: Nash equilibrium problems with congestion costs and shared con-
straints. In: Proceedings of the Conference on Decision and Control, December (2009)
139. Shen, H., Basar, T.: Differentiated Internet pricing using a hierarchical network game model.
In: Proceedings of the 2004 American Control Conference, June, pp. 2322–2327 (2004)
140. Slusarek, M.: Optimal online colouring of circular arc graphs. Inform. Theor. Appl. 29(5),
423–429 (1995)
141. Smith, J.M.: The theory of games and the evolution of animal conflicts. J. Theor. Biol. 7,
209–221 (1974)
142. Smith, J.M., Price, G.R.: The logic of animal conflict. Nature 246, 15–18 (1973)
143. Srikant, R.: The Mathematics of Internet Congestion Control. Systems & Control: Founda-
tions & Applications. Birkhäuser, Boston (2004)
144. Stefanovic, N.: Robust power control of optical networks with time-delays. Ph.D. thesis,
University of Toronto, Toronto, Canada (2010)
145. Stefanovic, N., Pavel, L.: Link level power control of optical networks with time-delay. In:
Proceedings of the 46th IEEE Conference on Decision and Control, December, pp. 5198–
5203 (2007)
146. Stefanovic, N., Pavel, L.: A Lyapunov–Krasovskii stability analysis for game-theoretic based
power control in optical networks. In: Proceedings of GameComm 2008, October (2008)
References 257

147. Stefanovic, N., Pavel, L.: Robust power control of optical links with time-delay. In: Proceed-
ings of European Control Conference, Budapest, Hungary, August (2009)
148. Stefanovic, N., Pavel, L.: Robust power control of single sink optical networks with time-
delays. In: Proceedings of the 48th IEEE Conference on Decision and Control, Shanghai,
China, December, pp. 2034–2039 (2009)
149. Stefanovic, N., Pavel, L.: A stability analysis with time-delay of primal-dual power control
in optical links. Automatica 45(5), 1319–1325 (2009)
150. Stefanovic, N., Pavel, L.: A Lyapunov–Krasovskii stability analysis for game-theoretic based
power control in optical links. J. Telecommun. Syst. 47(1), 19–33 (2011)
151. Stein, N.D., Ozdaglar, A., Parrilo, P.A.: Separable and low-rank continuous games. In: Pro-
ceedings of the 45th IEEE Conference on Decision and Control, December, pp. 2849–2854
(2006)
152. Strand, J., Chiu, A., Tkach, R.: Issues for routing in the optical layer. IEEE Commun. Mag.
39(2), 81–87 (2001)
153. Tan, C., Palomar, D., Chiang, M.: Solving nonconvex power control problems in wireless net-
works: low SIR regime and distributed algorithms. In: Proceedings of the 48th IEEE Global
Telecommunications Conference (GLOBECOM), November (2005)
154. Tonguz, O., Flood, F.: Gain equalization of EDFA cascades. IEEE J. Lightwave Technol.
15(10), 1832–1841 (1997)
155. Tonguz, O., Flood, F.: EDFA-based DWDM lightwave transmission systems with end-to-end
power and SNR equalization. IEEE Trans. Commun. 50(8), 1282–1292 (2002)
156. Turkcu, O., Subramaniam, S.: Blocking in reconfigurable optical networks. In: Proceedings
of the 26th IEEE Conference on Computer Communications (INFOCOM), May, pp. 188–
196 (2007)
157. van Damme, E.: Stability and Perfection of Nash Equilibria, 2nd edn. Springer, Berlin (1987)
158. Vincent, T.L., Brown, J.S.: Evolutionary Game Theory, Natural Selection, and Darwinian
Dynamics. Cambridge University Press, Cambridge (2005)
159. von Neumann, J., Morgenstern, O.: Theory of Games and Economic Behavior. Princeton
University Press, Princeton (1944)
160. Yaiche, H., Mazumdar, R., Rosenberg, C.: A game theoretic framework for bandwidth allo-
cation and pricing in broadband networks. IEEE/ACM Trans. Netw. 8(5), 667–678 (2000)
161. Yariv, A.: Signal-to-noise considerations in fiber links with periodic or distributed optical
amplification. Opt. Lett. 15(19), 1064–1066 (1990)
162. Yassine, A., et al.: Competitive game theoretic optimal routing in optical networks. In: Pro-
ceedings SPIE, May, pp. 27–36 (2002)
163. Yates, R.: A framework for uplink power control in cellular radio systems. IEEE J. Sel. Areas
Commun. 13(7), 1341–1347 (1995)
164. Zander, J.: Performance of optimum transmitter power control in cellular radio systems.
IEEE Trans. Veh. Technol. 41(1), 57–62 (1992)
165. Zang, H., Jue, J., Mukherjee, B.: A review of routing and wavelength assignment approaches
for wavelength-routed optical WDM networks. Opt. Netw. Mag. 1(1), 47–60 (2000)
166. Zhu, H., Mukherjee, B.: Online connection provisioning in metro optical WDM networks
using reconfigurable OADMs. IEEE/OSA J. Lightwave Technol. 23(10), 2893–2901 (2005)
167. Zhu, K., Mukherjee, B.: Traffic grooming in an optical WDM mesh network. IEEE J. Sel.
Areas Commun. 20(1), 122–133 (2002)
168. Zhu, Q., Pavel, L.: Enabling differentiated services using generalized power control model
in optical networks. IEEE Trans. Commun. 57(9), 2570–2575 (2009)
Subject Index

α–ω Continuous game, 18, 27

γ -Link topology, 92, 122 Continuous wave, 3
Continuous-time, 198, 206
A Continuously differentiable, 48, 100, 102, 109,
Absorption, 74 229–231
Action set, 12, 17, 32, 45, 46 Convex, 46, 50, 51, 53, 94, 147, 231
Additive uncertainty, 204 Coordination signal, 66
Adjustable, 87, 92 Correlated equilibria, 25
Adjustment factor, 87, 93 Cost function, 47, 144, 151, 215, 218
All-optical, 74, 211 Coupled action set, 5, 56
Amplified spontaneous emission noise, 79 Coupled constraint, 5, 52, 55–57, 106, 114,
Amplifier gain dynamics, 80 121, 128–130, 132, 134–136
ASE noise, 79, 86 Coupled Nash game, 56–59
Asymptotically stable, 241, 244 Coupled power constraint, 5, 94
Attenuation, 5, 79 Coupling, 75, 82
Augmented constraint, 115
Augmented function, 59
D
B Decentralized algorithm, 104, 114, 119
Berge’s Maximum Theorem, 49, 53, 109, 235 Decision-maker, 12
Best response, 22, 26, 36, 48, 49 Decision-making agent, 1
Bit-error rate, 5 Delay-dependent, 244
Boundary-layer system, 200, 202, 238 Delay-independent, 244
Brouwer’s Fixed-point Theorem, 50, 235 Demultiplexed, 74
Diagonally dominant, 233
C Dispersion, 75
Channel algorithm, 197 Distributed algorithm, 1, 73, 163
Channel transmission matrix, 86 Dual cost function, 62, 64, 115
Classical game, 11, 12 Dynamic game, 20
Closed-loop system, 189, 192, 203 Dynamic resource allocation, 1, 97
Color collision, 213 Dynamics, 200
Compact, 46, 50, 147, 231, 236
Complementary slackness, 63, 118, 232 E
Congestion control, 12 Efficiency, 25, 143, 151–153, 156, 213
Constrained minimization, 59, 60, 152 End-to-end case, 87
Constrained optimization, 230 End-to-end game, 121, 126, 135
Constrained set, 12 Exponential stability, 201, 241
Continuous, 48–50, 53, 235 Exponentially stable, 240, 241

L. Pavel, Game Theory for Control of Optical Networks, 259

Static & Dynamic Game Theory: Foundations & Applications,
DOI 10.1007/978-0-8176-8322-1, © Springer Science+Business Media New York 2012
260 Subject Index

Extensive form game, 13 Mesh topology, 76, 134

Minimax solution, 21
F Mixed strategy, 18, 30
Fast time-scale, 238 Mixed-strategy, 32
Feasible vector, 230 Mixed-strategy best response, 36
Feedback system, 72 Mixed-strategy Nash equilibria, 34, 36
Fictitious play, 26 Mixed-strategy Nash equilibrium, 216
Finite strategy game, 12 Modified Nash game, 64, 66, 116, 118
Finite strategy games, 7 Multi-link topology, 76, 122, 142
First-Fit algorithm, 220 Multiplexed, 74, 79
Fixed-point solution, 53, 54, 56, 59–62
Four-wave mixing, 75 N
Nash equilibrium, 25, 46, 47, 49, 213
G NE solution, 57
Game theory, 11, 12 Network element, 4
Games with coupled constraints, 114 Network resource, 2
Generalized Nash game, 56 Network system matrix, 91
Gradient algorithm, 110, 111 NG cost function, 51, 52, 58, 59, 61, 115
Graph coloring problem, 213 NG-feasibility, 59, 62, 63
Greedy algorithm, 220 Noise power, 76, 79, 80, 88, 89
Noncooperative game, 12, 47, 211–213
H Nonlinear effects, 75
Hessian, 53, 95, 229 Normal form game, 16
Hierarchical algorithm, 118 Normalized equilibrium, 64
Hierarchical decomposition, 56, 60, 114
O
Objective function, 12
I
Oblivious collision-free, 217
Information structure, 16, 25
One-shot game, 16
Information transmission, 3
One-shot games, 20
Inner NE solution, 48, 50, 52, 53, 107, 109 Optical add/drop multiplexer, 76
Intensity modulation, 74, 76 Optical amplified span, 76, 78, 79, 89
Intensity-modulated, 3 Optical amplifier, 74, 76, 79, 82, 86, 89, 93
Optical coupler, 161
K Optical cross-connect, 76
Kakutani Fixed-point Theorem, 235 Optical fiber, 3, 74, 76, 86
Optical line terminal, 4
L Optical link, 163
Ladder-nested, 122–124, 127, 136 Optical network, 74, 76, 211, 217
Lagrangian extension, 56, 60, 62, 63 Optical network client, 4
Lagrangian function, 232 Optical power, 72, 74, 79
Lagrangian optimality, 61, 63, 118 Optical signal, 3, 74
Laser, 74 Optical signal power, 80, 83, 87
Leader, 66 Optical signal-to-noise ratio, 5, 72, 78, 79
Lightpath, 75 Optical switch, 76
Link algorithm, 197 Optical-to-electrical, 74
Link capacity constraint, 75, 94, 98, 106, 107 Optimal channel power, 119
Link OSNR model, 91 Optimal routing, 12
Lipschitz continuity, 236 Optimal solution, 50, 51, 144, 148, 150, 153,
Loss, 79, 82, 89 157
Lyapunov function, 202, 238 Orthogonal, 12, 46
Lyapunov–Razumikhin theorem, 202, 240, 243
P
M Parallel update algorithm, 110
Matrix game, 17, 18, 27 Pareto efficient, 24, 25
Subject Index 261

Partitioned Nash game, 121, 122, 124, 125, Spectral gain shape, 79, 82, 86
127, 136, 137 Spectral radius, 234
Path, 214 Spectral shape, 75
Payment function, 215, 218 Stackelberg equilibrium, 25
Payoff function, 13, 215 Stackelberg game, 66
Penalty based approach, 114 Stage Nash game, 122–125, 136
Photodetector, 74 Stochastic games, 20
Physical layer, 5 Strategic form, 16
Physical layer impairment, 5 Strategic interaction, 11, 12
Player, 11, 12 Strategically dominant, 24
Point-to-point link topology, 76, 98 Strategy, 13, 16, 18, 211, 212, 215, 217
Point-to-point WDM link, 78, 83 Strategy space, 12, 13, 16, 18
Positive definite, 51, 113, 233 Strictly convex, 48, 51, 53
Positive semidefinite, 95 Strictly Pareto dominated, 24
Power allocation, 12 Symmetric game, 17
Power control, 73, 80, 82 System connection matrix, 86
Price algorithm, 197 System cost function, 144
Price of Anarchy, 25, 143, 213, 216, 222 System matrix, 89, 166
Price of Stability, 216
Pricing, 2, 7, 100, 117 T
Pricing function, 99, 107, 121, 124, 219 Time-delay, 189–192, 205
Primal-dual algorithm, 197 Time-delay system, 192, 240, 241
Probability, 18, 32 Time-scale, 197, 237
Projection set, 106 Time-scale decoupling, 198
Projection Theorem, 236 Transmitter, 74, 75, 87, 161
Pure strategy, 12, 32 Transport layer, 3, 5
Pure-strategy, 216 Two-argument, 51, 56, 58–60, 230
Pure-strategy Nash equilibria, 28 Two-player, 17, 50
Pure-strategy Nash Equilibrium, 216
U
R Uncoupled constraint, 45
Randomization, 13, 18, 32 Upper-semi-continuous, 53, 234, 235
Rational player, 25 Utility function, 13, 100–102, 124, 125, 137,
Reaction function, 48–50, 109 215, 218
Reaction set, 48, 53
Receiver, 74 V
Reconfigurable lightpath, 3 Variable optical attenuator, 161
Recursive OSNR model, 90 Variable optical filter, 82
Reduced system, 200, 238 Virtual optical link, 87
Repeated game, 25 Virtual topology, 4, 76
Repeated games, 20
Routing and Path Coloring, 211 W
Routing and wavelength assignment, 211 Wavelength, 74–76
Wavelength assignment, 211
S Wavelength converter, 76
Schur complement, 201 Wavelength division multiplexing, 3
Selfish-RPC, 216 Wavelength-dependent gain, 79
Separable, 46, 52, 58, 59, 65, 115 Wavelength-division multiplexed, 71
Single-sink topology, 76, 128–130 Wavelength-routed optical network, 4
Singleton, 48, 49, 54 Wavelength-routed optical networks, 75
Singular perturbation, 197, 238 Weakly Pareto dominated, 24
Slow time-scale, 238
Social cost, 213, 216, 217 Z
Socially optimal, 24, 143, 151, 213, 216 Zero-sum games, 19