Date of publication xxxx 00, 0000, date of current version xxxx 00, 0000.

Digital Object Identifier

A Mapping and State-of-the-Art Survey on

Multi-Objective Optimization Methods for
Multi-Agent Systems
SHOKOUFEH NADERI1 , MAUDE J. BLONDIN2 (Member, IEEE)
1
Département Génie Électrique & Génie Informatique, Université de Sherbrooke, Sherbrooke, QC, Canada (e-mail: [email protected])
2
Département Génie Électrique & Génie Informatique, Université de Sherbrooke, Sherbrooke, QC, Canada (e-mail: [email protected])
Corresponding author: Shokoufeh Naderi (e-mail: [email protected]).
This work was funded by an NSERC Discovery grant.

ABSTRACT Over the last decades, researchers have studied the Multi-Objective Optimization (MOO)
problem for Multi-Agent Systems (MASs). However, most of them consider the problem formulation to be
the sum of objective functions, and no work has reviewed problems with the formulation of the prioritized
sum of objective functions to facilitate the study of the subject and identify the needs arising from it. In
the context of Multi-Robot Systems (MRSs), most studies only focus on the mathematical development of
their proposed MOO algorithm without paying attention to the real application. There is no comprehensive
review to identify the reliable algorithms already applied to real platforms. Using a mapping and state-of-
the-art review, this paper aims to fill these gaps by first offering a detailed overview of the discrete-time
MOO methods for MASs. More specifically, we classify existing MOO methods based on the formulation
of the problem into the sum of objective functions and the prioritized sum of objective functions. Secondly,
we review the applications of these methods in MRSs and the practical implementation of MOO algorithms
on real MRSs. Finally, we suggest future research directions to extend the existing methods to more realistic
approaches, including open problems in the new research area of the prioritized sum of objective functions
and practical challenges for implementing the existing methods in robotics. This work introduces the field
of MAS to researchers and enables them to position themselves in the current research trends.

INDEX TERMS Multi-agent systems, multi-objective optimization, multi-robot systems, prioritized sum of
objective functions.

I. INTRODUCTION or not sharing their knowledge with each other. Their coor-
dination may be cooperative, competitive, or both, and the
MASs, such as a team of robots, can be applied in many
agents may work on the same or different set of goals [1], [2].
emergency scenarios to prevent humans from being harmed.
Although most MAS applications explore the optimization
These scenarios include landmine clearance, search and res-
of agents’ policies concerning a single objective, real-world
cue missions, extinguishing indoor and outdoor fires, and
applications are naturally multi-objective [3]. Each of these
exploring unknown environments. Apart from this, they can
objectives is presented as an objective function; however,
also be implemented to perform many non-emergency scenar-
optimizing all of them simultaneously is often challenging
ios, including target tracking, collaborative navigation, path-
since they may be contradictory [4]. In other words, im-
finding, and task assignment.
proving one of the objectives may negatively impact another.
In other words, a MAS is a set of agents acting in an Many MOO techniques have been proposed to solve these
environment in collaboration to achieve a common goal faster problems, including evolutionary algorithms [5] and gradient-
than individually. Sometimes, this common goal can be im- based algorithms [6].
possible for an individual agent to reach, such as exploring a
vast and complex environment by a single robot, which is im- MOO has been studied for a long time. Still, recently,
possible considering its limited sensors and battery capacity. it has attracted significant interest [7] due to its potential
Agents work towards fulfilling their tasks by either sharing to solve real-world problems and its various applications in

VOLUME 11, 2023 1

 Shokoufeh Naderi et al.: A Mapping and State-of-the-Art Survey on Multi-Objective Optimization Methods for Multi-Agent Systems

robotics [8], machine learning [9], power systems [10], wire- that class. This is the first work to review problems
less networks [11], and many more. In a multi-agent context, with the formulation of the sum of objective functions,
MOO techniques study conflicting objective functions on a providing the researchers with important gaps in this
region called the Pareto front to compute the Pareto-optimal new line of research.
solutions [12]. 3) It discusses the applications of MOO problems to MRS.
In robotics problems, the usage of MOO is limited to a few Although many applications can be imagined for MOO
applications, including task assignment [13], path-finding in MRS, few have been implemented in the literature.
[14], formation control [15], and simultaneous localization We review the implemented applications.
and mapping (SLAM) [16]. In the majority of these appli- 4) This is the first review paper that surveys MOO meth-
cations, MOO methods have only been applied to single- ods implemented on real MRS. This review will iden-
agent systems. Furthermore, optimization problems arising tify safe and reliable algorithms designed so far. Ensur-
in robotics problems are usually non-convex, while most of ing the applicability and safety of algorithms is very
the existing MOO methods designed for robotics applications important since implementing unreliable algorithms
consider the problem to be convex. As a result, they do not could have serious consequences. In the human-robot
meet the challenges of MOO applied to MRS. Another reason interaction application, for example, failing to perform
for having limited applications of MOO algorithms in MRS a task may result in loss of life.
is that these algorithms often do not consider communication 5) It proposes future research directions in MOO problems
limitations. For large groups of agents using vision tech- for MASs and MRS to identify and highlight the areas
niques, for example, the observed parameters are of large that need more attention from researchers.
dimensions. This incurs high communication costs, while There are also other ways of classifying the MOO meth-
there may be limited communication capabilities. ods, as we discuss in Sect. II. However, classifications are
Thus, a structured review of the existing methodologies and not complete, and overlapping may occur. In other words,
their characteristics is necessary to identify the gaps in the some methods may fall into more than one class or there
subject and ensure that research efforts will meet the needs may be combinations of different classes. In our paper, we
of the robotics field. Therefore, in this paper, we review the chose to classify the problems based on the formulation since
literature on MOO methods for MASs along with their appli- this classification simplifies the understanding of different
cations in MRS. This review provides valuable background MOO methods and their differences. That is because methods
for researchers desiring to take their first step in exploring this falling into the same classes often show similar characteris-
research area. Moreover, we provide a review of the research tics. We also identify future research directions. Therefore,
on MOO methods implemented on real-time MRS, identify- our review is a mapping review on the subject. On the other
ing reliable algorithms and highlighting further the current hand, we mainly aim to address current approaches on the
limitations and gaps in the real-time implementation of MOO subject and highlight an area that is in need of further re-
methods on MRSs. These gaps include considering delays search. That is to say, another part of our study falls within a
in the system and insufficient communication infrastructure, state-of-the-art review. This new research area is solving the
which are often omitted in many MOO algorithms. MOO problem with the formulation of the prioritized sum
The classification of MOO methods, in our paper, is based of objective functions. Our review is the first comprehensive
on the formulation of the problem, i.e., the sum of objective review paper to identify the gaps and future research direc-
functions and the prioritized sum of objective functions. This tions for this problem. This helps draw attention to a research
taxonomy allows us to present a structured view of the ex- subject that researchers omitted while having important ad-
isting methods and identify promising future research direc- vantages, such as improving agents’ decision-making power.
tions. As a result, we pinpoint the fact that problems with the Note that we only review first-order, discrete-time decen-
formulation of the prioritized sum of objective functions have tralized optimization algorithms. In the next paragraphs, a
not been deeply investigated in the literature, and there is no brief description of centralized and decentralized optimiza-
work implementing them on real-time platforms to validate tion algorithms along with the reason behind our selection is
their applicability. To the best of our knowledge, this is the given.
first paper reviewing the MOO problems with the formulation Centralized optimization method: In this optimization
of the prioritized sum of objective functions. method, there is a central node that has access to the entire
This paper mainly contains the following contributions: agents’ information. In each iteration of the optimization
1) It provides a detailed and comprehensive survey of the process, all agents’ information is sent to this central node.
recent MOO methods applied to MASs, along with The objective functions are predefined and fixed, and the
their applications. central node computes the overall objective function based
2) It classifies the existing MOO algorithms in terms of on the information it has received from agents. Then, it sends
formulating the problem into the sum of objective func- the result back to agents, notifying all or some of them of
tions and the prioritized sum of objective functions. The the optimal selection. Thus, no local-to-local communication
first category is further classified into different classes between agents exists, and agents cannot make decisions
based on the challenges addressed in the literature for individually. High communication is required in this method
2 VOLUME 11, 2023
Shokoufeh Naderi et al.: A Mapping and State-of-the-Art Survey on Multi-Objective Optimization Methods for Multi-Agent Systems

as all the agents need to be in contact with the central node. MOO techniques have been studied since the 1970s, and
As a disadvantage, if the central node gets out of reach or several comprehensive review papers on MOO methods have
the communication lines fail, the whole system may fail. been published since the 1980s. There are many papers re-
Furthermore, the computational burden is considerable [17]. viewing MOO methods for specific applications, such as
Decentralized optimization method: Decentralized op- wireless sensor networks, chemical process engineering, en-
timization strategies have been proposed to overcome the ergy forecasting, and so forth [23]–[27]. A more general work
limitations of centralized methods. In these approaches, the is [12], in which authors first conduct a survey on how to
agents locally exchange information with each other to mu- model multiple objectives to solve MOO problems. Then,
tually improve their knowledge of the environment, and they categorize the papers published between 2000 and 2016
the decision-making power is distributed across all agents. into three classes based on the characteristics of objectives,
Therefore, the agents can decide individually based on their i.e., whether the work deals with system objectives for global
previous information and the previous information of their welfare and/or individual objectives for individual welfare.
neighbors. An example is an estimation scenario for a group The publication [17] is another comprehensive review paper
of robots that need to localize a target given sensor measure- that classifies the distributed MOO algorithms, which are a
ments. They can compute the optimal solution individually, subcategory of decentralized MOO methods, into discrete-
given only their own observations (centralized method), or time and continuous-time algorithms. The focus of this paper
act in a networked system of robots. In the latter case, they is on the applications of distributed MOO algorithms to the
compute the optimal solution given all robots’ observations coordination of distributed energy resources. More recently,
(decentralized method) [18]. Generally, agents may have dif- a review paper presented in [28] classified distributed MOO
ferent priorities in this approach, like travel time, stop time, algorithms based on the constraints of the problem.
or fuel consumption in a fleet of self-driving cars [6]. Considering review papers on MASs, [29] presents an
The majority of recent MOO algorithms fall into the second overview that enables readers to gain a thorough knowledge
category because of the advantages they offer over the first of MASs along with their applications and challenges. Other
one. For instance, the convergence rate of decentralized meth- useful background material on MASs, their control and coor-
ods may be faster, enhancing the efficiency of the fleet [19]. dination schemes can be found in [30]–[33].
Moreover, the autonomy of the agents may make the fleet For those interested in applying different MOO algorithms
more flexible and more robust compared to centralized meth- on real robots, [34] presents a comprehensive survey on dif-
ods [20]. A small change in the network can, for example, ferent platforms used in swarm robotics. This paper mainly
necessitate a redesign of the centralized algorithm [21]. More explores the reason behind the lack of exploitation of swarm
importantly, recent advances in technology and having easy robotic systems in real-world applications. The publications
access to computing units, such as mobile phones or sensors [35], [36] are other review papers discussing different re-
equipped with a central processing unit, have put increasing search platforms and hardware architecture to investigate the
attention on decentralized optimization methods [22]. application of swarm algorithms to robots.
Among the papers surveying MOO methods, [18] is the
In particular, our paper aims to survey papers published
only work that specifically explores the applications of dis-
recently (we survey publications after 2000, but we mainly
tributed optimization methods (not decentralized optimiza-
focus on the most recent ones). Thus, only decentralized
tion methods) to MRS. This publication categorizes the dis-
optimization algorithms are included in the paper.
tributed optimization algorithms into three classes, named
The rest of the paper is organized as follows: Sect. II
distributed gradient descent, distributed sequential convex
presents the existing survey papers on the subject. Sect. III
programming, and alternating direction method of multipliers
provides some background on graph theory, MOO problems,
(ADMM). Then, it compares the performance of these algo-
and Pareto optimality. Problems with the formulation of the
rithms for applications in robotics. Our survey differs from
sum of objective functions and the prioritized sum of objec-
[18], in the sense that it studies all the MOO methods applied
tive functions are reviewed in Sect. IV and Sect. V, respec-
to MASs, and not only distributed optimization methods. In
tively. Sect. VI reviews applications of MOO in MRSs and
other words, we survey problems with both the sum of ob-
publications with the practical implementation of MOO on
jective functions and the prioritized sum of objective function
MRSs. We propose some future research directions in Sect.
formulation. The classification, in our paper, also differs from
VII, and finally, Sect. VIII concludes the paper.
that in [18]. Additionally, we review papers with a practical
experiment on real robots, that is not included in [18].
II. EXISTING SURVEYS Thus, our paper is the first review of the MOO problems
In this section, we will mention some review papers on with the problem formulation of the prioritized sum of objec-
MOO, MASs, and MRSs. The aim is first to explain the tive functions and the first review of the practical implemen-
difference between our survey and the existing ones, which tation of MOO algorithms on real-time robots. Although the
clarifies the necessity of our paper. Secondly, we will mention number of publications conducting research on the mentioned
some other survey papers that can provide readers with a areas is low, having a detailed review of the topic is essential
background or other useful materials on the topic. to identify the gaps and highlight the need for more research
VOLUME 11, 2023 3
 Shokoufeh Naderi et al.: A Mapping and State-of-the-Art Survey on Multi-Objective Optimization Methods for Multi-Agent Systems

on the topic. known only to agent i. X = {X1 , X2 , ..., Xn } is a vector of

the local constraint set. g(x) ∈ Rm and h(x) ∈ Rp are
III. PRELIMINARIES vectors of global inequality and equality constraint functions,
In this section, we introduce some background on graph respectively, imposed on decision variables, where m and p
theory, MOO, and Pareto front (see [37]–[39]) to explain are the numbers of global inequality and equality constraint
the problem and the terms we use to make the paper more functions, respectively. A ∈ Rn represents the feasible region
understandable for readers. determined by the constraints, and any point x ∈ A gives a
feasible solution.
A. GRAPH THEORY Note that not all algorithms proposed to solve (1) consider
MASs and their interactions are mostly modeled using the problem to have all the local and global constraints. Some
graph theory. More specifically, nodes in a graph represent the may only have local constraints, some may only have equality
agents, and edges represent communications and interactions. or inequality global constraints, and others may have both
In this section, we review basic graph theory definitions. local and global constraints.
More details can be found in [37]. The weight matrix of a MOO problem, W , is defined as
Let G = (V, E) be a graph of order n where V = {1, ..., n} an n × n matrix, with the scalars aij (k) being the elements of
is the set of nodes and E ⊆ V × V is the set of edges. A the matrix (i, j ∈ {1, 2, ..., n}), while aij (k) is the weight that
directed edge between node i to node j is denoted by (i, j) ∈ E, agent i assigns to the information received from a neighboring
meaning that nodes i and j are adjacent or that agent i is agent j at the time k.
a neighbor of agent j and communicates with it. The set of W is non-negative if all of its entries (aij ) are non-negative.
neighbors of node i is denoted by Ni = {j ∈ V : (j, i) ∈ E}. W is positive if all of its entries (aij ) are positive. A non-
The number of agents with whom agent i communicates is the negative matrix W is stochastic or column stochastic if the
degree of agent i, denoted by deg(i). A graph is undirected if entries of each column sum to 1. A non-negative matrix W
and only if (i, j) ∈ E implies (j, i) ∈ E. For simplification is row stochastic if the entries of each row sum to 1. A non-
purposes, it is assumed that the graph does not have any self- negative matrix W is doubly stochastic if the entries of each
loop for all i ∈ V, i.e., (i, i) ∈/ E although all agents have row and column sum to 1.
access to their own information. An undirected graph G is The aim of the MOO problem is to minimize all the
fully connected or complete if there is an edge between every objective functions simultaneously. However, the objective
pair of nodes [37]. A directed path is a sequence of ordered functions are usually in conflict with each other, i.e., an
edges in the form of (i1 , i2 ), (i2 , i3 ), ..., where ij ∈ V. The improvement in one of them may result in a lower fitness
node j is reachable from node i if a directed path exists from in others. So the problem is to minimize different objective
node i to node j. A directed graph G is strongly connected functions in the best way possible [39].
if there is a directed path from every node to every other
node [37]. For directed graphs, the out-degree of node i is the C. PARETO FRONT
number of other nodes to whom agent i sends information. Solving a MOO problem is generally associated with build-
Let B(G) = [bij ] ∈ Rn×n be the adjacency matrix associated ing a Pareto front. In this section, some basic information
with the graph G, where bij is the entry in the jth row and ith and the definition of Pareto optimality and Pareto front are
column, defined as: discussed.
( After solving the MOO problem, many feasible solutions
i 1 if (i, j) ∈ E are found. However, only a small subset of these solutions
bj =
0 otherwise is interesting. This subset only includes solutions that have a
domination relation with other ones. In other words, none of
them can be improved without the deterioration of at least one
other solution.
B. MULTI-OBJECTIVE OPTIMIZATION Mathematically, we say a solution x1 ∈ A dominates an-
The problem of MOO, which was originally proposed in other solution x2 ∈ A, if fi (x1 ) ≤ fi (x2 ) for all i = 1, 2, ..., n
[40], is defined as follows: and fi (x1 ) < fi (x2 ) for at least one i.
n
X The set of all Pareto optimal solutions is called the Pareto
min f (x) = fi (x) , (1) front. Fig. 1 illustrates the definition of globally, locally,
x∈A
i=1 and weakly Pareto optimal. This figure aims to make the
s.t. x ∈ X, definition of optimality in Pareto sense more straightforward.
g(x) ≤ 0 , A solution x∗ is called (globally) Pareto optimal or
(strongly) non-dominated, if and only if there exists no so-
h(x) = 0 ,
lution x ∈ A such that fi (x) ≤ fi (x∗ ) for all i = 1, 2, ..., n
n
x ∈ R is a vector of n decision variables, where n represents and fi (x) < fi (x∗ ) for at least one i.
the number of agents. f (x) is the global objective function. A solution x∗ is called locally Pareto optimal, if and only
fi , for all i ∈ {1, 2, ..., n}, is the local objective function if there exists δ > 0 such that there is no solution x which
4 VOLUME 11, 2023
Shokoufeh Naderi et al.: A Mapping and State-of-the-Art Survey on Multi-Objective Optimization Methods for Multi-Agent Systems

this method mostly develop the math behind the algorithm

𝑓2 without application or simulation results.
Globally Pareto optimal The first work introducing the DSG algorithm and an-
Locally Pareto optimal
Weakly Pareto optimal
alyzing its convergence for problems with locally convex
and potentially non-smooth objective functions is [43]. In
the proposed algorithm, agents estimate the solution and
communicate it to other agents asynchronously and over a
Feasible region time-varying network. More specifically, in each iteration
of the algorithm, agents update their estimates based on the
information received from their immediate neighbors and
the subgradient information of their own cost function. In
other words, a combination of a local subgradient step and a
consensus step (weighted average with immediate neighbors’
𝑓1 variables) is done in every iteration of the algorithm.
The framework the authors of [43] used for the analysis of
FIGURE 1. Optimality in the Pareto sense [41], [42] distributed computation models was developed in [40]. This
is also the case for most of the other publications developing
the DSG algorithm. So, they first introduce a transition matrix
dominates the solution x∗ with x ∈ A ∩ B(x∗ , δ), where and analyze its convergence behavior. Then, they use the
B(x∗ , δ) represents a bowl of center x∗ ∈ A and of radius convergence rate of the transition matrix in the analysis of
δ, i.e., B(x∗ , δ) = {x ∈ Rn | ∥ (x − x∗ ) ∥< δ} [39]. If the subgradient method.
x is globally Pareto optimal, it is also locally Pareto optimal; A constant step-size is used in this algorithm because the
however, the reverse does not always hold. authors want it to converge to the optimal solution fast. This
causes a constant error term in the estimate of the solution that
A solution x∗ is called weakly Pareto optimal, if and only
is proportional to the step-size value. Thus, with a fixed step-
if there exists no solution x ∈ A such that fi (x) < fi (x∗ ) for
size, the DSG algorithm only converges to a neighborhood of
all i = 1, 2, ..., n.
the optimal solution [45]. Using diminishing step-size in the
algorithm is also possible, but slows down the convergence
IV. PROBLEM FORMULATION OF SUM OF OBJECTIVE
rate.
FUNCTIONS
Different extensions of DSG methods have been studied
In this section, an overview of some existing decentral- in the literature, including locally constrained problems and
ized MOO methods along with different extensions of these problems with time-varying graphs or uncoordinated step
methods is provided. We classify the algorithms on the basis sizes. In the next paragraphs, we will discuss these extensions
of the reformulation of the problem, i.e., do they solve the with an overview given in Fig. 2.
original problem with respect to primal variables, or do they For diminishing step-sizes, [44] is the first work to develop
reformulate it as the Lagrangian function to be solved with the DSG algorithm for locally constrained MOO problems
respect to either both primal and dual variables or only dual and over time-varying networks. Each agent in this algorithm
variables. The reason behind choosing this classification is performs the DSG algorithm, then takes the projection of their
that the methods falling into the same classes usually show estimate on its individual constraint set. Under the assump-
similar characteristics. This feature, along with having the tions of the doubly stochastic weight matrix and strongly
same formulation, makes it more understandable for readers. connected graph, and with an appropriately selected step
Subsequently, this results in a better choice of method for size (converging to zero fast enough), the authors proved
researchers who want to explore the subject. that the algorithm converges to the optimal solution in both
cases of having identical or non-identical local constraints.
A. PRIMAL METHODS Other DSG algorithms designed for locally constrained MOO
In this section, we review the most known primal methods problems are presented in [47], [52], [55], [62], [63]. In [47],
for solving the MOO problem. [52], [63], the assumption of having doubly-stochastic weight
Distributed subgradient (DSG) method: DSG algorithm matrices have been reduced to only row-stochastic matrices.
minimizes the problem by averaging the local gradient de- Doubly-stochasticity is usually assumed in many works since
scent computed by each agent. The algorithm is capable of it simplifies the proofs of convergence at the cost of restricting
converging to the optimal solution on non-differentiable con- communication among agents [6].
vex functions with subgradients. Moreover, there are many Overall, the majority of existing constrained DSG algo-
extensions of this type of algorithm, offering fast convergence rithms are designed for locally constrained MOO problems
rates. These characteristics have made it ideally suitable for without equality or inequality global constraints. However,
a wide range of applications, such as resource allocation there are some publications that combined DSG methods
[43] and motion planning [44]. However, works presenting with other methods to present a solution to the globally
VOLUME 11, 2023 5
 Shokoufeh Naderi et al.: A Mapping and State-of-the-Art Survey on Multi-Objective Optimization Methods for Multi-Agent Systems

Limited
communication [57]

Fixed step-size Transmission

algorithms [59]–[61] noise [55], [56]

Communication
Proximal DSG [58] delay [54]
Event-triggered
Communication
Other extensions communication
problems
[46], [47]

Different extensions
of DSG algorithm

Locally constrained Directed time- Uncoordinated step-

problems varying graphs sizes [48], [49]

Subgradient-push
Subgradient-push
method [50], [51]
method [50]–[53]

Other methods with

row-stochastic weight
matrices [52], [53]

FIGURE 2. Extensions of DSG methods

constraint MOO problem. For problems with global inequal- Moreover, the cost functions need to be convex (not strongly
√
ity constraints, for instance, the publication [64] proposed convex), and the algorithm converges at a rate of O(ln t/ t),
a combination of a stochastic subgradient method with a where t is the number of iterations of the algorithm. The same
dynamic consensus tracking method for convex, non-smooth authors developed the subgradient-push method for strongly
cost functions, and non-affine constraint functions. Another convex cost function too [51]. With this stronger assumption,
work developing a primal-dual (PD) subgradient method for they proved the convergence of their algorithms with a rate
problems with global equality and inequality constraints is of O(ln t/t) even if agents’ information about their own cost
[65] (PD methods will be reviewed in the next subsection). function is noisy. In both [50] and [51], the weight matrices
For directed time-varying graphs, most existing algorithms need to be column stochastic, which necessitates each agent
are based on the push-sum method. It is a protocol for node to know its out-degree. To eliminate this requirement, some
interaction that allows nodes to compute aggregates and av- algorithms have been proposed in the literature that only
erage in the graph with directed communication links [50]1 . require row-stochastic weight matrices. One such algorithm
The reason for using this method is that not all directed is proposed in [52] for convex and Lipschitz continuous
graphs accept doubly stochastic weight matrices. On the other cost functions over time-varying general unbalanced directed
hand, the push-sum method reduces the assumption of having graphs. Under the requirement of having uniformly jointly
doubly stochastic weight matrices to only column-stochastic strongly connected directed graphs and row-stochastic weight
ones. Authors of [50] were the first to propose a subgra- matrices, the authors proved that the algorithm asymptotically
dient protocol for directed time-varying graphs, termed the converges to the exact optimal solution. Another algorithm
subgradient-push. The proposed algorithm does not require is proposed in [53], that also relaxes the compactness as-
any knowledge of the graph sequence or the number of agents sumption of the constraint sets3 . The authors removed the
but every agent has to know its out-degree2 at all times. assumption of boundedness of constraint sets and replaced it
with a milder assumption, called the regularity assumption,
1 The convergence of this protocol is illustrated in [66]–[68] which can be applied to problems with non-identical and
2 The out-degree of an agent i is the number of agents to them agent i sends
information. 3A subset S ⊂ R is compact if S is closed and bounded.

6 VOLUME 11, 2023

Shokoufeh Naderi et al.: A Mapping and State-of-the-Art Survey on Multi-Objective Optimization Methods for Multi-Agent Systems

unbounded constraint sets. with Kuhn-Tucker multipliers µj ≥ 0 for all j = 1, ..., m.

Although most of the existing DSG algorithms have been By introducing these multipliers, an equivalent unconstrained
developed for coordinated step-sizes, i.e., when the step- problem can be solved instead of (1).
size is the same for all agents in the network, there are Different papers have proposed different algorithms to find
some papers investigating uncoordinated step-size [48], [49]. the saddle point of (2). In [74], Uzawa’s algorithm [72]
The uncoordinated step-size is practical in cases where the is used, which is the first cloud-based optimization algo-
communication is restricted or the communication links are rithm presented for this purpose. The authors named it a
not always reliable. When having an uncoordinated step-size, quasi-decentralized approach because the computation in this
each agent can update its estimates independently of other method is distributed between the cloud and the agents. How-
agents. ever, It can still be considered a centralized method as the al-
Other extensions of DSG are also available in the literature, gorithm cannot be executed if the cloud is missing. Moreover,
such as Proximal DSG [58], but apart from all the extensions the agents do not have any communication with each other
mentioned so far, there are some other completely different in the proposed architecture. Another work in this direction,
extensions of DSG algorithms that use fixed step-size in their which assumes local communication between agents in the
algorithms and can achieve fast convergence to the exact opti- presence of the central cloud and mild smoothness condi-
mal solution. In other words, they converge much faster than tions imposed on the objective and constraints functions4 ,
DSG algorithms, with the same computational complexity is [75]. Here, communication delays with the cloud have
[69]. These extensions include the DIGing algorithm [59], been accounted for. In this work, the problem of finding the
EXact firsT-ordeR Algorithm (EXTRA) [60], and Network saddle point of (2) is restated as a variational inequality. Then,
InDependent Step-size algorithm (NIDS) [61]. Indeed, these a fixed Tikhonov regularization [76] with respect to both
methods can be reinterpreted as PD methods (which will be primal and dual variables is used to regularize the Lagrangian
investigated in detail in Subsect. IV-B) while not explicitly function. The authors present an algorithm that guarantees the
introducing dual variables. For instance, see [70], [71] to find closeness of the solution to the saddle point of the unregu-
the interpretation of EXTRA as a PD method. larized Lagrangian function if the regularization parameters
are small enough. Another asynchronous algorithm that uses
Overall, primal methods are effective and easy to imple-
a Tikhonov regularized primal-dual method and considers
ment; however, they use decaying step size in their algorithm,
communication delays without a uniform bound is [77], in
resulting in a slow rate of convergence. With a fixed step size,
which, the convergence rate in both the primal and the dual
they only converge to a neighborhood of the optimal solution
spaces is provided.
[59]. The communication burden is also more in primal al-
gorithms with a decaying step size. Moreover, these methods In all of these three algorithms, the cloud is responsible
are designed for non-constrained or locally constrained MOO for updating the dual variables and the agents are responsible
problems, meaning they do not handle problems with global for updating the system’s primal variables. In the last two
equality or inequality constraints. Even algorithms designed ones, an inseparable global cost is imposed on agents apart
for non-constrained or locally constrained MOO problems from the local costs f (x), which depends on the states of all
focus mainly on formal proofs of performance rather than agents. Assuming this inseparability in constraints appears
applications, experiments, or simulations. To address these more practical in general cases as decision variables are some-
issues, many methods have been proposed that use fixed step times strongly connected, and no other algorithm can work
size, including PD methods. PD methods use Lagrangian effectively in this case.
functions and work with them instead of the original problem. Some problems, such as the Credit Assignment Problem
This way, the constrained problem is replaced with a non- [78], may be solved more efficiently by centralized methods
constrained problem capable of converging to the exact opti- [75], and that is the reason for using a hybrid architecture
mal solution while enjoying the benefits of a fixed step-size (combination of the centralized and the decentralized meth-
algorithm. Apart from that, the applicability of PD methods in ods) in the literature. However, these algorithms are only
real-time has been proved in the literature, as we will discuss useful when having access to a central computer with large
in Sect. VI. processing power that can communicate with all the agents.
Otherwise, there might be a time cost.
Other PD algorithms without centralized dual updates are
B. PD METHODS
also available in the literature [79]–[83]. In particular, in
In the PD method, which was first introduced by Uzawa [81], a PD algorithm is presented under the assumption
[72], the constrained problem of (1) with only convex global of a strongly-concave Lagrangian function, which is asyn-
inequality constraint functions, g(x), and strictly convex chronous in primal communication and updates and in dis-
f (x) reduces to finding the saddle point of the following tributed dual updates. This algorithm is designed for mini-
Lagrangian, based on the Kuhn-Tucker Theorem [73]: mizing the single-objective optimization problem, but as the

4 It is assumed that objective and constraints functions are convex and

L(x, µ) = f (x) + µT g(x) (2) continuously differentiable.

VOLUME 11, 2023 7

 Shokoufeh Naderi et al.: A Mapping and State-of-the-Art Survey on Multi-Objective Optimization Methods for Multi-Agent Systems

authors assume that the constraint set, X, can be decom- • In a Flexible Primal-Dual framework (abbreviated as
posed into X = X1 × ... × Xn , their algorithm can also FlexPD), developed in [89], an arbitrary finite number
be applied for a MAS. In this publication, the Lagrangian of gradient descent steps is done to update the primal
function is regularized only with respect to µ, and instead variable, without presenting a theoretical analysis of the
of regularization with respect to x, the authors imposed the optimal choice of the number of steps, and a gradient
assumption of diagonal dominance on the Hessian matrix ascent step to update the dual variable. The linear con-
of the Lagrangian function. This is the first publication de- vergence of the algorithm to the exact optimal solution
veloping asynchronous PD variable updates for constrained, is established for strongly convex and Lipschitz gradient
convex problems of a general form. However, the regular- objective functions, and two other extensions of the al-
ization for convergence guarantee causes unavoidable but gorithm are also proposed to reduce the communication
bounded convergence errors. Thus, developing algorithms and computational cost.
whose convergence does not depend upon system parameters • In [90] a gradient descent is carried out to update the
or regularization is an open research question in this direction primal variable, and a gradient ascent to update the dual
and can eliminate convergence errors. variable. This is the first work establishing exponential
As stated earlier, the aim of the reformulation of the prob- convergence without strong convexity. The convergence
lem using the Lagrangian function is to solve it through a analysis in this publication is based on a Lyapunov
fixed step-size algorithm, which can converge to the exact op- function for the continuous-time algorithms; however,
timal solution. However, there exist some convergence errors the authors also present the discrete counterpart of the al-
in the algorithms proposed by references mentioned so far. gorithm by a discretization method based on Euler’s ap-
This issue has been addressed by proposing an augmented proximation and establish the exponential convergence
Lagrangian (AL)-based PD method and ADMM. of the discrete-time algorithm.
AL-based PD method: AL-based PD method, also known As mentioned, AL-based PD methods can handle generic
as the augmented Lagrangian method (ALM), is a PD method problems, including problems with non-smooth objective
applicable to problems with equality constraints. In this functions or problems with convex objective functions (not
method, the reformulation of (1) can be done through an strongly-convex objective functions). They also have strong
AL function formed by adding a penalty parameter to the convergence guarantees. However, they are not applicable to
Lagrangian function: problems with non-convex objective functions. For a more
ρ
L(x, µ) = f (x) + ||g(x)||2 + µT g(x) (3) detailed overview of AL-based PD methods and their com-
2 plexity, we refer readers to [71]
where µ is the dual variable, and ρ ≥ 0 is the AL penalty ADMM: ADMM is an AL-based method, in which, the
parameter. By (3), the task of solving (1) is converted into primal variables are split into M subsets (M ≥ 2) 5 . Then,
solving a sequence of easier unconstrained problems with re- instead of minimizing all the primal variables once in every
spect to x, where the influence of the constraints is controlled iteration, as is done in ALM, the M minimization steps is
through the dual variable, µ. done in every iteration of the algorithm (one for every subset).
ALM was first introduced in [84], but it has gained in-
This way, one can benefit from the properties of different
creasing attention recently because it can be applied to large-
sub-problems individually. In fact, the main reason behind
scale problems and because of its flexibility towards how
drawing much attention to ADMM is that when applying to
primal problems can be solved. In other words, there are
recent applications mentioned above, the sub-problems are
a lot of different methods to perform the primal and dual
easy to solve [95]. A non-convex problem, for an instance,
updates [71]. Some examples are as follows, with an overview
can be divided into some sub-problems with only one non-
presented in Table 1:
convex sub-problem that is easier to be solved that the first
• In a multi-step dual accelerated (MSDA) method, first
non-convex problem.
introduced in [85] for smooth and strongly-convex ob-
Generally, ADMM can converge to the exact solution, and
jective functions, exact minimization of Lagrangian
its convergence rate has been proven [96]–[100]. However,
function is used to update the primal variables, and
many researchers have proposed inexact ADMM algorithms
Chebyshev acceleration is used to update dual variables.
[93], [101] to reduce the computational cost. The inexact al-
This work was then developed in [86], [87] for other set-
gorithm has attracted much attention since there is no need for
tings, such as non-smooth objective functions. However,
solutions with extremely high accuracy in recent applications,
these methods may be inefficient in practice, because
such as those arising in artificial intelligence (AI) and big
they require exact minimization at every iteration, which
data. The inexact algorithm is usually achieved by a linear
incurs high computational cost [89].
[102] or a quadratic [103] approximation of the objective
• Some works [82], [88], [91] proposed inexact minimiza-
function in the primal variable update.
tion of primal and dual variables. In [88], for instance, an
inexact Nesterov gradient method is used to update the 5 Different methods can be used to split the primal variables, such as
primal variables. The computational complexity of these Douglas–Rachford splitting method [93] and Peaceman-Rachford splitting
inexact methods is studied in [92]. method [94].

8 VOLUME 11, 2023

Shokoufeh Naderi et al.: A Mapping and State-of-the-Art Survey on Multi-Objective Optimization Methods for Multi-Agent Systems

TABLE 1. An overview of AL-based PD methods

Algorithm References Exact solution Primal update method

MSDA [85]–[87] ✓ Exact minimization of Lagrangian func-

tion
- [88] ✘ Nesterov gradient method
FlexPD [89] ✓ Multiple gradient descent steps
- [90] ✓ Gradient descent

Another extension of ADMM, driven by the applications, networks (but under certain assumptions, including a row-
such as noisy color image restoration and asset allocation statistic weight matrix). They prove the convergence of the
[104], is the case where one of the sub-problems is non- algorithm when the exact gradients or only noisy gradient
convex [104]–[107]. These works often establish the conver- information is known to agents. The graph is assumed to
gence using Kurdyka–Lojasiewicz inequality [108] or similar be undirected in this publication, but they use a directed
properties. graph with different edges from the original ones to represent
There are also other extensions of ADMM that can handle it. This would be helpful in simplifying the definition of
different scenarios. These scenarios include cases with syn- constraints.
chronous or asynchronous algorithms, fixed or time-varying There are also some algorithms designed for ADMM
networks, fixed or time-varying cost functions, smooth or with time-varying cost functions. In [115], a dual-regularized
non-smooth cost functions, and directed or undirected net- ADMM is proposed, that is applicable to problems with both
works. In the next paragraphs of this subsection, we will static and time-varying, but smooth and strongly convex cost
discuss these extensions. functions. The idea of applying regularization to the dual
Some researchers investigated asynchronous algorithms variable comes from the fact that regularization improves the
because, in real-world implementations, some agents may fail convergence properties. By doing that, the cost function of
to share their information with others on time. Subsequently, the dual problem needs to be ϵ-strongly convex instead of
this failure will cause an immediate halt in the synchronous strongly convex (ϵ > 0). They show, by numerical results,
algorithm [109]. However, asynchronous algorithms are more that although there is a bounded regularization error, their
robust to these failures. In [94], a modified version of a method can outperform inexact gradient-based methods. For
relaxed ADMM is proposed while the agents are not syn- time-varying cost functions that contain a non-smooth part,
chronized among themselves. The relaxed ADMM is defined the publication [58] proposed an inexact algorithm. This
as the dual of Peaceman–Rachford splitting instead of the algorithm considers a source of additive noise, representing
augmented Lagrangian method. It offers more flexibility in communication noise or quantization.
obtaining better convergence properties, while on the other Very recently, researchers have developed ADMM algo-
hand, it increases the complexity of the updating equations. rithms over directed graphs [116]–[119] since, in practice,
With strongly convex and twice differentiable local objective not all communication channels are necessarily bidirectional.
functions, the authors of this work prove that the expectation In some cases, for instance, different agents have different
of the error converges to a neighborhood of the optimal so- capabilities of broadcasting messages, as a result of which,
lution. However, no form of delay is allowed meaning agents communication links between them are necessarily unidirec-
must perform their local computations using the most updated tional. The first work in this direction is [117]. The authors
information received from their neighbors. For non-convex of this publication proposed a directed-distributed ADMM
objective functions, an asynchronous distributed ADMM is algorithm. Under the assumptions of convex and not necessar-
proposed in [110], with detailed proofs of theorems in [111], ily differentiable objective functions and column-stochastic
[112], that also considers delay in the network. However, weighted matrix, they show that their proposed algorithm
the delay needs to be bounded to ensure convergence, and converges at a rate of O(1/t), with t being the iteration
the algorithm is implemented in a master-slave mode. An- number. Another algorithm working over directed networks
other asynchronous algorithm that converges to the exact was proposed in [119]. This is the first asynchronous algo-
solution under the assumption of convex and possibly non- rithm over directed graphs, converging at the same rate as the
differentiable objective functions is [113]. The communica- algorithm in [117]. However, there are some assumptions in
tions in this paper are subject to possibly unbounded delays. this algorithm, such as known graph diameter6 to all nodes.
Apart from the extensions mentioned so far, there are also
ADMM algorithms have also been studied over time-
other extensions of ADMM, such as accelerated [120], adap-
varying networks since in practical applications, communi-
cation links may be lost unexpectedly. In [114], as an ex- 6 The diameter of a graph is the maximum distance between every pair of
ample, the authors developed an algorithm with a possibly vertices in the graph. This distance is calculated as the number of edges in
stochastic gradient that works for both static and time-varying the shortest path connecting two vertices [37].

VOLUME 11, 2023 9

 Shokoufeh Naderi et al.: A Mapping and State-of-the-Art Survey on Multi-Objective Optimization Methods for Multi-Agent Systems

tive [121], and distributed linearized ADMM [122]. We refer One of the works in this direction is [130], which solves the
interested readers in the subject to other survey papers that problem of the prioritized sum of objective functions, with
specifically review ADMM methods. Some examples include local constraints (x1 = x2 = ... = xn ) and the priorities
[123], [124] and [125], that review the classical ADMM, given as the left eigenvector of the network’s weight matrix.
where M = 2. And, a recent paper, [95], reviews all ADMM The underlying network in this work is a directed unbal-
methods, including the cases where M > 2. anced random network. Under the assumptions of having sub-
Overall, PD methods can converge to the exact optimal differentiable and uniformly Lipschitz continuous objective
solution with provable rates and can handle generic problems. functions, the authors showed the convergence of their algo-
For decentralized static optimization, i.e., when the objective rithm to a Pareto optimal solution.
functions are not time-varying, it is discussed in [69] that Another work that presented a protocol for the mentioned
PD methods are superior to the primal methods. However, problem over an unbalanced but undirected and strongly con-
for decentralized dynamic optimization, this is not always nected network is [131]. Apart from the local constraints of
true. On the other hand, PD methods can only be applied to having a common xi to the network in the former work (
problems with a simple structure of the objective function so [130]), this latter one also considers equality and inequality
that the primal and dual updates can be done efficiently [126]. global constraints in the problem. The proposed protocol
In problems with the formulation of the sum of objective is based on the exact penalty method [132]. However, they
functions, that were reviewed in this section, all the objec- replace the constant penalty parameter with a sequence. This
tives are prioritized equally. In practice, however, agents in helps guarantee the consensus and the convergence of the se-
a system may have various priorities for different objectives. quence. Here, the priorities assigned to the objective functions
In a fleet of self-driving cars, for instance, objectives can be are again the left eigenvector of the graph’s weight matrix.
travel time, stop time, or fuel consumption, and each car may Although the problem solved in [130] and [131] is formu-
prioritize them differently than others. Nevertheless, most lated as the prioritized sum of objective functions, the agents
of the existing literature formulates the MOO problem as a do not have any control over giving different priorities to
problem of an unprioritized sum of objective functions, which the objective functions for other agents. Indeed, the weights
restricts the control that agents have on their priorities. In the are given as the left eigenvector of the network’s weight
next section, we will review publications proposing a solution matrix. Therefore, the proposed algorithm does not achieve
to the problem formulation of the prioritized sum of objective the desired autonomy of the agents, and the agents cannot
functions with the aim of identifying the gaps in this new line individually prioritize the objectives.
of research. This issue is addressed in [6], where an agent’s priorities
given to other agents’ objective functions are considered in
V. PROBLEM FORMULATION OF THE PRIORITIZED SUM the update rule of the algorithm. That is, the priority that
OF OBJECTIVE FUNCTIONS each agent assigns to the information received from the neigh-
All methods reviewed so far consider the MOO problem to boring agents affects its decision variable. Consequently, the
be unprioritized, meaning that agents cannot individually pri- agents are given increased flexibility in their choices. This
oritize information received from other agents in the network. allows them to explore the Pareto front of the problem and
The MOO problem with the formulation of the prioritized find different optimal solutions along the Pareto front. In
sum of objective functions is defined as follows: other words, agents achieve optimums that conform to their
needs and preferences. The decision variables, in [6], are
n
X locally constrained to lie in a set, and this constraint set is
min f (x) = wi fi (x) , (4) the same for all agents. An example would be a case where
x∈A
i=1
agents are robots with the same range of motion. The weight
s.t. x ∈ X, matrix in the algorithm is non-doubly stochastic, the graph is
g(x) ≤ 0 , connected and undirected, and the step size is assumed to be
h(x) = 0 , decreasing. Under these conditions, the authors in [6] proved
the convergence of their proposed algorithm and provided
where wi is the priority assigned to fi . the convergence rate. They also showed how the agents’
In this section, we will conduct a review of papers propos- initial choices of priorities affect the long-run behavior of the
ing a solution to the problem formulation of the prioritized algorithm.
sum of objective functions. There are only a handful of pub- All of the literature reviewed in this section developed an
lications proposing decentralized algorithms for solving this algorithm over an unbalanced graph as they removed the need
problem. On the other hand, there is a large body of work to have doubly-stochastic weight matrices. The requirement
proposing a method to solve the problem of the prioritized of having a sequence of doubly-stochastic matrices reflects a
sum of objective functions for centralized cases [39], [127]– form of balancedness [50]. By removing it, all agents’ weights
[129]. However, these methods constrain the decision-making assigned to a particular objective function need not sum to
autonomy of agents since the central decision-maker selects one. As a consequence, the applicability of these methods
the solution that best satisfies its priorities. increases since in real-time applications, checking the bal-
10 VOLUME 11, 2023
Shokoufeh Naderi et al.: A Mapping and State-of-the-Art Survey on Multi-Objective Optimization Methods for Multi-Agent Systems

ancedness of the network is not an easy task [133]. Apart from algorithms, real-time running can lead to oscillatory behavior
that, in some applications, communication is costly. There- [141].
fore, it is desired to avoid surplus communication needed to Target tracking: In multi-robot target tracking, teams of
coordinate weights. robots track the position of multiple targets. The robots may
To the best of our knowledge, the publications reviewed be heterogeneous, meaning that they may have different sen-
in this section are the only ones with the formulation of sors, and therefore, measuring various signals. Like with the
the prioritized sum of objective functions, which are mini- path-finding application, there may also be a need to allocate
mal numbers. This means that there is a critical gap in the tasks to robots, which may be easy or difficult depending on
literature on the subject. On the other hand, the need for the number of targets and the way they are distributed.
these methods in the real world is growing. Many real-time In the literature, MOO algorithms are usually combined
applications require agents to have a high degree of flexibility with other methods to track targets. In [142], the ADMM is
in their choices. combined with a Kalman-filter-liked update in the algorithm
to track vehicles. In another work, [143], the ADMM has
VI. APPLICATIONS IN MULTI-ROBOT SYSTEMS been used to design the interactions for multi-robot trajectory
One of our focuses in this review is the applications of planning while the approach is based on nonlinear model
MOO to MRSs. MOO can be applied to many areas in predictive control (MPC).
MRS. Examples of these areas are the military, transportation Formation control: Formation control is the cooperation
systems, agriculture, security and surveillance, search and of a group of robots to complete a task, while they are required
rescue, package delivery, household chores, health care, and to get into and maintain a certain shape. In other words,
elderly care. In the literature, however, only a few of these it is controlling the relative position between agents [144].
areas have been studied. We review the applications of MOO Applications of formation control include landmine removal,
to the mentioned areas in Subsect. VI-A and the practical search and rescue operations in dangerous environments, and
experiments conducted on real hardware in Subsect. VI-B. surveillance [145], [146].
Note that applications reviewed in Subsect. VI-A are only Among the strategies used in the field of robot formation
those that have been studied in the literature. The aim is, first, control, MPC has been widely explored in recent years. In
to highlight the fact that most studies have only focused on the [144], as an instance, an ADMM-based distributed MPC
mathematical development of MOO algorithms rather than is applied to the formation control of multiple underwater
their applications. Secondly, we review the applications of vehicles. In another work, [147], ADMM is again used to
MOO to MRSs to identify methods with proven applicability solve an MPC problem but with an application for the path
to real-time platforms. tracking of unmanned aerial vehicles (UAVs).
From the review in this subsection, it can be concluded that
A. APPLICATIONS OF MOO TO MRS most of the algorithms implemented in multi-robot systems
Path finding (path planning): Multi-agent path finding applications are PD algorithms, especially ADMM. Overall,
(MAPF) is the problem of finding paths for multiple agents ADMM is a more attractive option for problems in MRS
in a given graph such that each agent reaches its goal without since it has a fast convergence speed and is computationally
colliding with other agents [134]. Recent real-world applica- efficient [18].
tions in robotics, AI, and operational research have led to a
growing interest in MAPF. Examples of these applications B. PRACTICAL EXPERIMENTS OF MOO ON MRSS
could be using warehouse robots to move products rapidly The number of works implementing MOO problems on
and safely without human intervention [135] and using multi- multi-robot hardware is minimal. This is, first, because op-
robot systems to spray pesticides in agricultural fields [136]. timization problems arising in robotics problems usually in-
In path-finding, agents are usually assigned to a target and volve non-convex constraints and objective functions, while
have to optimize a cost function to reach their target. The most of the existing MOO methods designed for robotics
cost function can be fuel consumption or completion time. applications, such as target tracking and SLAM [18], consider
In some applications, targets are predefined for each agent the problem to be convex. In our review, the only algorithms
meaning that agents cannot be changed with each other while that are designed for non-convex objective functions are ex-
performing the task. However, others also include the task tensions of ADMM with the possibility of having one non-
allocation process, which increases the agent’s autonomy in convex sub-problem [104]–[107] or non-convex objective
managing the task at hand [137]. Some other works propose functions [110]–[112]. As a result of this limitation, most of
the combination of MOO algorithms with other algorithms to the designed MOO algorithms are not yet fully applicable to
solve the path planning problem [138], [139]. MRS.
Another issue that arises for agents when navigating to find Secondly, there is a limited number of real-time implemen-
the path is collision avoidance, which has been dealt with in tations of MOO algorithms on MRS since these algorithms
some publications [137], [140]. However, centralized opti- often do not consider agents’ communication capabilities.
mization algorithms can better treat the collision avoidance Although there are more papers investigating delays in com-
constraints than decentralized ones since in decentralized munication or limited communication in both DSG and PD
VOLUME 11, 2023 11
 Shokoufeh Naderi et al.: A Mapping and State-of-the-Art Survey on Multi-Objective Optimization Methods for Multi-Agent Systems

algorithms, they do not fully satisfy the limitations of the VII. FUTURE RESEARCH DIRECTIONS
real world. In practice, MRSs usually operate in regions with From this mapping and review survey, we could identify
insufficient communication infrastructure. important research gaps and needs that require researchers’
Similar to the previous section, all the works we found that attention as follows:
report the results of practical experiments of implementing
MOO algorithms on MRSs fall into the category of PD meth- • As the review indicates, the MOO problem with the
ods. Their number is less than a handful, showing the huge formulation of the sum of objective functions has been
gap in the practical implementation of designed algorithms widely investigated in the literature. Admittedly, some
on robotic platforms. In other words, most works on the subcategories have room for improvement, such as de-
topic tend to be highly abstract and focus mainly on the veloping DSG algorithms for globally constrained MOO
mathematical proof of algorithms’ convergence. problems. However, the method has overall been ex-
The authors of [75], which was also mentioned and ex- plored by many researchers.
plained in Subsect. IV-B, applied their proposed algorithm On the other hand, problems with the formulation of the
on a team of 8 Khepera III robots. Khepera is a mobile prioritized sum of objective functions. To our knowl-
robot developed at the Swiss Federal Institute of Technology edge, the publications reviewed in Sect. V are the only
of Lausanne (EPFL) to be used in education and research ones with the formulation of the prioritized sum of ob-
[148]. The results in this publication indicate close agreement jective functions, whose number is very limited.
between the simulation and experiments. This means that there is a huge gap in the literature on the
[149] presented a method of coordination for trajectory subject. In fact, the existing work on MOO gives agents a
optimization of multi-robot motion planning. The method low level of autonomy as they consider agents’ priorities
is based on a distributed nonlinear MPC. The convergence to be fixed and the same for all objective functions. In
of the proposed algorithm is proved using ADMM. In their other words, agents do not have any control over their
algorithm, the authors assume that each robot has an onboard priorities. Whereas, in the real world, many applications
computation unit to communicate with neighboring robots require agents to have a high degree of flexibility in
through a wireless network and solve a local control prob- their choices. Enabling them to prioritize their objectives
lem. They proposed two methods for synchronous and asyn- enhances the decision-making skills of the multi-agent
chronous information exchange between robots. They tested system, making it more robust and autonomous. How-
the asynchronous distributed nonlinear MPC to coordinate a ever, this increased robustness and autonomy come with
team of three small-scale autonomous cars. The vehicles are some disadvantages: As non-doubly stochastic weight
based on a platform presented to replicate the behavior of matrices are often needed in the algorithm, the conver-
full-scale vehicles, named the Waveshare JetRacer Pro AI5 gence analysis is more complex than problems with the
kit [150]. The results in the paper, along with a video of sum of objective functions.
the experiments provided in [151] show that the vehicles can Therefore, in this new line of research that has not been
accomplish the planning goals without collisions safely. explored yet, new algorithms can be developed for dif-
ferent scenarios, such as algorithms over time-varying
Another practical implementation of MOO algorithms is
networks or with directed communication. Other re-
presented in [152]. The application is a multi-robot SLAM.
search directions can be considering non-convex and
Again, the authors use ADMM in the optimization process
globally constrained cost functions.
to guarantee convergence. They propose a method for outlier
• For both problem formulations, practical restrictions
detection between any number of maps. The method is used
have not been usually considered in the designed al-
in cases where multiple robots perform SLAM or one robot
gorithms. Without considering these restrictions, imple-
generates various maps. They implement their method for
menting the proposed methods would be impossible,
experiments on actual data obtained from an office environ-
interfering with the development of the robotics field.
ment by taking four independent sequences of RGB-D images
For instance, most of them often consider the problem
using an Intel RealSense D435 camera. Their results show
to be convex, while optimization problems arising in
that the objects in a map are better aligned.
robotics are usually non-convex. They also do not usu-
Again, a similar conclusion to the previous subsection can ally consider communication problems, such as delays
be drawn. All methods applied to MRS hardware fall into or having limited access to communication tools.
the category of PD methods. Although the number of prac- • Another critical gap in the literature is the practical
tical implementations is low, the results show an agreement implementation of the designed algorithms. While in
between simulation and experiments. In other words, the practice, we need reliable, robust, and safe implemen-
applicability of the PD method in practice is proved. However, tation of designed algorithms, most of the studies in the
the applicability of algorithms proposed to solve the problem literature do not implement their proposed method. Even
formulation of the prioritized sum of objective functions has for the problem formulation of the unprioritized sum of
not been practically proved. objective functions, the designed algorithms have not
been widely implemented in practice, and most of the
12 VOLUME 11, 2023
Shokoufeh Naderi et al.: A Mapping and State-of-the-Art Survey on Multi-Objective Optimization Methods for Multi-Agent Systems

works on the topic focus mainly on the mathematical [6] M. J. Blondin and M. Hale, ‘‘A decentralized multi-objective optimization
proof of convergence. Therefore, the applicability of the algorithm,’’ Journal of Optimization Theory and Applications, vol. 189,
no. 2, pp. 458–485, 2021.
existing MOO methods and their robustness is not well [7] S. C. Cerda-Flores, A. A. Rojas-Punzo, and F. Nápoles-Rivera, ‘‘Applica-
assured. tions of multi-objective optimization to industrial processes: a literature
Indeed, without practical implementations or a testbed, review,’’ Processes, vol. 10, no. 1, p. 133, 2022.
[8] L. A. Márquez-Vega, M. Aguilera-Ruiz, and L. M. Torres-Treviño,
robotic complexities cannot always be accurate the first ‘‘Multi-objective optimization of a quadrotor flock performing target zone
time in simulations. In other words, we need algorithms search,’’ Swarm and Evolutionary Computation, vol. 60, p. 100733, 2021.
to be implemented on practical systems to develop more [9] P. Mannion, S. Devlin, J. Duggan, and E. Howley, ‘‘Reward shaping for
knowledge-based multi-objective multi-agent reinforcement learning,’’
accurate models and subsequently design more efficient The Knowledge Engineering Review, vol. 33, 2018.
algorithms. Therefore, implementing algorithms on real- [10] L. Yin and Z. Sun, ‘‘Multi-layer distributed multi-objective consensus
time platforms will help in going further than simula- algorithm for multi-objective economic dispatch of large-scale multi-area
tions, getting real feedback and data in real settings, and interconnected power systems,’’ Applied Energy, vol. 300, p. 117391,
2021.
redesigning algorithms based on them. That is to say, [11] S. E. Bouzid, Y. Seresstou, K. Raoof, M. N. Omri, M. Mbarki, and
the real feedback will iteratively feed the development C. Dridi, ‘‘Moonga: multi-objective optimization of wireless network
of algorithms. Examples of real feedback could be the approach based on genetic algorithm,’’ IEEE Access, vol. 8, pp. 105 793–
105 814, 2020.
time when resources are limited, when agents fail, when [12] J.-H. Cho, Y. Wang, R. Chen, K. S. Chan, and A. Swami, ‘‘A survey on
communication is noisy, or when robots interact with modeling and optimizing multi-objective systems,’’ IEEE Communica-
humans. tions Surveys & Tutorials, vol. 19, no. 3, pp. 1867–1901, 2017.
[13] R. N. Haksar, O. Shorinwa, P. Washington, and M. Schwager,
• Finally, in the existing practical applications, all the ‘‘Consensus-based admm for task assignment in multi-robot teams,’’ in
robots that are used are identical while in the real world, International Symposium on Robotics Research, 2019.
there is a need for a variety of designs to accomplish [14] A. A. H. Qizilbash, C. Henkel, and S. Mostaghim, ‘‘Ant colony opti-
mization based multi-robot planner for combined task allocation and path
complex tasks successfully. Developing algorithms and finding,’’ in 2020 17th International Conference on Ubiquitous Robots
platforms that can be applied to scenarios with different (UR). IEEE, 2020, pp. 487–493.
kinds of robots, such as UAVs and unmanned ground [15] J. Alonso-Mora, E. Montijano, T. Nägeli, O. Hilliges, M. Schwager, and
vehicles (UGVs), is, therefore a valuable research direc- D. Rus, ‘‘Distributed multi-robot formation control in dynamic environ-
ments,’’ Autonomous Robots, vol. 43, no. 5, pp. 1079–1100, 2019.
tion. [16] M. J. Schuster, C. Brand, H. Hirschmüller, M. Suppa, and M. Beetz,
‘‘Multi-robot 6d graph slam connecting decoupled local reference filters,’’
in 2015 IEEE/RSJ International Conference on Intelligent Robots and
VIII. CONCLUSION Systems (IROS). IEEE, 2015, pp. 5093–5100.
In this survey, we provided a review of MOO methods for [17] T. Yang, X. Yi, J. Wu, Y. Yuan, D. Wu, Z. Meng, Y. Hong, H. Wang,
Z. Lin, and K. H. Johansson, ‘‘A survey of distributed optimization,’’
MASs. By categorizing the existing methods based on the for- Annual Reviews in Control, vol. 47, pp. 278–305, 2019.
mulation of the problem, we discussed the main problem for- [18] T. Halsted, O. Shorinwa, J. Yu, and M. Schwager, ‘‘A survey of dis-
mulations and the distinctions between them. Our taxonomy tributed optimization methods for multi-robot systems,’’ arXiv preprint
arXiv:2103.12840, 2021.
makes it easier to identify the most recent trends in the subject [19] X. Lian, C. Zhang, H. Zhang, C.-J. Hsieh, W. Zhang, and J. Liu, ‘‘Can
and also makes it more understandable for readers to explore decentralized algorithms outperform centralized algorithms? a case study
the subject. We also surveyed the literature on the practical for decentralized parallel stochastic gradient descent,’’ Advances in Neu-
ral Information Processing Systems, vol. 30, 2017.
applications of these methods on MRSs for the first time. [20] R. Dubois, A. Eudes, J. Moras, and V. Frémont, ‘‘Dense decentralized
This highlights the need to implement designed algorithms multi-robot slam based on locally consistent tsdf submaps,’’ in 2020
on real platforms while identifying reliable methods already IEEE/RSJ International Conference on Intelligent Robots and Systems
(IROS). IEEE, 2020, pp. 4862–4869.
applied to real systems. We proposed a new line of research
[21] S. Yang, S. Tan, and J.-X. Xu, ‘‘Consensus based approach for economic
on the subject, i.e., the problem with the formulation of the dispatch problem in a smart grid,’’ IEEE Transactions on Power Systems,
prioritized sum of objective functions. vol. 28, no. 4, pp. 4416–4426, 2013.
[22] G. Zhang and R. Heusdens, ‘‘Distributed optimization using the primal-
dual method of multipliers,’’ IEEE Transactions on Signal and Informa-
REFERENCES tion Processing over Networks, vol. 4, no. 1, pp. 173–187, 2017.
[23] Z. Fei, B. Li, S. Yang, C. Xing, H. Chen, and L. Hanzo, ‘‘A survey of multi-
[1] P. Balaji and D. Srinivasan, ‘‘An introduction to multi-agent systems,’’ in objective optimization in wireless sensor networks: Metrics, algorithms,
Innovations in multi-agent systems and applications-1. Springer, 2010, and open problems,’’ IEEE Communications Surveys & Tutorials, vol. 19,
pp. 1–27. no. 1, pp. 550–586, 2016.
[2] K. Nishi and S. Arai, ‘‘Modeling multi-objectivization mechanism in [24] G. P. Rangaiah, Z. Feng, and A. F. Hoadley, ‘‘Multi-objective optimiza-
multi-agent domain,’’ in 2019 IEEE International Conference on Agents tion applications in chemical process engineering: Tutorial and review,’’
(ICA). IEEE, 2019, pp. 61–64. Processes, vol. 8, no. 5, p. 508, 2020.
[3] R. Rădulescu, P. Mannion, D. M. Roijers, and A. Nowé, ‘‘Multi-objective [25] H. Liu, Y. Li, Z. Duan, and C. Chen, ‘‘A review on multi-objective
multi-agent decision making: a utility-based analysis and survey,’’ Au- optimization framework in wind energy forecasting techniques and ap-
tonomous Agents and Multi-Agent Systems, vol. 34, no. 1, pp. 1–52, 2020. plications,’’ Energy Conversion and Management, vol. 224, p. 113324,
[4] L. N. Asha, A. Dey, N. Yodo, and L. G. Aragon, ‘‘Optimization ap- 2020.
proaches for multiple conflicting objectives in sustainable green supply [26] H. M. Ridha, C. Gomes, H. Hizam, M. Ahmadipour, A. A. Heidari,
chain management,’’ Sustainability, vol. 14, no. 19, p. 12790, 2022. and H. Chen, ‘‘Multi-objective optimization and multi-criteria decision-
[5] X. Zhang, H. Liu, and L. Tu, ‘‘A modified particle swarm optimization making methods for optimal design of standalone photovoltaic system:
for multimodal multi-objective optimization,’’ Engineering Applications A comprehensive review,’’ Renewable and Sustainable Energy Reviews,
of Artificial Intelligence, vol. 95, p. 103905, 2020. vol. 135, p. 110202, 2021.

VOLUME 11, 2023 13

 Shokoufeh Naderi et al.: A Mapping and State-of-the-Art Survey on Multi-Objective Optimization Methods for Multi-Agent Systems

[27] Y. Cui, Z. Geng, Q. Zhu, and Y. Han, ‘‘Multi-objective optimization [52] H. Li, Q. Lü, and T. Huang, ‘‘Distributed projection subgradient algorithm
methods and application in energy saving,’’ Energy, vol. 125, pp. 681– over time-varying general unbalanced directed graphs,’’ IEEE Transac-
704, 2017. tions on Automatic Control, vol. 64, no. 3, pp. 1309–1316, 2018.
[28] Y. Zheng and Q. Liu, ‘‘A review of distributed optimization: Problems, [53] V. S. Mai and E. H. Abed, ‘‘Distributed optimization over directed graphs
models and algorithms,’’ Neurocomputing, 2021. with row stochasticity and constraint regularity,’’ Automatica, vol. 102,
[29] A. Dorri, S. S. Kanhere, and R. Jurdak, ‘‘Multi-agent systems: A survey,’’ pp. 94–104, 2019.
Ieee Access, vol. 6, pp. 28 573–28 593, 2018. [54] T. T. Doan, C. L. Beck, and R. Srikant, ‘‘Convergence rate of distributed
[30] J. Qin, Q. Ma, Y. Shi, and L. Wang, ‘‘Recent advances in consensus of subgradient methods under communication delays,’’ in 2018 Annual
multi-agent systems: A brief survey,’’ IEEE Transactions on Industrial American Control Conference (ACC). IEEE, 2018, pp. 5310–5315.
Electronics, vol. 64, no. 6, pp. 4972–4983, 2016. [55] L. Zhang and S. Liu, ‘‘Projected subgradient based distributed convex
[31] K.-K. Oh, M.-C. Park, and H.-S. Ahn, ‘‘A survey of multi-agent formation optimization with transmission noises,’’ Applied Mathematics and Com-
control,’’ Automatica, vol. 53, pp. 424–440, 2015. putation, vol. 418, p. 126794, 2022.
[32] Y. Cao, W. Yu, W. Ren, and G. Chen, ‘‘An overview of recent progress in [56] J. Zhang, K. You, and T. Başar, ‘‘Distributed discrete-time optimization in
the study of distributed multi-agent coordination,’’ IEEE Transactions on multiagent networks using only sign of relative state,’’ IEEE Transactions
Industrial informatics, vol. 9, no. 1, pp. 427–438, 2012. on Automatic Control, vol. 64, no. 6, pp. 2352–2367, 2018.
[33] J. Cortés and M. Egerstedt, ‘‘Coordinated control of multi-robot systems: [57] T. T. Doan, S. T. Maguluri, and J. Romberg, ‘‘Fast convergence rates
A survey,’’ SICE Journal of Control, Measurement, and System Integra- of distributed subgradient methods with adaptive quantization,’’ IEEE
tion, vol. 10, no. 6, pp. 495–503, 2017. Transactions on Automatic Control, vol. 66, no. 5, pp. 2191–2205, 2020.
[34] N. Nedjah and L. S. Junior, ‘‘Review of methodologies and tasks in swarm [58] N. Bastianello and E. Dall’Anese, ‘‘Distributed and inexact proximal
robotics towards standardization,’’ Swarm and Evolutionary Computa- gradient method for online convex optimization,’’ in 2021 European
tion, vol. 50, p. 100565, 2019. Control Conference (ECC). IEEE, 2021, pp. 2432–2437.
[35] M. Schranz, M. Umlauft, M. Sende, and W. Elmenreich, ‘‘Swarm robotic [59] A. Nedic, A. Olshevsky, and W. Shi, ‘‘Achieving geometric convergence
behaviors and current applications,’’ Frontiers in Robotics and AI, vol. 7, for distributed optimization over time-varying graphs,’’ SIAM Journal on
p. 36, 2020. Optimization, vol. 27, no. 4, pp. 2597–2633, 2017.
[36] G. Seeja et al., ‘‘A survey on swarm robotic modeling, analysis and [60] W. Shi, Q. Ling, G. Wu, and W. Yin, ‘‘Extra: An exact first-order
hardware architecture,’’ Procedia Computer Science, vol. 133, pp. 478– algorithm for decentralized consensus optimization,’’ SIAM Journal on
485, 2018. Optimization, vol. 25, no. 2, pp. 944–966, 2015.
[61] Z. Li, W. Shi, and M. Yan, ‘‘A decentralized proximal-gradient method
[37] C. Godsil and G. F. Royle, Algebraic graph theory. Springer Science &
with network independent step-sizes and separated convergence rates,’’
Business Media, 2001, vol. 207.
IEEE Transactions on Signal Processing, vol. 67, no. 17, pp. 4494–4506,
[38] P. M. Pardalos, A. Žilinskas, J. Žilinskas et al., Non-convex multi-
2019.
objective optimization. Springer, 2017.
[62] J. Zhu, P. Xie, M. Zhang, R. Zheng, L. Xing, and Q. Wu, ‘‘Distributed
[39] Y. Collette and P. Siarry, Multiobjective optimization: principles and case
stochastic subgradient projection algorithms based on weight-balancing
studies. Springer Science & Business Media, 2004.
over time-varying directed graphs,’’ Complexity, vol. 2019, 2019.
[40] J. N. Tsitsiklis, ‘‘Problems in decentralized decision making and compu-
[63] W. Li, Z. Chen, Y. Lou, and Y. Hong, ‘‘Convergence properties of the
tation.’’ Massachusetts Inst of Tech Cambridge Lab for Information and
distributed projected subgradient algorithm over general graphs,’’ arXiv
Decision Systems, Tech. Rep., 1984.
preprint arXiv:2103.16993, 2021.
[41] N. Ryu and S. Min, ‘‘Multiobjective optimization with an adaptive weight [64] V. S. Mai, R. J. La, T. Zhang, and A. Battou, ‘‘Distributed optimiza-
determination scheme using the concept of hyperplane,’’ International tion with global constraints using noisy measurements,’’ arXiv preprint
Journal for Numerical Methods in Engineering, vol. 118, no. 6, pp. 303– arXiv:2106.07703, 2021.
319, 2019. [65] M. Zhu and S. Martinez, ‘‘On distributed convex optimization under
[42] V. Khare, X. Yao, and K. Deb, ‘‘Performance scaling of multi-objective inequality and equality constraints,’’ IEEE Transactions on Automatic
evolutionary algorithms,’’ in Evolutionary Multi-Criterion Optimization: Control, vol. 57, no. 1, pp. 151–164, 2011.
Second International Conference, EMO 2003, Faro, Portugal, April 8–11, [66] F. Bénézit, V. Blondel, P. Thiran, J. Tsitsiklis, and M. Vetterli, ‘‘Weighted
2003. Proceedings 2. Springer, 2003, pp. 376–390. gossip: Distributed averaging using non-doubly stochastic matrices,’’ in
[43] A. Nedic and A. Ozdaglar, ‘‘Distributed subgradient methods for multi- 2010 ieee international symposium on information theory. IEEE, 2010,
agent optimization,’’ IEEE Transactions on Automatic Control, vol. 54, pp. 1753–1757.
no. 1, pp. 48–61, 2009. [67] A. D. Domínguez-García and C. N. Hadjicostis, ‘‘Distributed strategies
[44] A. Nedic, A. Ozdaglar, and P. A. Parrilo, ‘‘Constrained consensus and for average consensus in directed graphs,’’ in 2011 50th IEEE Conference
optimization in multi-agent networks,’’ IEEE Transactions on Automatic on Decision and Control and European Control Conference. IEEE, 2011,
Control, vol. 55, no. 4, pp. 922–938, 2010. pp. 2124–2129.
[45] I. Matei and J. S. Baras, ‘‘Performance evaluation of the consensus-based [68] D. Kempe, A. Dobra, and J. Gehrke, ‘‘Gossip-based computation of
distributed subgradient method under random communication topolo- aggregate information,’’ in 44th Annual IEEE Symposium on Foundations
gies,’’ IEEE Journal of Selected Topics in Signal Processing, vol. 5, no. 4, of Computer Science, 2003. Proceedings. IEEE, 2003, pp. 482–491.
pp. 754–771, 2011. [69] K. Yuan, W. Xu, and Q. Ling, ‘‘Can primal methods outperform primal-
[46] Y. Kajiyama, N. Hayashi, and S. Takai, ‘‘Distributed subgradient method dual methods in decentralized dynamic optimization?’’ IEEE Transac-
with edge-based event-triggered communication,’’ IEEE Transactions on tions on Signal Processing, vol. 68, pp. 4466–4480, 2020.
Automatic Control, vol. 63, no. 7, pp. 2248–2255, 2018. [70] A. Mokhtari and A. Ribeiro, ‘‘Dsa: Decentralized double stochastic aver-
[47] X. Bao, B. Zhou, and H. Wang, ‘‘Distributed event-triggered projection aging gradient algorithm,’’ The Journal of Machine Learning Research,
subgradient algorithm over unbalanced digraphs based on row-stochastic vol. 17, no. 1, pp. 2165–2199, 2016.
matrices,’’ in 2021 11th International Conference on Information Science [71] D. Jakovetić, D. Bajović, J. Xavier, and J. M. Moura, ‘‘Primal–dual
and Technology (ICIST). IEEE, 2021, pp. 188–194. methods for large-scale and distributed convex optimization and data
[48] A. Nedic, ‘‘Asynchronous broadcast-based convex optimization over a analytics,’’ Proceedings of the IEEE, vol. 108, no. 11, pp. 1923–1938,
network,’’ IEEE Transactions on Automatic Control, vol. 56, no. 6, pp. 2020.
1337–1351, 2010. [72] H. Uzawa, ‘‘Iterative methods for concave programming,’’ Studies in
[49] X. Ren, D. Li, Y. Xi, and H. Shao, ‘‘Distributed subgradient algorithm for linear and nonlinear programming, vol. 6, pp. 154–165, 1958.
multi-agent optimization with dynamic stepsize,’’ IEEE/CAA Journal of [73] ——, ‘‘The kuhn-tucker theorem in concave programming,’’ in Traces
Automatica Sinica, vol. 8, no. 8, pp. 1451–1464, 2021. and Emergence of Nonlinear Programming. Springer, 2014, pp. 307–
[50] A. Nedić and A. Olshevsky, ‘‘Distributed optimization over time-varying 312.
directed graphs,’’ IEEE Transactions on Automatic Control, vol. 60, no. 3, [74] M. Hale and M. Egerstedt, ‘‘Cloud-based optimization: A quasi-
pp. 601–615, 2014. decentralized approach to multi-agent coordination,’’ in 53rd IEEE Con-
[51] ——, ‘‘Stochastic gradient-push for strongly convex functions on time- ference on Decision and Control. IEEE, 2014, pp. 6635–6640.
varying directed graphs,’’ IEEE Transactions on Automatic Control, [75] M. T. Hale, A. Nedić, and M. Egerstedt, ‘‘Cloud-based central-
vol. 61, no. 12, pp. 3936–3947, 2016. ized/decentralized multi-agent optimization with communication de-

14 VOLUME 11, 2023

Shokoufeh Naderi et al.: A Mapping and State-of-the-Art Survey on Multi-Objective Optimization Methods for Multi-Agent Systems

lays,’’ in 2015 54th IEEE Conference on Decision and Control (CDC). [99] M. Tao and X. Yuan, ‘‘On glowinski’s open question on the alternating
IEEE, 2015, pp. 700–705. direction method of multipliers,’’ Journal of Optimization Theory and
[76] G. H. Golub, P. C. Hansen, and D. P. O’Leary, ‘‘Tikhonov regularization Applications, vol. 179, no. 1, pp. 163–196, 2018.
and total least squares,’’ SIAM journal on matrix analysis and applica- [100] Z. Liu, F. Guo, W. Wang, and X. Wu, ‘‘A distributed parallel optimization
tions, vol. 21, no. 1, pp. 185–194, 1999. algorithm via alternating direction method of multipliers,’’ arXiv preprint
[77] M. T. Hale, A. Nedić, and M. Egerstedt, ‘‘Asynchronous multiagent arXiv:2111.10494, 2021.
primal-dual optimization,’’ IEEE Transactions on Automatic Control, [101] J. Xie, ‘‘On inexact admms with relative error criteria,’’ Computational
vol. 62, no. 9, pp. 4421–4435, 2017. Optimization and Applications, vol. 71, no. 3, pp. 743–765, 2018.
[78] B. A. Richards and T. P. Lillicrap, ‘‘Dendritic solutions to the credit [102] W. Li, Y. Liu, Z. Tian, and Q. Ling, ‘‘Cola: Communication-censored
assignment problem,’’ Current opinion in neurobiology, vol. 54, pp. 28– linearized admm for decentralized consensus optimization,’’ in ICASSP
36, 2019. 2019-2019 IEEE International Conference on Acoustics, Speech and
[79] J. Koshal, A. Nedić, and U. V. Shanbhag, ‘‘Multiuser optimization: Dis- Signal Processing (ICASSP). IEEE, 2019, pp. 5237–5241.
tributed algorithms and error analysis,’’ SIAM Journal on Optimization, [103] A. Mokhtari, W. Shi, Q. Ling, and A. Ribeiro, ‘‘Dqm: Decentralized
vol. 21, no. 3, pp. 1046–1081, 2011. quadratically approximated alternating direction method of multipliers,’’
[80] G. Qu and N. Li, ‘‘On the exponential stability of primal-dual gradient IEEE Transactions on Signal Processing, vol. 64, no. 19, pp. 5158–5173,
dynamics,’’ IEEE Control Systems Letters, vol. 3, no. 1, pp. 43–48, 2018. 2016.
[81] K. R. Hendrickson and M. T. Hale, ‘‘Towards totally asynchronous [104] Y. Wang, W. Yin, and J. Zeng, ‘‘Global convergence of admm in noncon-
primal-dual convex optimization in blocks,’’ in 2020 59th IEEE Confer- vex nonsmooth optimization,’’ Journal of Scientific Computing, vol. 78,
ence on Decision and Control (CDC). IEEE, 2020, pp. 3663–3668. no. 1, pp. 29–63, 2019.
[82] C. A. Uribe, S. Lee, A. Gasnikov, and A. Nedić, ‘‘A dual approach for [105] F. Wang, W. Cao, and Z. Xu, ‘‘Convergence of multi-block bregman
optimal algorithms in distributed optimization over networks,’’ in 2020 admm for nonconvex composite problems,’’ Science China Information
Information Theory and Applications Workshop (ITA). IEEE, 2020, pp. Sciences, vol. 61, no. 12, pp. 1–12, 2018.
1–37. [106] Q. Liu, X. Shen, and Y. Gu, ‘‘Linearized admm for nonconvex nons-
[83] H. Sakuma, N. Hayashi, and S. Takai, ‘‘Distributed primal-dual perturba- mooth optimization with convergence analysis,’’ IEEE Access, vol. 7, pp.
tion algorithm over unbalanced directed networks,’’ IEEE Access, vol. 9, 76 131–76 144, 2019.
pp. 75 324–75 335, 2021. [107] J. Zhang and Z.-Q. Luo, ‘‘A proximal alternating direction method of mul-
[84] M. R. Hestenes, ‘‘Multiplier and gradient methods,’’ Journal of optimiza- tiplier for linearly constrained nonconvex minimization,’’ SIAM Journal
tion theory and applications, vol. 4, no. 5, pp. 303–320, 1969. on Optimization, vol. 30, no. 3, pp. 2272–2302, 2020.
[108] H. Attouch, J. Bolte, P. Redont, and A. Soubeyran, ‘‘Proximal alternat-
[85] K. Scaman, F. Bach, S. Bubeck, Y. T. Lee, and L. Massoulié, ‘‘Optimal
ing minimization and projection methods for nonconvex problems: An
algorithms for smooth and strongly convex distributed optimization in
approach based on the kurdyka-łojasiewicz inequality,’’ Mathematics of
networks,’’ in international conference on machine learning. PMLR,
operations research, vol. 35, no. 2, pp. 438–457, 2010.
2017, pp. 3027–3036.
[109] R. Zhang and J. Kwok, ‘‘Asynchronous distributed admm for consensus
[86] K. Scaman, F. Bach, S. Bubeck, L. Massoulié, and Y. T. Lee, ‘‘Optimal al-
optimization,’’ in International conference on machine learning. PMLR,
gorithms for non-smooth distributed optimization in networks,’’ Advances
2014, pp. 1701–1709.
in Neural Information Processing Systems, vol. 31, 2018.
[110] T.-H. Chang, M. Hong, W.-C. Liao, and X. Wang, ‘‘Asynchronous dis-
[87] K. Scaman, F. Bach, S. Bubeck, Y. Lee, and L. Massoulié, ‘‘Optimal con-
tributed alternating direction method of multipliers: Algorithm and con-
vergence rates for convex distributed optimization in networks,’’ Journal
vergence analysis,’’ in 2016 IEEE International Conference on Acoustics,
of Machine Learning Research, vol. 20, pp. 1–31, 2019.
Speech and Signal Processing (ICASSP). IEEE, 2016, pp. 4781–4785.
[88] G. Lan and R. D. Monteiro, ‘‘Iteration-complexity of first-order aug-
[111] ——, ‘‘Asynchronous distributed admm for large-scale optimiza-
mented lagrangian methods for convex programming,’’ Mathematical
tion—part i: Algorithm and convergence analysis,’’ IEEE Transactions
Programming, vol. 155, no. 1, pp. 511–547, 2016.
on Signal Processing, vol. 64, no. 12, pp. 3118–3130, 2016.
[89] F. Mansoori and E. Wei, ‘‘Flexpd: A flexible framework of first-order [112] T.-H. Chang, W.-C. Liao, M. Hong, and X. Wang, ‘‘Asynchronous dis-
primal-dual algorithms for distributed optimization,’’ IEEE Transactions tributed admm for large-scale optimization—part ii: Linear convergence
on Signal Processing, vol. 69, pp. 3500–3512, 2021. analysis and numerical performance,’’ IEEE Transactions on Signal Pro-
[90] S. Liang, G. Yin et al., ‘‘Exponential convergence of distributed primal– cessing, vol. 64, no. 12, pp. 3131–3144, 2016.
dual convex optimization algorithm without strong convexity,’’ Automat- [113] T. Wu, K. Yuan, Q. Ling, W. Yin, and A. H. Sayed, ‘‘Decentralized
ica, vol. 105, pp. 298–306, 2019. consensus optimization with asynchrony and delays,’’ IEEE Transactions
[91] D. Jakovetić, J. M. Moura, and J. Xavier, ‘‘Linear convergence rate of on Signal and Information Processing over Networks, vol. 4, no. 2, pp.
a class of distributed augmented lagrangian algorithms,’’ IEEE Transac- 293–307, 2017.
tions on Automatic Control, vol. 60, no. 4, pp. 922–936, 2014. [114] M. Hong and T.-H. Chang, ‘‘Stochastic proximal gradient consensus over
[92] V. Nedelcu, I. Necoara, and Q. Tran-Dinh, ‘‘Computational complexity random networks,’’ IEEE Transactions on Signal Processing, vol. 65,
of inexact gradient augmented lagrangian methods: application to con- no. 11, pp. 2933–2948, 2017.
strained mpc,’’ SIAM Journal on Control and Optimization, vol. 52, no. 5, [115] N. Bastianello, A. Simonetto, and R. Carli, ‘‘Distributed prediction-
pp. 3109–3134, 2014. correction admm for time-varying convex optimization,’’ in 2020 54th
[93] J. Eckstein and W. Yao, ‘‘Relative-error approximate versions of douglas– Asilomar Conference on Signals, Systems, and Computers. IEEE, 2020,
rachford splitting and special cases of the admm,’’ Mathematical Pro- pp. 47–52.
gramming, vol. 170, no. 2, pp. 417–444, 2018. [116] W. Jiang and T. Charalambous, ‘‘Distributed alternating direction method
[94] N. Bastianello, R. Carli, L. Schenato, and M. Todescato, ‘‘Asynchronous of multipliers using finite-time exact ratio consensus in digraphs,’’ in 2021
distributed optimization over lossy networks via relaxed admm: Stabil- European Control Conference (ECC). IEEE, 2021, pp. 2205–2212.
ity and linear convergence,’’ IEEE Transactions on Automatic Control, [117] V. Khatana and M. V. Salapaka, ‘‘D-distadmm: Ao (1/k) distributed admm
vol. 66, no. 6, pp. 2620–2635, 2020. for distributed optimization in directed graph topologies,’’ in 2020 59th
[95] D.-R. Han, ‘‘A survey on some recent developments of alternating direc- IEEE Conference on Decision and Control (CDC). IEEE, 2020, pp.
tion method of multipliers,’’ Journal of the Operations Research Society 2992–2997.
of China, pp. 1–52, 2022. [118] V. Khatana, G. Saraswat, S. Patel, and M. V. Salapaka, ‘‘Gradient-
[96] E. Wei and A. Ozdaglar, ‘‘Distributed alternating direction method of mul- consensus method for distributed optimization in directed multi-agent
tipliers,’’ in 2012 IEEE 51st IEEE Conference on Decision and Control networks,’’ in 2020 American Control Conference (ACC). IEEE, 2020,
(CDC). IEEE, 2012, pp. 5445–5450. pp. 4689–4694.
[97] W. Shi, Q. Ling, K. Yuan, G. Wu, and W. Yin, ‘‘On the linear convergence [119] W. Jiang, A. Grammenos, E. Kalyvianaki, and T. Charalambous, ‘‘An
of the admm in decentralized consensus optimization,’’ IEEE Transac- asynchronous approximate distributed alternating direction method of
tions on Signal Processing, vol. 62, no. 7, pp. 1750–1761, 2014. multipliers in digraphs,’’ arXiv preprint arXiv:2104.11866, 2021.
[98] A. Makhdoumi and A. Ozdaglar, ‘‘Convergence rate of distributed admm [120] S. Zhang and C. P. Bailey, ‘‘Accelerated primal-dual algorithm for dis-
over networks,’’ IEEE Transactions on Automatic Control, vol. 62, no. 10, tributed non-convex optimization,’’ in 2021 IEEE Symposium Series on
pp. 5082–5095, 2017. Computational Intelligence (SSCI). IEEE, 2021, pp. 01–08.

VOLUME 11, 2023 15

 Shokoufeh Naderi et al.: A Mapping and State-of-the-Art Survey on Multi-Objective Optimization Methods for Multi-Agent Systems

[121] Z. Xu, G. Taylor, H. Li, M. A. Figueiredo, X. Yuan, and T. Goldstein, national Conference on Robotics and Automation (ICRA). IEEE, 2020,
‘‘Adaptive consensus admm for distributed optimization,’’ in Interna- pp. 3495–3501.
tional Conference on Machine Learning. PMLR, 2017, pp. 3841–3850. [143] L. Ferranti, R. R. Negenborn, T. Keviczky, and J. Alonso-Mora, ‘‘Co-
[122] N. S. Aybat, Z. Wang, T. Lin, and S. Ma, ‘‘Distributed linearized alter- ordination of multiple vessels via distributed nonlinear model predictive
nating direction method of multipliers for composite convex consensus control,’’ in 2018 European Control Conference (ECC). IEEE, 2018, pp.
optimization,’’ IEEE Transactions on Automatic Control, vol. 63, no. 1, 2523–2528.
pp. 5–20, 2017. [144] R. Zhao, M. Miao, J. Lu, Y. Wang, and D. Li, ‘‘Formation control of
[123] S. Boyd, N. Parikh, E. Chu, B. Peleato, J. Eckstein et al., ‘‘Distributed multiple underwater robots based on admm distributed model predictive
optimization and statistical learning via the alternating direction method control,’’ Ocean Engineering, vol. 257, p. 111585, 2022.
of multipliers,’’ Foundations and Trends® in Machine learning, vol. 3, [145] W. Guanghua, L. Deyi, G. Wenyan, and J. Peng, ‘‘Study on formation
no. 1, pp. 1–122, 2011. control of multi-robot systems,’’ in 2013 Third International Conference
[124] R. Glowinski, ‘‘On alternating direction methods of multipliers: a histor- on Intelligent System Design and Engineering Applications. IEEE, 2013,
ical perspective,’’ in Modeling, simulation and optimization for science pp. 1335–1339.
and technology. Springer, 2014, pp. 59–82. [146] E. Bahceci, O. Soysal, and E. Sahin, ‘‘A review: Pattern formation and
[125] J. Eckstein and W. Yao, ‘‘Understanding the convergence of the alter- adaptation in multi-robot systems,’’ Robotics Institute, Carnegie Mellon
nating direction method of multipliers: Theoretical and computational University, Pittsburgh, PA, Tech. Rep. CMU-RI-TR-03-43, 2003.
perspectives,’’ Pac. J. Optim., vol. 11, no. 4, pp. 619–644, 2015. [147] Z. Cheng, J. Ma, X. Zhang, and T. H. Lee, ‘‘Semi-proximal admm for
[126] A. Nedić, A. Olshevsky, and M. G. Rabbat, ‘‘Network topology and model predictive control problem with application to a uav system,’’ in
communication-computation tradeoffs in decentralized optimization,’’ 2020 20th International Conference on Control, Automation and Systems
Proceedings of the IEEE, vol. 106, no. 5, pp. 953–976, 2018. (ICCAS). IEEE, 2020, pp. 82–87.
[127] J.-h. Ryu, S. Kim, and H. Wan, ‘‘Pareto front approximation with adaptive [148] F. Mondada, E. Franzi, and A. Guignard, ‘‘The development of khepera,’’
weighted sum method in multiobjective simulation optimization,’’ in in Experiments with the Mini-Robot Khepera, Proceedings of the First
Proceedings of the 2009 Winter Simulation Conference (WSC). IEEE, International Khepera Workshop, no. CONF, 1999, pp. 7–14.
2009, pp. 623–633. [149] L. Ferranti, L. Lyons, R. R. Negenborn, T. Keviczky, and J. Alonso-Mora,
[128] A. M. Jubril, ‘‘A nonlinear weights selection in weighted sum for convex ‘‘Distributed nonlinear trajectory optimization for multi-robot motion
multiobjective optimization,’’ Facta Universitatis, vol. 27, no. 3, pp. 357– planning,’’ IEEE Transactions on Control Systems Technology, 2022.
372, 2012. [150] F. E. Schneider and D. Wildermuth, ‘‘A potential field based approach to
[129] M. Majidi, S. Nojavan, N. N. Esfetanaj, A. Najafi-Ghalelou, and K. Zare, multi robot formation navigation,’’ in IEEE International Conference on
‘‘A multi-objective model for optimal operation of a battery/pv/fuel Robotics, Intelligent Systems and Signal Processing, 2003. Proceedings.
cell/grid hybrid energy system using weighted sum technique and fuzzy 2003, vol. 1. IEEE, 2003, pp. 680–685.
satisfying approach considering responsible load management,’’ Solar [151] L. Ferranti, L. Lyons, R. R. Negenborn, T. Keviczky, and J. Alonso-
Energy, vol. 144, pp. 79–89, 2017. Mora. Distributed nonlinear trajectory optimization for multi-robot
[130] T. Wada, I. Masubuchi, K. Hanada, T. Asai, and Y. Fujisaki, ‘‘Dis- motion planning. Youtube. [Online]. Available: https://www.youtube.
tributed multi-objective optimization over randomly varying unbalanced com/watch?v=o2W2OhNf5yc
networks,’’ IFAC-PapersOnLine, vol. 50, no. 1, pp. 2403–2408, 2017. [152] A. Karimian, Z. Yang, and R. Tron, ‘‘Statistical outlier identification in
[131] I. Masubuchi, T. Wada, Y. Fujisaki, and F. Dabbene, ‘‘Distributed multi- multi-robot visual slam using expectation maximization,’’ arXiv preprint
agent optimization for pareto optimal problem over unbalanced networks arXiv:2002.02638, 2020.
via exact penalty methods with equality and inequality constraints,’’ in
Proceedings of the 23rd international symposium on mathematical theory
of networks and systems, 2018, pp. 447–452.
[132] D. P. Bertsekas, ‘‘Nonlinear programming,’’ Journal of the Operational
Research Society, vol. 48, no. 3, pp. 334–334, 1997.
[133] P. Xie, K. You, R. Tempo, S. Song, and C. Wu, ‘‘Distributed convex
optimization with inequality constraints over time-varying unbalanced
digraphs,’’ IEEE Transactions on Automatic Control, vol. 63, no. 12, pp.
4331–4337, 2018.
[134] R. Stern, ‘‘Multi-agent path finding–an overview,’’ Artificial Intelligence,
pp. 96–115, 2019.
[135] H. Ma, S. Koenig, N. Ayanian, L. Cohen, W. Hönig, T. Kumar,
T. Uras, H. Xu, C. Tovey, and G. Sharon, ‘‘Overview: Generalizations
of multi-agent path finding to real-world scenarios,’’ arXiv preprint
arXiv:1702.05515, 2017.
[136] R. Lal, A. Sharda, and P. Prabhakar, ‘‘Optimal multi-robot path planning
for pesticide spraying in agricultural fields,’’ in 2017 IEEE 56th Annual
Conference on Decision and Control (CDC). IEEE, 2017, pp. 5815–
5820.
[137] S.-S. Park, Y. Min, J.-S. Ha, D.-H. Cho, and H.-L. Choi, ‘‘A distributed
admm approach to non-myopic path planning for multi-target tracking,’’
IEEE Access, vol. 7, pp. 163 589–163 603, 2019.
[138] I. Mordatch and E. Todorov, ‘‘Combining the benefits of function approx-
imation and trajectory optimization.’’ in Robotics: Science and Systems,
vol. 4, 2014.
[139] Z. Yang and R. Tron, ‘‘Multi-agent path planning under observation SHOKOUFEH NADERI received her B.Sc degree
schedule constraints,’’ in 2020 IEEE/RSJ International Conference on
in electrical engineering-control from Sahand Uni-
Intelligent Robots and Systems (IROS). IEEE, 2020, pp. 6990–6997.
versity of Technology, Sahand, Iran, in 2015 and
[140] H. Y. Ong and J. C. Gerdes, ‘‘Cooperative collision avoidance via prox-
imal message passing,’’ in 2015 American Control Conference (ACC).
her M.Sc. degree in electrical engineering-control
IEEE, 2015, pp. 4124–4130. from Babol Noshirvani University of Technology,
[141] D. Guhathakurta, F. Rastgar, M. A. Sharma, M. Krishna, and A. K. Singh, Babol, Iran, in 2019.
‘‘Fast joint multi-robot trajectory optimization by gpu accelerated batch She is currently a Ph.D. candidate at the Uni-
solution of distributed sub-problems,’’ Frontiers in Robotics and AI, p. versité de Sherbrooke, Sherbrooke, Canada. Her
170, 2022. research interests include multi-objective opti-
[142] O. Shorinwa, J. Yu, T. Halsted, A. Koufos, and M. Schwager, ‘‘Distributed mization, multi-robot systems, intelligent and au-
multi-target tracking for autonomous vehicle fleets,’’ in 2020 IEEE Inter- tonomous systems and control.

16 VOLUME 11, 2023

Shokoufeh Naderi et al.: A Mapping and State-of-the-Art Survey on Multi-Objective Optimization Methods for Multi-Agent Systems

MAUDE J. BLONDIN earned a master’s degree

and a doctorate in electrical engineering from
the Université du Québec à Trois-Rivières, Trois-
Rivières, Québec, Canada, in 2014 and 2018 re-
spectively. She conducted postdoctoral research in
mechanical and aerospace engineering at the Uni-
versity of Florida, Gainesville, USA, from 2018 to
2020.
She has been an Assistant Professor in the De-
partment of Electrical and Computer Engineering
at the Université de Sherbrooke, Québec, Canada, since September 2020. She
has solid training in optimization applied to practical systems, with advanced
skills in distributed optimization algorithms applied to multiagent systems.
Invited by a Springer editor, she published a book on recent advances in
control optimization: Controller Tuning Optimization Methods for Multi-
Constraints and Nonlinear Systems: A Metaheuristic Approach, Springer,
2021. She has also co-edited two books in the Springer Optimization and
Its Applications series. She has published a journal article presenting a new
class of distributed optimization algorithms: A decentralized multi-objective
optimization algorithm, Springer, 2021. Her current research is driven by the
development of new distributed optimization algorithms to broaden the range
of operations for multi-agent systems.
Dr. Blondin is a member of the editorial board of the journal SN Operations
Research Forum. She is a member of INFORMS. She has won several awards
and grants corroborating her expertise and the impact of her research. Most
notably, she received the Young Researcher Prize from the International
Society of Global Optimization in the Summer of 2023.

VOLUME 11, 2023 17