1 Achieving Linear Convergence in Distributed Asynchronous … · 2019-09-12 · arXiv:1803.10359v4...
Transcript of 1 Achieving Linear Convergence in Distributed Asynchronous … · 2019-09-12 · arXiv:1803.10359v4...
arX
iv:1
803.
1035
9v4
[m
ath.
OC
] 1
1 Se
p 20
191
Achieving Linear Convergence in Distributed
Asynchronous Multi-agent OptimizationYe Tian, Ying Sun, and Gesualdo Scutari
Abstract—This papers studies multi-agent (convex and noncon-vex) optimization over static digraphs. We propose a general dis-tributed asynchronous algorithmic framework whereby i) agentscan update their local variables as well as communicate withtheir neighbors at any time, without any form of coordination;and ii) they can perform their local computations using (possibly)delayed, out-of-sync information from the other agents. Delaysneed not be known to the agent or obey any specific profile, andcan also be time-varying (but bounded). The algorithm buildson a tracking mechanism that is robust against asynchrony (inthe above sense), whose goal is to estimate locally the average ofagents’ gradients. When applied to strongly convex functions, weprove that it converges at an R-linear (geometric) rate as longas the step-size is sufficiently small. A sublinear convergencerate is proved, when nonconvex problems and/or diminishing,uncoordinated step-sizes are considered. To the best of ourknowledge, this is the first distributed algorithm with provablegeometric convergence rate in such a general asynchronoussetting. Preliminary numerical results demonstrate the efficacyof the proposed algorithm and validate our theoretical findings.
Index Terms—Asynchrony, Delay, Directed graphs, Distributedoptimization, Linear convergence, Nonconvex optimization.
I. INTRODUCTION
We study convex and nonconvex distributed optimization
over a network of agents, modeled as a directed fixed graph.
Agents aim at cooperatively solving the optimization problem
minx∈Rn
F (x) ,I∑
i=1
fi(x)
(P)
where fi : Rn → R is the cost function of agent i, assumed to
be smooth (nonconvex) and known only to agent i. In this
setting, optimization has to be performed in a distributed,
collaborative manner: agents can only receive/send informa-
tion from/to its immediate neighbors. Instances of (P) that
require distributed computing have found a wide range of
applications in different areas, including network information
processing, resource allocation in communication networks,
swarm robotic, and machine learning, just to name a few.
Many of the aforementioned applications give rise to ex-
tremely large-scale problems and networks, which naturally
call for asynchronous, parallel solution methods. In fact, asyn-
chronous modus operandi reduces the idle times of workers,
mitigate communication and/or memory-access congestion,
save power (as agents need not perform computations and
communications at every iteration), and make algorithms more
fault-tolerant. In this paper, we consider the following very
general, abstract, asynchronous model [3]:
Part of this work has been presented at the 56th Annual Allerton Conference[1] and posted on arxiv [2] on March 2018. This work has been supported bythe USA National Science Foundation under Grants CIF 1632599 and CIF1719205; and in part by the Office of Naval Research under Grant N00014-16-1-2244, and the Army Research Office under Grant W911NF1810238.
The authors are with the School of Industrial Engineer-ing, Purdue University, West-Lafayette, IN, USA; Emails:{tian110,sun578,gscutari}@purdue.edu.
(i) Agents can perform their local computations as well as
communicate (possibly in parallel) with their immediate
neighbors at any time, without any form of coordination
or centralized scheduling; and
(ii) when solving their local subproblems, agents can use
outdated information from their neighbors.
In (ii) no constraint is imposed on the delay profiles: delays
can be arbitrary (but bounded), time-varying, and (possibly)
dependent on the specific activation rules adopted to wakeup
the agents in the network. This model captures in a unified
fashion several forms of asynchrony: some agents execute
more iterations than others; some agents communicate more
frequently than others; and inter-agent communications can be
unreliable and/or subject to unpredictable, time-varying delays.
Several forms of asynchrony have been studied in the
literature–see Sec. I-A for an overview of related works.
However, we are not aware of any distributed algorithm that
is compliant to the asynchrony model (i)-(ii) and distributed
(nonconvex) setting above. Furthermore, when considering the
special case of a strongly convex function F , it is not clear how
to design a (first-order) distributed asynchronous algorithm (as
specified above) that achieves linear convergence rate. This
paper answers these questions–see Sec. I-B and Table 1 for a
summary of our contributions.
A. Literature Review
Since the seminal work [11], asynchronous parallelism has
been applied to several centralized optimization algorithms, in-
cluding block coordinate descent (e.g., [11]–[13]) and stochas-
tic gradient (e.g., [14], [15]) methods. However, these schemes
are not applicable to the networked setup considered in this
paper, because they would require the knowledge of the
function F from each agent. Distributed methods exploring
(some form of) asynchrony over networks with no centralized
node have been studied in [4]–[10], [16]–[26]. We group next
these works based upon the features (i)-(ii) above.
(a) Random activations and no delays [16]–[20]: These
schemes considered distributed convex unconstrained opti-
mization over undirected graphs. While substantially different
in the form of the updates performed by the agents– [16], [18],
[20] are instances of primal-dual (proximal-based) algorithms,
[19] is an ADMM-type algorithm, while [17] is based on the
distributed gradient tracking mechanism introduced in [27]–
[29]–all these algorithms are asynchronous in the sense of
feature (i) [but not (ii)]: at each iteration, a subset of agents
[16], [18], [20] (or edge-connected agents [17], [19]), chosen
at random, is activated, performing then their updates and
communications with their immediate neighbors; between two
activations, agents are assumed to be in idle mode (i.e., able
to continuously receive information). However, no form of
2
AlgorithmNonconvex
Cost FunctionNo IdleTime
ArbitraryDelays
ParallelStep Sizes
Digraph Global Convergence to Exact SolutionsRate Analysis
FixedUncoordinatedDiminishing
Linear Rate forStrongly Convex
Nonconvex
Asyn. Broadcast [4] X X X In expectation (w. diminishing step)
Asyn. Diffusion [5] XAsyn. ADMM [6] X X Deterministic
Dual Ascent in [7] X Restricted Restricted Xra-NRC [8] X XARock [9] X Restricted X Almost surely In expectation
ASY-PrimalDual [10] X Restricted X Almost surely
ASY-SONATA X X X X X X X Deterministic Deterministic Deterministic
Table 1COMPARISON WITH STATE-OF-ART DISTRIBUTED ASYNCHRONOUS ALGORITHMS. CURRENT SCHEMES CAN DEAL WITH UNCOORDINATED ACTIVATIONS
BUT ONLY WITH SOME FORMS OF DELAYS. ASY-SONATA ENJOYS ALL THE DESIRABLE FEATURES LISTED IN THE TABLE.
delays is allowed: every agent must perform its local com-
putations/updates using the most updated information from
its neighbors. This means that all the actions performed by
the agent(s) in an activation must be completed before a new
activation (agent) takes place (wakes-up), which calls for some
coordination among the agents. Finally, no convergence rate
was provided for the aforementioned schemes but [17], [19].
(b) Synchronous activations and delays [21]–[26]: These
schemes considered distributed constrained convex optimiza-
tion over undirected graphs. They study the impact of delayed
gradient information [21], [22] or communication delays (fixed
[23], uniform [22], [26] or time-varying [24], [25]) on the
convergence rate of distributed gradient (proximal [21], [22]
or projection-based [25], [26]) algorithms or dual-averaging
distributed-based schemes [23], [24]. While these schemes
are all synchronous [thus lacking of feature (i)], they can
tolerate communication delays [an instantiation of feature (ii)],
converging at a sublinear rate to an optimal solution. Delays
must be such that no losses occur–every agent’s message will
eventually reach its destination within a finite time.
(c) Random/cyclic activations and some form of delays [4]–
[10]: The class of optimization problems along with the key
features of the algorithms proposed in these papers are sum-
marized in Table 1 and briefly discussed next. The majority of
these works studied distributed (strongly) convex optimization
over undirected graphs, with [5] assuming that all the functions
fi have the same minimizer, [6] considering also nonconvex
objectives, and [8] being implementable also over digraphs.
The algorithms in [4], [5] are gradient-based schemes; [6] is a
decentralized instance of ADMM; [9] applies an asynchronous
parallel ADMM scheme to distributed optimization; and [10]
builds on a primal-dual method. The schemes in [7], [8]
instead build on (approximate) second-order information. All
these algorithms are asynchronous in the sense of feature (i):
[4]–[6], [9], [10] considered random activations of the agents
(or edges-connected agents) while [7], [8] studied determin-
istic, uncoordinated activation rules. As far as feature (ii) is
concerned, some form of delays is allowed. More specifically,
[4]–[6], [8] can deal with packet losses: the information sent
by an agent to its neighbors either gets lost or received with no
delay. They also assume that agents are always in idle mode
between two activations. Closer to the proposed asynchronous
framework are the schemes in [9], [10] wherein a probabilistic
model is employed to describe the activation of the agents
and the aged information used in their updates. The model
requires that the random variables triggering the activation of
the agents are i.i.d and independent of the delay vector used
by the agent to performs its update. While this assumption
makes the convergence analysis possible, in reality, there is a
strong dependence of the delays on the activation index; see
[13] for a detailed discussion on this issue and several counter
examples. Other consequences of this model are: the schemes
[9], [10] are not parallel–only one agent per time can perform
the update–and a random self-delay must be used in the
update of each agent (even if agents have access to their most
recent information). Furthermore, [9] calls for the solution of a
convex subproblem for each agent at every iteration. Referring
to the convergence rate, [9] is the only scheme exhibiting linear
convergence in expectation, when each fi is strongly convex
and the graph undirected. No convergence rate is available in
any of the aforementioned papers, when F is nonconvex.
B. Summary of Contributions
This paper proposes a general distributed, asynchronous
algorithmic framework for (strongly) convex and nonconvex
instances of Problem (P), over directed graphs. The algorithm
leverages a perturbed “sum-push” mechanism that is robust
against asynchrony, whose goal is to track locally the average
of agents’ gradients; this scheme along with its convergence
analysis are of independent interest. To the best of our knowl-
edge, the proposed framework is the first scheme combining
the following attractive features (cf. Table 1): (a) it is parallel
and asynchronous [in the sense (i) and (ii)]–multiple agents
can be activated at the same time (with no coordination) and/or
outdated information can be used in the agents’ updates; our
asynchronous setting (i) and (ii) is less restrictive than the
one in [9], [10]; furthermore, in contrast with [9], our scheme
avoids solving possibly complicated subproblems; (b) it is
applicable to nonconvex problems, with probable convergence
to stationary solutions of (P); (c) it is implementable over
digraph; (d) it employs either a constant step-size or uncoor-
dinated diminishing ones; (e) it converges at an R-linear rate
(resp. sublinear) when F is strongly convex (resp. nonconvex)
and a constant (resp. diminishing, uncoordinated) step-size(s)
is employed; this contrasts [9] wherein each fi needs to be
strongly convex; and (f) it is “protocol-free”, meaning that
agents need not obey any specific communication protocols or
asynchronous modus operandi (as long as delays are bounded
and agents update/communicate uniformly infinitely often).
On the technical side, convergence is studied introducing
two techniques of independent interest, namely: i) the asyn-
chronous agent system is reduced to a synchronous “aug-
mented” one with no delays by adding virtual agents to the
graph. While this idea was first explored in [30], [31], [32],
the proposed enlarged system and algorithm differ from those
used therein, which cannot deal with the general asynchronous
model considered here–see Remark 13, Sec.VI; and ii) the
rate analysis is employed putting forth a generalization of
the small gain theorem (widely used in the literature [33]
3
to analyze synchronous schemes), which is expected to be
broadly applicable to other distributed algorithms.
C. Notation
Throughout the paper we use the following notation. Given
the matrix M , (Mij)Ii,j=1, Mi,: and M:,j denote its i-th row
vector and j-th column vector. Given the sequence {Mt}kt=s,
with k ≥ s, we define Mk:s , MkMk−1 · · ·Ms+1Ms, if
k > s; and Mk:s , Ms otherwise. Given two matrices
(vectors) A and B of same size, by A 4 B we mean that
B − A is a nonnegative matrix (vector). The dimensions of
the all-one vector 1 and the i-th canonical vector ei will be
clear from the context. We use ‖·‖ to represent the Euclidean
norm for a vector whereas the spectral norm for a matrix. The
indicator function 1[E] of an event E equals to 1 when the
eventE is true, and 0 otherwise. Finally, we use the convention∑t∈∅ x
t = 0 and∏
t∈∅ xt = 1.
II. PROBLEM SETUP AND PRELIMINARIES
A. Problem Setup
We study Problem (P) under the following assumptions.
Assumption 1 (On the optimization problem).
a. Each fi : Rn → R is proper, closed and Li-Lipschitz
differentiable;
b. F is bounded from below. �
Note that fi need not be convex. We also make the blanket
assumption that each agent i knows only its own fi, but
not∑
j 6=i fj . To state linear convergence, we will use the
following extra condition on the objective function.
Assumption 2 (Strong convexity). Assumption 1(i) holds and,
in addition, F is τ -strongly convex. �
On the communication network: The communication net-
work of the agents is modeled as a fixed, directed graph
G = (V , E), where V = {1, . . . , I} is the set of nodes (agents),
and E ⊆ V × V is the set of edges (communication links).
If (i, j) ∈ E , it means that agent i can send information to
agent j. We assume that the digraph does not have self-loops.
We denote by N ini the set of in-neighbors of node i, i.e.,
N ini , {j ∈ V | (j, i) ∈ E} while N out
i , {j ∈ V | (i, j) ∈ E}is the set of out-neighbors of agent i. We make the following
standard assumption on the graph connectivity.
Assumption 3. The graph G is strongly connected. �
B. Preliminaries: The SONATA algorithm [34], [35]
The proposed asynchronous algorithmic framework builds
on the synchronous SONATA algorithm, proposed in [34],
[35] to solve (nonconvex) multi-agent optimization problems
over time-varying digraphs. This is motivated by the fact that
SONATA has the unique property of being provably applicable
to both convex and nonconvex problems, and it achieves linear
convergence when applied to strongly convex objectives F .
We thus begin reviewing SONATA, tailored to (P); then we
generalized it to the asynchronous setting (cf. Sec. IV).
Every agent controls and iteratively updates the tuple
(xi,yi, zi, φi): xi is agent i’s copy of the shared variables
x in (P); yi acts as a local proxy of the sum-gradient ∇F ;
and zi and φi are auxiliary variables instrumental to deal with
communications over digraphs. Let xki , z
ki , φki , and yk
i denote
the value of the aforementioned variables at iteration k ∈ N0.
The update of each agent i reads:
xk+1i =
∑
j∈N ini ∪{i}
wij
(xkj − αk yk
j
), (1)
zk+1i =
∑
j∈N ini ∪{i}
aijzkj +∇fi(xk+1
i )−∇fi(xki ), (2)
φk+1i =
∑
j∈N ini ∪{i}
aijφkj , (3)
yk+1i = zk+1
i /φk+1i , (4)
with z0i = y0i = ∇fi(x0
i ) and φ0i = 1, for all i ∈V . In (1), yk
i is a local estimate of the average-gradient
(1/I)∑I
i=1∇fi(xki ). Therefore, every agent, first moves
along the estimated gradient direction, generating xki −αk yk
i
(αk is the step-size); and then performs a consensus step
to force asymptotic agreement among the local variables xi.
Steps (2)-(4) represent a perturbed-push-sum update, aiming at
tracking the gradient (1/I)∇F [28], [29], [35]. The weight-
matrices W , (wij)Ii,j=1 and A , (aij)
Ii,j=1 satisfy the
following standard assumptions.
Assumption 4 (On the weight-matrices). The weight-matrices
W , (wij)Ii,j=1 and A , (aij)
Ii,j=1 satisfy (we will write
M , (mij)Ii,j=1 to denote either A or W):
a. ∃ m > 0 such that mii ≥ m, for all i ∈ V; and mij ≥ m,
for all (j, i) ∈ E; mij = 0, otherwise;
b. W is row-stochastic, that is, W1 = 1;
c. A is column-stochastic, that is, AT 1 = 1; �
In [33], a special instance of SONATA, was proved to
converges at an R-linear rate when F is strongly convex. This
result was extended to constrained, nonsmooth (composite),
distributed optimization in [36]. A natural question is whether
SONATA works also in an asynchronous setting still converg-
ing at a linear rate. Naive asynchronization of the updates (1)-
(4)–such as using uncoordinated activations and/or replacing
instantaneous information with a delayed one–would not work.
For instance, the tracking (2)-(4) calls for the invariance of the
averages, i.e.,∑I
i=1 zki =
∑Ii=1∇fi(xk), for all k ∈ N0. It is
not difficult to check that any perturbation in (2)-e.g., in the
form of delays or packet losses–puts in jeopardy this property.
To cope with the above challenges, a first step is robustifying
the gradient tracking scheme. In Sec. III, we introduce P-
ASY-SUM-PUSH–an asynchronous, perturbed, instance of
the push-sum algorithm [37], which serves as a unified algo-
rithmic framework to accomplish several tasks over digraphs
in an asynchronous manner, such as solving the average
consensus problem and tracking the average of agents’ time-
varying signals. Building on P-ASY-SUM-PUSH, in Sec. IV,
we finally present the proposed distributed asynchronous op-
timization framework, termed ASY-SONATA.
III. PERTURBED ASYNCHRONOUS SUM-PUSH
We present P-ASY-SUM-PUSH; the algorithm was first
introduced in our conference paper [1], which we refer to for
4
details on the genesis of the scheme and intuitions; here we
directly introduce the scheme and study its convergence.
Consider an asynchronous setting wherein agents compute
and communicate independently without coordination. Every
agent i maintains state variables zi, φi, yi, along with the
following auxiliary variables that are instrumental to deal
with uncoordinated activations and delayed information: i)
the cumulative-mass variables ρji and σji, with j ∈ N outi ,
which capture the cumulative (sum) information generated
by agent i up to the current time and to be sent to agent
j ∈ N outi ; consequently, ρij and σij are received by i from its
in-neighbors j ∈ N ini ; and ii) the buffer variables ρij and σij ,
with j ∈ N ini , which store the information sent from j ∈ N in
i
to i and used by i in its last update. Values of these variables
at iteration k ∈ N0 are denoted by the same symbols with the
superscript “k”. Note that, because of the asynchrony, each
agent i might have outdated ρij and σij ; ρk−dk
j
ij (resp. σk−dk
j
ij )
is a delayed version of the current ρkij (resp. σkij ) owned by j
at time k, where 0 ≤ dkj ≤ D <∞ is the delay. Similarly, ρijand σij might differ from the last information generated by jfor i, because agent i might not have received that information
yet (due to delays) or never will (due to packet losses).
The proposed asynchronous algorithm, P-ASY-SUM-
PUSH, is summarized in Algorithm 1. A global iteration clock
(not known to the agents) is introduced: k → k+1 is triggered
based upon the completion from one agent, say ik, of the fol-
lowing actions. (S.2): agent ik maintains a local variable τikj ,
for each j ∈ N inik , which keeps track of the “age” (generated
time) of the (ρ, σ)-variables that it has received from its in-
neighbors and already used. If k−dkj is larger than the current
counter τk−1ikj
, indicating that the received (ρ, σ)-variables are
newer than those currently stored, agent ik accepts ρk−dk
j
ikjand
σk−dk
j
ikj, and updates τikj as k − dkj ; otherwise, the variables
will be discarded and τikj remains unchanged. Note that (5)
can be performed without any coordination. It is sufficient that
each agent attaches a time-stamp to its produced information
reflecting it local timing counter. We describe next the other
steps, assuming that new information has come in to agent
ik, that is, τikj = k − dkj . (S.3.1): In (6), agent ik builds the
intermediate “mass” zk+ 1
2
ikbased upon its current information
zkik and ρkikj , and the (possibly) delayed one from its in-
neighbors, ρk−dk
j
ikj; and ǫk ∈ R
n is an exogenous perturbation
(later this perturbation will be properly chosen to accomplish
specific goals, see Sec. IV). Note that the way agent ik forms
its own estimates ρk−dk
j
ikjis immaterial to the description of the
algorithm. The local buffer ρkikj stores the value of ρikj that
agent ik used in its last update. Therefore, if the information
in ρk−dk
j
ikjis not older than the one in ρkikj , the difference
ρk−dk
j
ikj− ρkikj in (6) will capture the sum of the aikjzj ’s
that have been generated by j ∈ N inik for ik up until k − dkj
and not used by agent ik yet. For instance, in a synchronous
setting, one would have ρkikj − ρkikj = aikjzk+ 1
2
j . (S.3.2): the
generated zk+ 1
2
ikis “pushed back” to agent ik itself and its out-
Algorithm 1 P-ASY-SUM-PUSH (Global View)
Data: z0i ∈ Rn, φ0i = 1, ρ0ij = 0, σ0ij = 0, τ−1
ij = −D,
for all j ∈ N ini and i ∈ V ; σt
ij = 0 and ρtij = 0, for all
t = −D, . . . , 0; and {ǫk}k∈N0. Set k = 0.
While: a termination criterion is not met do
(S.1) Pick (ik,dk), with dk , (dkj )j∈N in
ik;
(S.2) Set (purge out the old information):
τkikj = max(τk−1ikj
, k − dkj), ∀j ∈ N in
ik ; (5)
(S.3) Update the variables performing
• (S.3.1) Sum step:
zk+ 1
2
ik= z
kik +
∑
j∈N in
ik
(
ρ
τk
ikj
ikj− ρ
kikj
)
+ ǫk
(6)
φk+ 1
2
ik= φk
ik +∑
j∈N in
ik
(
στk
ikj
ikj− σk
ikj
)
• (S.3.2) Push step:
zk+1
ik= aikik z
k+ 1
2
ik, φk+1
ik= aikik φ
k+ 1
2
ik
ρk+1
jik= ρ
kjik + ajik z
k+ 1
2
ik, (7)
σk+1
jik= σk
jik + ajik φk+ 1
2
ik, ∀j ∈ N
out
ik
• (S.3.3) Mass-Buffer update:
ρk+1
ikj= ρ
τk
ikj
ikj, σk+1
ikj= σ
τk
ikj
ikj, ∀j ∈ N
inik (8)
• (S.3.4) Set: yk+1
ik= z
k+1
ik/φk+1
ik.
(S.4) Untouched state variables shift to state k + 1while keeping the same value; k ← k + 1.
neighbors. Specifically, out of the total mass zk+ 1
2
ikgenerated,
agent ik gets aii zk+ 1
2
i , determining the update zki → zk+1i
while the remaining is allocated to the agents j ∈ N outik , with
ajik zk+ 1
2
ikcumulating to the mass buffer ρkjik and generating
the update ρkjik → ρk+1jik
, to be sent to agent j. (S.3.3): each
local buffer variable ρkikj is updated to account for the use of
new information from j ∈ N inik . The final information is then
read on the y-variables [cf. (S.3.4)].
Remark 5. (Global view description) Note that each agent’s
update is fully defined, once ik and dk are given. The selection
(ik,dk) in (S.1) is not performed by anyone; it is instead
an a-posteriori description of agents’ actions: All agents act
asynchronously and continuously; the agent completing the
“push” step and updating its own variables triggers retrospec-
tively the iteration counter k → k + 1 and determines the
pair (ik,dk) along with all quantities involved in the other
steps. Differently from most of the current literature, this
“global view” description of the agents’ actions allows us to
abstract from specific computation-communication protocols
and asynchronous modus operandi and captures by a unified
model a gamut of asynchronous schemes.
Convergence is given under the following assumptions.
Assumption 6 (On the asynchronous model). Suppose:
5
a. ∃ 0 < T <∞ such that ∪k+T−1t=k it = V , for all k ∈ N0;
b. ∃ 0 < D < ∞ such that 0 ≤ dkj ≤ D, for all j ∈ N inik
and k ∈ N0. �
The next theorem studies convergence of P-ASY-SUM-
PUSH, establishing geometric decay of the error ‖yki −
(1/I) · mkz‖, even in the presence of unknown (bounded)
perturbations, where mkz ,
∑Ii=1 z
ki +
∑(j,i)∈E(ρ
kij − ρkij)
represents the “total mass” of the system at iteration k.
Theorem 7. Let {yk , [yk1 , . . . ,y
kI ]
⊤, zk , [zk1 , . . . , zkI ]
⊤,(ρkij , ρ
kij)(j,i)∈E}k∈N0
be the sequence generated by Algo-
rithm 1, under Assumption 3, 6, and with A , (aij)Ii,j=1 sat-
isfying Assumption 4 (i),(iii). Define K1 , (2 I−1) ·T +I ·D.There exist constants ρ ∈ (0, 1) and C1 > 0, such that
∥∥∥yk+1i − (1/I) ·mk+1
z
∥∥∥ ≤ C1
(ρk∥∥z0∥∥+
k∑
l=0
ρk−l∥∥ǫl∥∥),
(9)
for all i ∈ V and k ≥ K1 − 1.
Furthermore, mkz =
∑Ii=1 z
0i +
∑k−1t=0 ǫ
t.
Proof. See Sec. VI.
Discussion: Several comments are in order.
1) On the asynchronous model: Algorithm 1 captures a
gamut of asynchronous parallel schemes and architectures,
through the mechanism of generation of (ik,dk). Assump-
tion 6 on (ik,dk) is quite mild: (a) controls the frequency of
the updates whereas (b) limits the age of the old information
used in the computations; they can be easily enforced in
practice. For instance, (a) is readily satisfied if each agent
wakes up and performs an update whenever some independent
internal clock ticks or it is triggered by some of the neighbors;
(b) imposes conditions on the frequency and quality of the
communications: information used by each agent cannot be-
come infinitely old, implying that successful communications
must occur sufficiently often. This however does not enforce
any specific protocol on the activation/idle time/communica-
tion. For instance, i) agents need not perform the actions in
Algorithm 1 sequentially or inside the same activation round;
or ii) executing the “push” step does not mean that agents
must broadcast their new variables in the same activation; this
would just incur a delay (or packet loss) in the communication.
Note that the time-varying nature of the delays dk permits
to model also packet losses, as detailed next. Suppose that
at iteration k1 agent j sends its current ρ, σ-variables to its
out-neighbor ℓ and they get lost; and let k2 be the subsequent
iteration when j updates again. Let t be the first iteration after
k1 when agent ℓ performs its update; it will use information
from j such that t−dtj /∈ [k1+1, k2], for some dtj ≤ D <∞.
If t−dtj < k1+1, no newer information from j has been used
by ℓ; otherwise t− dtj ≥ k2 + 1 (implying k2 < t), meaning
that agent ℓ has used information not older than k2 + 1.
2) Comparison with [8], [30], [38]: The use of counter
variables [such as (ρ, σ, ρ, σ)-variables in our scheme] was
first introduced in [30] to design a synchronous average
consensus algorithm robust to packet losses. In [38], this
scheme was extended to deal with uncoordinated (deter-
ministic) agents’ activations whereas [8] built on [38] to
design, in the same setting, a distributed Newton-Raphson
algorithm. There are important differences between P-ASY-
SUM-PUSH and the aforementioned schemes, namely: i)
none of them can deal with delays but packet losses; ii) [30]
is synchronous; and iii) [8], [38] are not parallel schemes, as
at each iteration only one agent is allowed to wake up and
transmit information to its neighbors. For instance, [8], [38]
cannot model synchronous parallel (Jacobi) updates. Hence,
the convergence analysis of P-ASY-SUM-PUSH calls for a
new line of proof, as introduced in Sec. VI.
3) Beyond average consensus: By choosing properly the
perturbation signal ǫk, P-ASY-SUM-PUSH can solve differ-
ent problems. Some examples are discussed next.
(i) Error free: ǫk = 0. P-ASY-SUM-PUSH solves the
average consensus problem and (9) reads
∥∥∥yk+1i − (1/I) ·
I∑
i=1
z0i
∥∥∥ ≤ C1 ρk∥∥z0∥∥ .
(ii) Vanishing error: limk→∞ ‖ǫk‖ = 0. Using [29,
Lemma 7(a)], (9) reads limk→∞ ‖yk+1i −m
k+1z ‖ = 0.
(iii) Asynchronous tracking. Each agent i owns a (time-
varying) signal {uki }k∈N0
; the average tracking problem con-
sists in asymptotically track the average signal uk , (1/I) ·∑Ii=1 u
ki , that is,
limk→∞
∥∥yk+1i − uk+1
∥∥ = 0, ∀i ∈ V . (10)
Under mild conditions on the signal, this can be accomplished
in a distributed and asynchronous fashion, using P-ASY-
SUM-PUSH, as formalized next.
Corollary 7.1. Consider, the following setting in P-ASY-
SUM-PUSH: z0i = u0i , for all i ∈ V; ǫk = uk+1
ik− uk
ik ,
with
uk+1i =
{uk+1i if i = ik;
uki otherwise;
u0i = u0
i ;
Then (9) holds, with mk+1z =
∑Ii=1 u
k+1i . Furthermore, if
limk→∞∑I
i=1
∥∥uk+1i − uk
i
∥∥ = 0, then (10) holds.
Proof. See Appendix E.
This instance of P-ASY-SUM-PUSH will be used in
Sec. IV to perform asynchronous gradient tracking.
Remark 8 (Asynchronous average consensus). To the best of
our knowledge, the error-free instance of the P-ASY-SUM-
PUSH discussed above is the first (stepsize-free) scheme that
provably solves the average consensus problem at a linear
rate, under the general asynchronous model described by
Assumption 6. In fact, the existing asynchronous consensus
schemes [31] [32] achieve an agreement among the agents’
local variables whose value is not in general the average of
their initial values, but instead some unknown function of them
and the asynchronous modus operandi of the agents. Related
to the P-ASY-SUM-PUSH is the ra-AC algorithm in [38],
which enjoys the same convergence property but under a more
restrictive and specific asynchronous model (no delays but
packet losses and single-agent activation per iteration).
6
IV. ASYNCHRONOUS SONATA (ASY-SONATA)
We are ready now to introduce our distributed asynchronous
algorithm–ASY-SONATA. The algorithm combines SONATA
(cf. Sec. II-B) with P-ASY-SUM-PUSH (cf. Sec. III), the
latter replacing the synchronous tracking scheme (2)-(4). The
“global view” of the scheme is given in Algorithm 2.
Algorithm 2 ASY-SONATA (Global View)
Data: For all agent i and ∀j ∈ N ini , x0
i ∈ Rn, z0i =∇fi(x0
i ), φ0i = 1, ρ0ij = 0, σ0
ij = 0, τ−1ij = −D. And
for t = −D,−D + 1, . . . , 0, ρtij = 0, σtij = 0, vt
i = 0. Set
k = 0.
While: a termination criterion is not met do
(S.1) Pick (ik,dk);(S.2) Set:
τkikj = max(τk−1ikj
, k − dkj ), ∀j ∈ N inik .
(S.3) Local Descent:
vk+1ik
= xkik − γkzkik . (11)
(S.4) Consensus:
xk+1ik
= wikikvk+1ik
+∑
j∈N in
ik
wikjvτk
ikj
j .
(S.5) Gradient Tracking:
• (S.5.1) Sum step:
zk+ 1
2
ik=z
kik +
∑
j∈N in
ik
(
ρ
τk
ikj
ikj− ρ
kikj
)
+∇fik (xk+1
ik)−∇fik (x
kik )
• (S.5.2) Push step:
zk+1
ik= aikik z
k+ 1
2
ik,
ρk+1
jik= ρ
kjik + ajik z
k+ 1
2
ik, ∀j ∈ N
out
ik
• (S.5.3) Mass-Buffer update:
ρk+1
ikj= ρ
τk
ikj
ikj, ∀j ∈ N
inik
(S.6) Untouched state variables shift to state k + 1while keeping the same value; k ← k + 1.
In ASY-SONATA, agents continuously and with no coordi-
nation perform: i) their local computations [cf. (S.3)], possibly
using an out-of-sync estimate zkik of the average gradient; in
(11), γk is a step-size (to be properly chosen); ii) a consensus
step on the x-variables, using possibly outdated information
vτk
ikj
j from their in-neighbors [cf. (S.4)]; and iii) gradient
tracking [cf. (S.5)] to update the local estimate zkik , based
on the current cumulative mass variables ρτk
ikj
ikj, and buffer
variables ρkikj , j ∈ N inik .
Note that in Algorithm 1, the tracking variable yk+1ik
is
obtained rescaling zk+1ik
by the factor 1/φk+1ik
. In Algorithm 2,
we absorbed the scaling 1/φk+1ik
in the step size and use
directly zk+1ik
as a proxy of the average gradient, eliminating
thus the φ-variables (and the related σ-, σ-variables). Also,
for notational simplicity and without loss of generality, we
assumed that the v- and ρ- variables are subject to the same
delays (e.g., they are transmitted within the same packet); same
convergence results hold if different delays are considered.
We study now convergence of the scheme, under a constant
step-size or diminishing, uncoordinated ones.
A. Constant Step-size
Theorem 9 below establishes linear convergence of ASY-
SONATA when F is strongly convex.
Theorem 9 (Geometric convergence). Consider (P) under
Assumption 2, and let x⋆ denote its unique solution. Let
{(xki )
Ii=1}k∈N0
be the sequence generated by Algorithm 2,
under Assumption 3, 6, and with weight-matrices W and A
satisfying Assumption 4. Then, there exists a constant γ1 > 0[cf. (46)] such that if γk ≡ γ ≤ γ1, it holds
Msc(xk) , ‖xk − 1I ⊗ x⋆‖ = O(λk), (12)
with λ ∈ (0, 1) given by
λ =
{1− τm2K1γ
2 if γ ∈ (0, γ1],
ρ+√J1γ if γ ∈ (γ1, γ2),
(13)
where γ1 and γ2 are some constants strictly smaller than γ1,
and J1 , (1 − ρ)2/γ2.
Proof. See Sec. VII.
When F is convex (resp. nonconvex), we introduce the
following merit function to measure the progresses of the al-
gorithm towards optimality (resp. stationarity) and consensus:
MF (xk) , max{
∥∥∇F (xk)∥∥2 ,
∥∥xk − 1I ⊗ xk∥∥2}, (14)
where xk , [xk⊤1 , · · · ,xk⊤
I ]⊤ and xk , (1/I) ·∑Ii=1 x
ki .
Note that MF is a valid merit function, since it is continuous
and MF (x) = 0 if and only if all xi’s are consensual and
optimal (resp. stationary solutions).
Theorem 10 (Sublinear convergence). Consider (P) under
Assumption 1 (thus possibly nonconvex). Let {(xki )
Ii=1}k∈N0
be the sequence generated by Algorithm 2, in the same setting
of Theorem 9. Given δ > 0, let Tδ be the first iteration k ∈ N0
such that MF (xk) ≤ δ. Then, there exists a γ2 > 0 [cf. (55)],
such that if γk ≡ γ ≤ γ2, Tδ = O(1/δ). The values of the
above constants is given in the proof.
Proof. See Sec. VIII.
Theorem 9 states that consensus and optimization errors of
the sequence generated by ASY-SONATA vanish at a linear
rate. We are not aware of any other scheme enjoying such
a property in such a distributed, asynchronous computing
environment. For general, possibly nonconvex instances of
Problem (P), Theorem 10 shows that both consensus and op-
timization errors of the sequence generated by ASY-SONATA
vanish at O(1/δ) sublinear rate.
The choice of a proper stepsize calls for the estimates of γ1and γ2 in Theorems 9 and 10, which depend on the following
quantities: the optimization parameters Li (Lipschitz constants
of the gradients) and τ (strongly convexity constant), the
7
network connectivity parameter ρ, and the constants D and
T due to the asynchrony (cf. Assumption 6). Notice that the
dependence of the stepsize on Li, τ , and ρ is common to all
the existing distributed synchronous algorithms and so is that
on T and D to (even centralized) asynchronous algorithms
[3]. While Li, τ , and ρ can be acquired following approaches
discussed in the literature (see, e.g., [33, Remark 4]), it is
less clear how to estimate D and T , as they are related to
the asynchronous model, generally not known to the agents.
As an example, we address this question considering the
following fairly general model for the agents’ activations and
asynchronous communications. Suppose that the length of any
time window between consecutive “push” steps of any agent
belongs to [pmin, pmax], for some pmax ≥ pmin > 0, and one
agent always sends out its updated information immediately
after the completion of its “push” step. The traveling time of
each packet is at most Dtv. Also, at least one packet is success-
fully received every Dls successive one-hop communications.
Note that there is a vast literature on how to estimate Dtv and
Dls, based upon the specific channel model under consider-
ation; see, e.g., [39], [40]. In this setting, it is not difficult
to check that one can set T = (I − 1) ⌈pmax/pmin⌉ + 1 and
D = I ⌈Dtv/pmin⌉Dls. To cope with the issue of estimating
γ1 and γ2, in the next section we show how to employ in
ASY-SONATA diminishing, uncoordinated stepsizes.
B. Uncoordinated diminishing step-sizes
The use of a diminishing stepsize shared across the agents
is quite common in synchronous distributed algorithms. How-
ever, it is not clear how to implement such option in an asyn-
chronous setting, without enforcing any coordination among
the agents (they should know the global iteration counter k).
In this section, we provide for the first time a solution to this
issue. Inspired by [41], our model assumes that each agent,
independently and with no coordination with the others, draws
the step-size from a local sequence {αt}t∈N0, according to its
local clock. The sequence {γk}k∈N0in (11) will be thus the
result of the “uncoordinated samplings” of the local out-of-
sync sequences {αt}t∈N0. The next theorem shows that in this
setting, ASY-SONATA converges at a sublinear rate for both
convex and nonconvex objectives.
Theorem 11. Consider Problem (P) under Assumption 1
(thus possibly nonconvex). Let {(xki )
Ii=1}k∈N0
be the sequence
generated by Algorithm 2, in the same setting of Theorem 9,
but with the agents using a local step-size sequence {αt}t∈N0
satisfying αt ↓ 0 and∑∞
t=0 αt = ∞. Given δ > 0, let Tδ be
the first iteration k ∈ N0 such that MF (xk) ≤ δ. Then
Tδ ≤ inf{k ∈ N0
∣∣∣k∑
t=0
γt ≥ c/δ}, (15)
where c is a positive constant.
Proof. See Sec. VIII.
V. NUMERICAL RESULTS
We test ASY-SONATA on the least square regression and
the binary classification problems. The MATLAB code can be
found at https://github.com/YeTian-93/ASY-SONATA.
0 100 200 300 400 500 600 700 800
Number of rounds
10-8
10-6
10-4
10-2
100
Opt
imal
ity g
ap
SONATAASYSONATAASYSONATA-dimi
Figure 1. Directed graphs: optimality gap Jk versus number of rounds.
A. Least square regression
In the LS problem, each agent i aims to estimate an
unknown signal x0 ∈ Rn through linear measurements bi =Mix0 + ni, where Mi ∈ Rdi×n is the sensing matrix, and
ni ∈ Rdi is the additive noise. The LS problem can be written
in the form of (P), with each fi(x) = ‖Mix− bi‖2.
Data: We fix x0 with its elements being i.i.d. random
variables drawn from the standard normal distribution. For
each Mi, we firstly generate all its elements as i.i.d. random
variables drawn from the standard normal distribution, and
then normalize the matrix by multiplying it with the reciprocal
of its spectral norm. The elements of the additive noise ni
are i.i.d. Gaussian distributed, with zero mean and variance
equal to 0.04. We set n = 200 and di = 30 for each agent.
Network model: We simulate a network of I = 30 agents.
Each agent i has 3 out-neighbors; one of them belongs to
a directed cycle graph connecting all the agents while the
other two are picked uniformly at random. Asynchronous
model: Agents are activated according to a cyclic rule where
the order is randomly permuted at the beginning of each
round. Once activated, every agent performs all the steps as in
Algorithm 2 and then sends its updates to all its out-neighbors.
Each transmitted message has (integer) traveling time which
is drawn uniformly at random within the interval [0, Dtv]. We
set Dtv = 40.We test ASY-SONATA with a constant step size γ = 3.5,
and also a diminishing step-size rule with each agent updating
its local step size according to αt+1 = αt (1− 0.001 · αt) and
α0 = 3.5; as benchmark, we also simulate its synchronous
instance, with step size γ = 0.8. In Fig. 1, we plot Jk ,
(1/I)
√∑Ii=1 ‖xk
i − x⋆‖22 versus the number of rounds (one
round corresponds to one update of all the agents). The curves
are averaged over 100 Monte-Carlo simulations, with different
graph and data instantiations. The plot clearly shows linear
convergence of ASY-SONATAwith a constant step-size.
B. Binary classification
In this subsection, we consider a strongly convex and
nonconvex instance of Problem (P) over digraphs, namely:
the regularized logistic regression (RLR) and the robust clas-
sification (RC) problems. Both formulations can be abstracted
as:
minx
1
|D|I∑
i=1
∑
j∈Di
V (yj · ℓx(uj)) + λ ‖∇ℓx(·)‖22 , (16)
8
where D = ∪Ii=1Di is the set of indices of the data distributed
across the agents, with agent i owning Di, and Di ∩ Dl = ∅,for all i 6= l; uj and yj ∈ {−1, 1} are the feature vector and
associated label of the j-th sample in D; ℓx(·) is a linear
function, parameterized by x; and V is the loss function.
More specifically, if the RLR problem is considered, V reads
V (r) = 11+e−r while for the RC problem, we have [42]
V (r) =
0, if r > 1;14r
3 − 34r +
12 , if − 1 ≤ r ≤ 1;
1, if r < −1.Data: We use the following data sets for the RLR and
RC problems. (RLR): We set ℓx(u) = x⊤u, n = 100,
each |Di| = 20, and λ = 0.01. The underlying statistical
model is the following: We generated the ground truth x
with i.i.d. N (0, 1) components; each training pair (uj , yj)is generated independently, with each element of uj being
i.i.d. N (0, 1) and yj is set as 1 with probability V (ℓx(uj)),and −1 otherwise. (RC): We use the Cleveland Heart Disease
Data set with 14 features [43], preprocessing it by deleting
observations with missing entries, scaling features between 0-
1, and distributing the data to agents evenly. We set ℓx(u) =e⊤15x +
∑14d=1 e
⊤d xe⊤d u. Network model: We simulated a
digraph of I = 30 agents. Each agent has 7 out-neighbors;
one of them belongs to a directed cycle connecting all the
agents while the other 6 are picked uniformly at random. One
row and one column stochastic matrix with uniform weights
are generated. Asynchronous model: a) Activation lists are
generated by concatenating random rounds. To generate one
round, we first sample its length uniformly from the interval
[I, T ], with T = 90. Within a round, we first have each agent
appearing exactly once and then sample agents uniformly for
the remaining spots. Finally a random shuffle of the agents
order is performed on each round; b) Each transmitted message
has (integer) traveling time which is sampled uniformly from
the interval [0, Dtv], with Dtv = 90.
We compare the performance of our algorithm with Asy-
SubPush [44] and AsySPA [45], which appeared online during
the revision process of our paper. AsySubPush and AsySPA
differ from ASY-SONATA in the following aspects: i) they do
not employ any gradient tracking mechanism; ii) they cannot
handle packet losses and purge out old information from the
system (information is used as it is received); iii) when Fis strongly convex, they provably converge at sublinear rate;
and iv) they cannot handle nonconvex F . The step sizes of
all algorithms are manually tuned to obtain the best practical
performance. We run two instances of ASY-SONATA, one
employing a constant step size γ = 0.4 and the other one using
the diminishing step size rule αt+1 = αt (1− 0.001 · αt),where α0 = 0.5 and t is the local iteration counter. For Asy-
SubPush (resp. AsySPA) we set, for each agent i, αi = 0.0001(resp. ρ(k) = c/
√k with c = 0.01) in RLC and αi = 0.00001
(resp. ρ(k) = c/√k with c = 0.001) in RC. The result is
averaged over 20 Monte Carlo experiments with different di-
graph instances, and is presented in Fig. 2; for each algorithm,
we plot the merit functions Msc (left panel) and MF (right
panel) evaluated in the generated trajectory versus the global
iteration counter k. Consistently with the convergence theory,
Global iterations×104
0 2 4 6 8
Msc
10-10
10-5
100
ASYSONATAASYSONATA-dimiAsySubPushAsySPA
Global iterations×104
0 2 4 6 8
MF
10-20
10-10
100
1010
ASYSONATAASYSONATA-dimiAsySubPushAsySPA
Figure 2. L: regularized logistic regression; R: robust classification.
ASY-SONATA with a constant step size exhibits a linear
convergence rate. Also, ASY-SONATA outperforms the other
two algorithms; this is mainly due to i) the presence in ASY-
SONATA of an asynchronous gradient tracking mechanism
which provides, at each iteration, a better estimate of ∇F ; and
ii) the possibility in ASY-SONATA to discard old information
when received after a newer one [cf. (5)].
VI. CONVERGENCE ANALYSIS OF P-ASY-SUM-PUSH
We prove Theorem 7; we assume n = 1, without loss of
generality. The proof is organized in the following two steps.
Step 1: We first reduce the asynchronous agent system to
a synchronous “augmented” one with no delays. This will
be done adding virtual agents to the graph G along with
their state variables, so that P-ASY-SUM-PUSH will be
rewritten as a (synchronous) perturbed push-sum algorithm
on the augmented graph. While this idea was first explored in
[30], [31], there are some important differences between the
proposed enlarged systems and those used therein, see Remark
13. Step 2: We conclude the proof establishing convergence
of the perturbed push-sum algorithm built in Step 1.
A. Step 1: Reduction to a synchronous perturbed push-sum
1) The augmented graph: We begin constructing the aug-
mented graph–an enlarged agent system obtained adding vir-
tual agents to the original graph G = (V , E). Specifically, we
associate to each edge (j, i) ∈ E an ordered set of virtual nodes
(agents), one for each of the possible delay values, denoted
with a slight abuse of notation by (j, i)0, (j, i)1, . . . , (j, i)D;
see Fig. 3. Roughly speaking, these virtual nodes store the
“information on fly” based upon its associated delay, that is,
the information that has been generated by j ∈ N ini for i
but not used (received) by i yet. Adopting the terminology
in [31], nodes in the original graph G are termed computing
agents while the virtual nodes will be called noncomput-
ing agents. With a slight abuse of notation, we define the
set of computing and noncomputing agents as V , V ∪{(i, j)d| (i, j) ∈ E , d = 0, 1, . . . , D}, and its cardinality as
S ,∣∣∣V∣∣∣ = (I + (D + 1) |E|). We now identify the neighbors
of each agent in this augmented systems. Computing agents
no longer communicate among themselves; each j ∈ V can
only send information to the noncomputing nodes (j, i)0, with
i ∈ N outj . Each noncomputing agent (j, i)d can either send
information to the next noncomputing agent, that is (j, i)d+1
(if any), or to the computing agent i; see Fig. 3(b).
9
(i, ℓ)2 (i, ℓ)1 (j, i)0(j, i)1(j, i)2
ℓ i j
(i, ℓ)0
ℓ i j
(a) Snapshot of the original graph
(b) Augmented graph associated with (a)
Figure 3. Example of augmented graph, when the maximum delay D = 2;three noncomputing agents are added for each edge (j, i) ∈ E .
To describe the information stored by the agents in the
augmented system at each iteration, let us first introduce the
following quantities: Ti ,{k∣∣ ik = i, k ∈ N0
}is the set of
global iteration indices at which the computing agent i ∈ Vwakes up; and, given k ∈ N0, let T k
i ,{t ∈ Ti
∣∣ t ≤ k}
. It is
not difficult to conclude from (7) and (8) that
ρkij =∑
t∈T k−1
j
aijzt+1/2j and ρkij =ρ
τk−1
ij
ij , (j, i) ∈ E . (17)
At iteration k = 0, every computing agent i stores z0i , whereas
the values of the noncomputing agents are initialized to 0. At
the beginning of iteration k, every computing agent i will store
zki whereas every noncomputing agent (j, i)d, with 0 ≤ d ≤D − 1, stores the mass aijzj (if any) generated by j for i at
iteration k−d−1 (thus k−d−1 ∈ T k−1j ), i.e., aijz
k−(d+1)+1/2j
(cf. Step 3.2), and not been used by i yet (thus k−d > τk−1ij );
otherwise it stores 0. Formally, we have
zk(j,i)d ,aijzt+1/2j
· 1[t = k − d− 1 ∈ T k−1
j & t+ 1 > τk−1ij
]. (18)
The virtual node (j, i)D cumulates all the masses
aijzk−(d+1)+1/2j with d ≥ D, not received by i yet:
zk(j,i)D ,∑
t∈T k−D−1
j , t+1>τk−1
ij
aijzt+1/2j .
(19)
We write next P-ASY-SUM-PUSH on the augmented graph
in terms of the z-variables of both the computing and noncom-
puting agents, absorbing the (ρ, ρ)-variables using (17)-(19).
The sum-step over the augmented graph. In the sum-step,
the update of the z-variables of the computing agents reads:
zk+ 1
2
ik= zkik +
∑
j∈N in
ik
(ρτk
ikj
ikj− ρkikj
)+ ǫk
(17)−(19)= zkik +
∑
j∈N in
ik
D∑
d=k−τk
ikj
zk(j,ik)d + ǫk; (20a)
zk+ 1
2
j = zkj , j ∈ V \ {ik}. (20b)
In words, node ik builds the update zkik → zk+ 1
2
ikbased
upon the masses transmitted by the noncomputing agents
(j, ik)k−τk
ikj , (j, ik)k−τk
ikj+1, . . . , (j, ik)D [cf. (20a)]. All the
(ik, ℓ)2 (ik, ℓ)1 (j, ik)0
(j, ik)1(j, ik)2
111 1
1
ℓ ik
1
j
1
(ik, ℓ)0
1
1
Figure 4. Sum step on the augmented graph: τkikj
= k− 1 (delay one); the
two noncomputing agents, (j, ik)1 and (j, ik)2, send their masses to ik.
(ik, ℓ)2 (ik, ℓ)1 (j, ik)0(j, ik)1(j, ik)2
1
ℓ ik
1
j
1
(ik, ℓ)0
1
1
111
aikik
aℓik
Figure 5. Push step on the augmented graph: Agent ik keeps aikikzk+1/2
ik
while sending aℓikzk+1/2
ikto the virtual nodes (ik , ℓ)0, ℓ ∈ N out
ik.
other computing agents keep their masses unchanged [cf.
(20b)]. The updates of the noncomputing agents is set to
zk+ 1
2
(j,ik)d, 0, d = k − τkikj , . . . , D, j ∈ N in
ik ; (20c)
zk+ 1
2
(j′,i)τ , zk(j′,i)τ , for all the other (j′, i)τ ∈ V . (20d)
The noncomputing agents in (20c) set their variables to zero
(as they transferred their masses to ik) while the other non-
computing agents keep their variables unchanged [cf. (20d)].
Fig. 4 illustrates the sum-step over the augmented graph.
The push-step over the augmented graph. In the push-step,
the update of the z-variables of the computing agents reads:
zk+1ik
= aikik zk+ 1
2
ik; (21a)
zk+1j = z
k+ 1
2
j , for j ∈ V \ {ik}. (21b)
In words, agent ik keeps the portion aikikzk+ 1
2
ikof the new
generated mass [cf. (21a)] whereas the other computing agents
do not change their variables [cf. (21b)]. The noncomputing
agents update as:
zk+1(ik,ℓ)0
, aℓik zk+1/2
ik, ℓ ∈ N out
ik ; (21c)
zk+1(i,j)0 , 0, (i, j) ∈ E , i 6= ik; (21d)
zk+1(i,j)d
, zk+ 1
2
(i,j)d−1 , d = 1, . . . , D − 1, (i, j) ∈ E ; (21e)
zk+1(i,j)D
, zk+ 1
2
(i,j)D+ z
k+ 1
2
(i,j)D−1 , (i, j) ∈ E . (21f)
In words, the computing agent ik pushes its masses aℓikzk+ 1
2
ik
to the noncomputing agents (ik, ℓ)0, with ℓ ∈ N outik [cf.
(21c)]. As the other noncomputing agents (i, j)0, i 6= ik, do
not receive any mass for their associated computing agents,
they set their variables to zero [cf. (21d)]. Finally the other
noncomputing agents (i, j)d, with 0 ≤ d ≤ D − 1, transfers
their mass to the next noncomputing node (j, i)d+1 [cf. (21f),
(21e)]. This push-step is illustrated in Fig. 5.
The following result establishes the equivalence between the
update of the enlarged system with that of Algorithm 1.
Proposition 12. Consider the setting of Theorem 7. The
values of the z-variables of the computing agents in (20)-(21)
coincide with those of the z-variables generated by P-ASY-
SUM-PUSH (Algorithm 1), for all iterations k ∈ N0.
10
Proof. By construction, the updates of the computing agents as
in (20a)-(20b) and (21a)-(21b) coincide with the z-updates in
the sum- and push-steps of P-ASY-SUM-PUSH, respectively.
Therefore, we only need to show that the updates of the
noncomputing agents are consistent with those of the (ρ, ρ)-variables in P-ASY-SUM-PUSH. This follows using (17) and
noting that the updates (21c)-(21f) are compliant with (18)
and (19). For instance, by (17)-(18), it must be zk+1(ik,j)0
=
ajikzt+1/2j · 1[t = k ∈ T k
ik and t + 1 > τkjik ] = ajikzk+1/2j ,
which in fact coincides with (21c). The other equations (21d)–
(21f) can be similarly validated.
Proposition 12 opens the way to study convergence of
P-ASY-SUM-PUSH via that of the synchronous perturbed
push-sum algorithm (20)-(21). To do so, it is convenient to
rewrite (20)-(21) in vector-matrix form, as described next.
We begin introducing an enumeration rule for the compo-
nents of the z-vector in the augmented system. We enumerate
all the elements of E as 1, 2, . . . , |E| . The computing agents in
V are indexed as in V , that is, 1, 2, . . . , I . Each noncomputing
agent (j, i)d is indexed as I + d |E| + s, where s is the
index associated with (j, i) in E ; we will use interchangeably
zI+d|E|+s and z(j,i)d . We define the z-vector as z = [zi]Si=1;
and its value at iteration k ∈ N0 is denoted by zk.
The transition matrix Sk of the sum step is defined as
Skhm ,
1, if m ∈ {(j, ik)d | k − τkikj ≤ d ≤ D}and h = ik;
1, if m ∈ V \ {(j, ik)d | k − τkikj ≤ d ≤ D}and h = m;
0, otherwise.
Let εk , ǫkeik be the S−dimensional perturbation vector.
The sum-step can be written in compact form as
zk+1
2 = Skzk + εk. (22)
Define the transition matrix Pk of the push step as
P khm ,
ajik , if m = ik and h = (j, ik)0, j ∈ N outik ;
aikik , if m = h = ik;
1, if m = h ∈ V \ ik;1, if m = (i, j)d, h = (i, j)d+1,
(i, j) ∈ E , 0 ≤ d ≤ D − 1;
1, if m = h = (i, j)D, (i, j) ∈ E ;0, otherwise
Then, the push-step can be written as
zk+1 = Pkzk+1
2 . (23)
Combing (22) and (23), yields
zk+1 = Akzk + pk, Ak , PkSk, pk , Pkεk. (24)
The updates of the φ variables and the definition of the φ-
vector are similar as above. In summary, the P-ASY-SUM-
PUSH algorithm can be rewritten in compact form as
zk+1 = Ak zk + pk, pk = ǫk Pk eik ; (25a)
φk+1 = Akφk; (25b)
with initialization: z0i ∈ R and φ0i = 1, for i ∈ V ; and z0i = 0and φ0i = 0, for i ∈ V \ V .
Remark 13 (Comparison with [30]–[32], [38]). The idea of re-
ducing asynchronous (consensus) algorithms into synchronous
ones over an augmented system was already explored in [31],
[32], [38]. However, there are several important differences
between the models therein and the proposed augmented
graph. First of all, [38] extends the analysis in [30] to deal
with asynchronous activations, but both work consider only
packet losses (no delays). Second, our augmented graph model
departs from that in [31], [32] in the following aspects: i) in
our model, the virtual nodes are associated with the edges of
the original graph rather than the nodes; ii) the noncomputing
nodes store the information on fly (i.e., generated by a sender
but not received by the intended receiver yet), while in [31],
[32], each noncomputing agent owns a delayed copy of the
message generated by the associated computing agent; and iii)
the dynamics (25) over the augmented graph used to describe
the P-ASY-SUM-PUSH procedure is different from those of the
asynchronous consensus schemes [31, (1)] and [32, (1)].
B. Step 2: Proof of Theorem 7
1) Preliminaries: We begin studying some properties of
the matrix product Ak:t, which will be instrumental to prove
convergence of the perturbed push-sum scheme (25).
Lemma 14. Let {Ak}k∈N0be the sequence of matrices in
(25), generated by Algorithm 1, under Assumption 6, and with
A , (aij)Ii,j=1 satisfying Assumption 4 (i),(iii). The following
hold: for all k ∈ N0, a) Ak is column stochastic; and b) the
entries of the first I rows of Ak+K1−1:k are uniformly lower
bounded by η , mK1 ∈ (0, 1), with K1 , (2I−1) ·T +I ·D.
Proof. The lemma essentially proves that (Ak+K1−1:k)⊤ is
a SIA (Stochastic Indecomposable Aperiodic) matrix [32],
by showing that for any time length of K1 iterations, there
exists a path from any node m in the augmented graph to
any computing node h. While at a high level the proof shares
some similarities with that of [31, Lemma 2] and [32, Lemma
5 (a)], there are important differences due to the distinct
modeling of our augmented system. The complete proof is
in Appendix A.
The key result of this section is stated next and shows that
as k − t increases, Ak:t approaches a column stochastic rank
one matrix at a linear rate. Given Lemma 14, the proof follows
the path of [31, Lemma 4, Lemma 5], [32, Lemma 4, Lemma
5(b,c)] and thus is omitted.
Lemma 15. In the setting above, there exists a sequence of
stochastic vectors {ξk}k∈N0such that, for any k ≥ t ∈ N0
and i, j ∈ {1, · · · , S}, there holds∣∣∣Ak:t
ij − ξki∣∣∣ ≤ Cρk−t, (26)
with
C , 21 + m−K1
1− mK1
, ρ , (1− mK1)1
K1 ∈ (0, 1).
Furthermore, ξki ≥ η, for all i ∈ V and k ∈ N0.
11
2) Proof of Theorem 7: Applying (25) telescopically,
yields: zk+1 = Ak:0z0 +∑k
l=1 Ak:lpl−1 + pk and φk+1 =
Ak:0φ0,which using the column stochasticity of Ak:t, yields
1⊤zk+1 = 1⊤z0 +k∑
l=0
1⊤pl, 1⊤φk+1 = 1⊤φ0 = I. (27)
Using (27) and φk+1i ≥ Iη, for all i ∈ V and k ≥ K1 − 1
[due to Lemma 14(b)], we have: for i ∈ V and k ≥ K1 − 1,∣∣∣∣∣zk+1i
φk+1i
− 1⊤zk+1
I
∣∣∣∣∣ ≤1
Iη
∣∣∣∣∣zk+1i − φk+1
i
I(1⊤zk+1)
∣∣∣∣∣
≤ 1
Iη
∣∣zk+1i − ξki 1⊤zk+1
∣∣+ 1
Iη
∣∣∣∣∣
(ξki −
φk+1i
I
)1⊤zk+1
∣∣∣∣∣
≤ 1
Iη
∣∣zk+1i − ξki 1⊤zk+1
∣∣
+1
Iη
∣∣∣∣∣ξki −
Ak:0i,: φ
0
I
∣∣∣∣∣ ·∣∣∣∣∣1
⊤z0 +k∑
l=0
1⊤pl
∣∣∣∣∣(26)
≤ 1
Iη
∣∣zk+1i − ξki 1⊤zk+1
∣∣+ Cρk√Iη
(∥∥z0∥∥+
k∑
l=0
∣∣ǫl∣∣)
(28)
The next lemma provides a bound of∣∣zk+1
i − ξki 1⊤zk+1∣∣ .
Lemma 16. Let {zk}∞k=0 be the sequence generated by the
perturbed system (25a), under Assumption 6, A = (aij)Ii,j=1
satisfying Assumption 4 (i), (iii), and given {ǫk}k∈N0. For any
i ∈ V and k ≥ 0, there holds
∣∣zk+1i − ξki 1⊤zk+1
∣∣ ≤ C0
(ρk∥∥z0∥∥+
k∑
l=0
ρk−l∣∣ǫl∣∣), (29)
with {ξk}n∈N0defined in Lemma 15 and C0 , C
√2S/ρ.
Proof.
∣∣zk+1i − ξki 1⊤zk+1
∣∣ (25a)=
∣∣∣∣∣
(Ak:0
i,: z0 +
k∑
l=1
Ak:li,: p
l−1 + pki
)
− ξki
(1⊤z0 +
k∑
l=0
1⊤pl
) ∣∣∣∣∣ ≤∣∣pki∣∣+∣∣1⊤pk
∣∣
+∥∥∥Ak:0
i,: − ξki 1⊤∥∥∥∥∥z0∥∥+
k∑
l=1
∥∥∥Ak:li,: − ξki 1⊤
∥∥∥∥∥pl−1
∥∥
(26)
≤√S
ρC
(ρk∥∥z0∥∥+
k∑
l=0
ρk−l∥∥Pl
∥∥ ∣∣ǫl∣∣)
(a)
≤ C0
(ρk∥∥z0∥∥+
k∑
l=0
ρk−l∣∣ǫl∣∣),
where in (a) we used∥∥Pl
∥∥ ≤√‖Pl‖1 ‖Pl‖∞ ≤
√2.
Combing Eq. (28) and (29) leads to∣∣∣∣∣zk+1i
φk+1i
− 1⊤zk+1
I
∣∣∣∣∣ ≤ C1
(ρk∥∥z0∥∥+
k∑
l=0
ρk−l∣∣ǫl∣∣),
where we defined C1 , C0 · 2/(I η).Recalling the definition of mk
z ,∑I
i=1 zki +∑
(j,i)∈E(ρkij−
ρkij), to complete the proof, it remains to show that
mkz
(I)=
I∑
i=1
z0i +
k−1∑
t=0
ǫt(II)= 1⊤zk. (30)
We prove next the equalities (I) and (II) separately.
Proof of (I): Since m0z =
∑Ii=1 z
0i , it suffices to show that
mk+1z = m
kz + ǫk for all k ∈ N0. Since agent ik triggers
k → k + 1, we only need to show that
zk+1ik
+∑
j∈N in
ik
(ρk+1ikj− ρk+1
ikj) +
∑
j∈N out
ik
(ρk+1jik− ρk+1
jik)
=zkik +∑
j∈N in
ik
(ρkikj − ρkikj) +∑
j∈N out
ik
(ρkjik − ρkjik ) + ǫk.
We have
zk+1ik
+∑
j∈N in
ik
(ρk+1ikj− ρk+1
ikj) +
∑
j∈N out
ik
(ρk+1jik− ρk+1
jik)
(a)=aikikz
k+ 1
2
ik+∑
j∈N in
ik
(ρkikj − ρτk
ikj
ikj)
+∑
j∈N out
ik
(ρkjik + ajikzk+ 1
2
ik− ρkjik )
(b)=z
k+ 1
2
ik+∑
j∈N in
ik
(ρkikj − ρτk
ikj
ikj) +
∑
j∈N out
ik
(ρkjik − ρkjik)
(c)=zkik +
∑
j∈N in
ik
(ρτk
ikj
ikj− ρkikj) + ǫk +
∑
j∈N in
ik
(ρkikj − ρτk
ikj
ikj)
+∑
j∈N out
ik
(ρkjik − ρkjik )
=zkik +∑
j∈N in
ik
(ρkikj − ρkikj) +∑
j∈N out
ik
(ρkjik − ρkjik ) + ǫk,
where in (a) we used: the definition of the push step, ρk+1ikj
=
ρkikj for all j ∈ N inik , and ρk+1
jik= ρkjik for all j ∈ N out
ik ; (b)
follows from aikik +∑
j∈N out
ikajik = 1; and in (c), we used
the sum-step.
Proof of (II): Using (27), yields 1⊤zk+1 = 1⊤z0 +∑kl=0 1
⊤pl = 1⊤zk + 1⊤εk =∑I
i=1 z0i +
∑kt=0 ǫ
t.
VII. ASY-SONATA–PROOF OF THEOREM 9
We organize the proof in the following steps: Step 1: We
introduce and study convergence of an auxiliary perturbed
consensus scheme, which serves as a unified model for the
descent and consensus updates in ASY-SONATA–the main re-
sult is summarized in Proposition 18; Step 2: We introduce the
consensus and gradient tracking errors along with a suitably
defined optimization error; and we derive bounds connecting
these quantities, building on results in Step 1 and convergence
of P-ASY-SUM-PUSH–see Proposition 19. The goal is to
prove that the aforementioned errors vanish at a linear rate.
To do so, Step 3 introduces a general form of the small
gain theorem–Theorem 23–along with some technical results,
which allows us to establish the desired linear convergence
through the boundedness of the solution of an associated
linear system of inequalities. Step 4 builds such a linear
system for the error quantities introduced in Step 2 and proves
12
the boundedness of its solution, proving thus Theorem 9.
The rate expression (13) is derived in Appendix D. Through
the proof we assume n = 1 (scalar variables); and define
CL , maxi=1,...,I Li and L ,∑I
i=1 Li.
Step 1: A perturbed asynchronous consensus scheme
We introduce a unified model to study the dynamics of the
consensus and optimization errors in ASY-SONATA, which
consists in pulling out the tracking update (Step 5) and treat
the z-variables–the term −γkzkik in (11)–as an exogenous
perturbation δk. More specifically, consider the following
scheme (with a slight abuse of notation, we use the same
symbols as in ASY-SONATA):
vk+1ik
= xkik + δk, (31a)
xk+1ik
= wikikvk+1ik
+∑
j∈N in
ik
wikjvk−dk
j
j , (31b)
vk+1j = vkj , x
k+1j = xkj , ∀j ∈ V \ {ik}, (31c)
with given x0i ∈ R, vti = 0, t = −D,−D + 1, . . . , 0, for all
i ∈ V . We make the blanket assumption that agents’ activations
and delays satisfy Assumption 6.
Let us rewrite (31) in a vector-matrix form. Define xk ,[xk1 , · · · , xkI ]⊤ and vk , [vk1 , · · · , vkI ]⊤. Construct the (D +2)I dimensional concatenated vectors
hk , [xk⊤,vk⊤,vk−1⊤, · · · ,vk−D⊤]⊤, δk , δk eik ; (32)
and the augmented matrix Wk, defined as
W krm ,
wikik , if r = m = ik;
wikj , if r = ik, m = j + (dkj + 1)I;
1, if r = m ∈ {1, 2, . . . , 2I} \ {ik, ik + I};1, if r ∈ {2I + 1, 2I + 2, . . . , (D + 2)I}
∪ {ik + I} and m = r − I;0, otherwise.
System (31) can be rewritten in compact form as
hk+1 = Wk(hk + δk), (33)
The following lemma captures the asymptotic behavior of Wk.
Lemma 17. Let {Wk}k∈N0be the sequence of matrices in
(33), generated by (31), under Assumption 6 and with W
satisfying Assumption 4 (i), (ii). The following hold: for all
k ∈ N0, a) Wk is row stochastic; b) there exists a sequence
of stochastic vectors {ψk}k∈N0such that
∥∥∥Wk:t − 1ψt⊤∥∥∥ ≤ C2ρ
k−t, C2 ,2√(D + 2)I(1 + m−K1)
1− m−K1
(34)
Furthermore, ψki ≥ η = mK1 , for all k ≥ 0 and i ∈ V .
Proof. The proof follows similar techniques as in [31], [32],
and can be found in Appendix G.
We define now a proper consensus error for (33). Writing
hk in (33) recursively, yields
hk+1 = Wk:0h0 +
k∑
l=0
Wk:lδl. (35)
Using Lemma 17, for any fixed N ∈ N0, we have
limk→∞
(Wk:0h0 +N∑
l=0
Wk:lδl) = 1ψ0⊤h0 +N∑
l=0
1ψl⊤δl.
Define
x0ψ , ψ0⊤h0, xk+1ψ , ψ0⊤h0+
k∑
l=0
ψl⊤δl, k ∈ N0. (36)
Applying (36) inductively, it is easy to check that
xk+1ψ = xkψ +ψk⊤δk = xkψ + ψk
ikδk. (37)
We are now ready to state the main result of this subsection,
which is a bound of the consensus disagreement ‖hk+1 −1xk+1ψ ‖ in terms of the magnitude of the perturbation.
Proposition 18. In the above setting, the consensus error
‖hk+1 − 1xk+1ψ ‖ satisfies: for all k ∈ N0,
∥∥∥hk+1 − 1xk+1ψ
∥∥∥ ≤ C2ρk∥∥h0 − 1x0ψ
∥∥+ C2
k∑
l=0
ρk−l∣∣δl∣∣ .
Proof. The proof follows readily from (35), (36), and Lemma
17; we omit further details.
Step 2: Consensus, tracking, and optimization errors
1) Consensus disagreement: As anticipated, the updates of
ASY-SONATA are also described by (31), if one sets therein
δk = −γkzkik (with zkik satisfying Step 5 of ASY-SONATA).
Let hk and xkψ be defined as in (32) and (36), respectively, with
δk = −γkzkik . The consensus error at iteration k is defined as
Ekc ,
∥∥hk − 1xkψ∥∥ . (38)
2) Gradient tracking error: The gradient tracking step in
ASY-SONATA is an instance of P-ASY-SUM-PUSH, with
ǫk = ∇fik(xk+1ik
) − ∇fik(xkik ). By Proposition 12, P-ASY-
SUM-PUSH is equivalent to (25). In view of Lemma 16
and the following property 1⊤zk =∑I
i=1∇fi(x0i ) +∑k−1t=0
(∇fit(xt+1
it )−∇fit(xtit))=∑I
i=1∇fi(xki ) where the
first equality follows from (30) and ǫk = ∇fik(xk+1ik
) −∇fik(xk
ik) while in the second equality we used xt+1j = xtj ,
for j 6= it, the tracking error at iteration k along with the
magnitude of the tracking variables are defined as
Ekt ,
∣∣zkik − ξk−1ik
gk∣∣ , Ek
z ,∣∣zkik∣∣ , gk ,
I∑
i=1
∇fi(xki ), (39)
with ξ−1i , η, i ∈ V . Let gk , [∇f1(xk1), . . . ,∇fI(xkI )]⊤.
3) Optimization error: Let x⋆ be the unique minimizer of
F . Given the definition of consensus disagreement in (38), we
define the optimization error at iteration k as
Eko ,
∣∣xkψ − x⋆∣∣ . (40)
Note that this is a natural choice as, if consensual, all agents’
local variables will converge to a limit point of {xkψ}k∈N0.
4) Connection among Ekc , Ek
t , Ekz , and Ek
o : The following
proposition establishes bounds on the above quantities.
Proposition 19. Let {xk,vk, zk}k∈N0be the sequence gener-
ated by ASY-SONATA, in the setting of Theorem 9, but possibly
with a time-varying step-size {γk}k∈N0. The error quantities
Ekc , Ek
t , Ekz , and Ek
o satisfy: for all k ∈ N0,
13
Ek+1c ≤C2ρ
kE0c + C2
k∑
l=0
ρk−lγlElz. (41a)
Ek+1t ≤3C0CL
k∑
l=0
ρk−l(El
c + γlElz
)+ C0ρ
k∥∥g0
∥∥ ; (41b)
Ekz ≤ Ek
t + CL
√IEk
c + LEko (41c)
Further assume γk ≤ 1/L, k ∈ N0; then
Ek+1o ≤
k∑
l=0
( k∏
t=l+1
(1− τη2γt
))(CL
√IEl
c + Elt
)γl
+
k∏
t=0
(1− τη2γt
)E0
o, (41d)
where η ∈ (0, 1) is defined in Lemma 15 and τ is the strongly
convexity constant of F .
Proof. Eq. (41a) follows readily from Proposition 18.
We prove now (41b). Recall 1⊤zk = gk. Using Lemma 16
with ǫk = ∇fik(xk+1ik
)−∇fik(xkik ), we obtain: for all i ∈ V ,
∣∣zk+1i − ξki gk+1
∣∣
≤ C0
(ρk∥∥g0
∥∥+k∑
l=0
ρk−l∣∣∇f l+1
il−∇f l
il
∣∣)
≤ C0 ρk∥∥g0
∥∥+ C0CL
k∑
l=0
ρk−l∣∣xl+1
il− xlil
∣∣
≤ C0 ρk∥∥g0
∥∥+ C0CL
k∑
l=0
ρk−l∥∥hl+1 − hl
∥∥
= C0ρk∥∥g0
∥∥+ C0CL
k∑
l=0
ρk−l∥∥∥Wl
(hl + δl
)− hl
∥∥∥
(a)= C0 ρ
k∥∥g0
∥∥
+ C0CL
k∑
l=0
ρk−l∥∥∥(Wl − I
)(hl − 1xlψ
)− γlzlilWleil
∥∥∥
≤ C0ρk∥∥g0
∥∥+ C0CL
k∑
l=0
ρk−l
(‖Wl‖γlEl
z
+
(‖Wl‖+ ‖I‖
)El
c
)
(b)
≤ C0ρk∥∥g0
∥∥+ 3C0CL
k∑
l=0
ρk−l(El
c + γlElz
),
where in (a) we used (33) and the row stochasticity
of Wk [Lemma 17(a)]; and (b) follows from ‖Wl‖ ≤√‖Wl‖1‖Wl‖∞ ≤
√3. This proves (41b).
Eq. (41c) follows readily from
Ekz =
∣∣zkik∣∣ ≤
∣∣zkik − ξk−1ik
gk∣∣+ ξk−1
ik
∣∣gk −∇F (xkψ)∣∣
+ ξk−1ik
∣∣∇F (xkψ)−∇F (x⋆)∣∣ .
Finally, we prove (41d). Using (41c) and xk+1ψ = xkψ−γψk
ikzkik
[cf. (37) and recall δk = −γzkik ], we can write
Ek+1o =
∣∣xkψ − γkψkikz
kik − x⋆
∣∣≤ γkψk
ikξk−1ik
∣∣∇F (xkψ)− gk∣∣+ γkψk
ik
∣∣ξk−1ik
gk − zkik∣∣
+∣∣xkψ − γkψk
ikξk−1ik∇F (xkψ)− x⋆
∣∣(a)
≤(1− τη2γk
)Ek
o + CL
√Iγk
∥∥hk − 1xkψ∥∥+ γkEk
t
(b)
≤k∑
l=0
( k∏
t=l+1
(1− τη2γt
))(CL
√IEl
c + Elt
)γl
+
k∏
t=0
(1− τη2γt
)E0
o
where in (a) we used η2 ≤ ψkikξ
k−1ik
< 1 (cf. Lemma 15)
and |x− γ∇F (x)− x⋆| ≤ (1−τγ) |x− x⋆| , which holds for
γ ≤ 1/L; (b) follows readily by applying the above inequality
telescopically.
Step 3: The generalized small gain theorem
The last step of our proof is to show that the error quantities
Ekc , Ek
t , Ekz , and Ek
o vanish linearly. This is not a straight-
forward task, as these quantities are interconnected through
the inequalities (41). This subsection provides tools to address
this issue. The key result is a generalization of the small gain
theorem (cf. Theorem 23), first used in [33].
Definition 20 ( [33]). Given the sequence {uk}∞k=0, a constant
λ ∈ (0, 1), and N ∈ N, let us define
|u|λ,N = maxk=0,...,N
∣∣uk∣∣
λk, |u|λ = sup
k∈N0
∣∣uk∣∣
λk.
If |u|λ is upper bounded, then uk = O(λk), for all k ∈ N0.
The following lemma shows how one can interpret the in-
equalities in (41) using the notions introduced in Definition 20.
Lemma 21. Let {uk}∞k=0, {vki }∞k=0, i = 1, . . . ,m, be non-
negative sequences; let λ0, λ1, . . . , λm ∈ (0, 1); and let
R0, R1, . . . , Rm ∈ R+ such that
uk+1 ≤ R0(λ0)k +
m∑
i=1
Ri
k∑
l=0
(λi)k−lvli, ∀k ∈ N0.
Then, there holds
|u|λ,N ≤ u0 + R0
λ+
m∑
i=1
Ri
λ− λi|vi|λ,N ,
for any λ ∈ ( maxi=0,1,...,m
λi, 1) and N ∈ N.
Proof. See Appendix B.
Lemma 22. Let {uk}∞k=0 and {vk}∞k=0 be two nonnegative
sequences. The following hold
a. uk ≤ vk, for all k ∈ N0 =⇒ |u|λ,N ≤ |v|λ,N , for any
λ ∈ (0, 1) and N ∈ N;b.
|β1u+ β2v|λ,N ≤ |β1| |u|λ,N + |β2| |v|λ,N ,
for any β1, β2 ∈ R, λ ∈ (0, 1), and positive integer N .
14
|Ez|λ,N|Ec|λ,N|Et|λ,N|Eo|λ,N
4
0 b1 1 LC2γλ−ρ 0 0 0b2γλ−ρ
b2λ−ρ 0 0
0 b2γλ−L(γ)
γλ−L(γ) 0
︸ ︷︷ ︸,K
|Ez|λ,N|Ec|λ,N|Et|λ,N|Eo|λ,N
+
0(1 + C2
λ
)E0
c
C0‖g0‖λ + E0
t1+λλ E0
o
, b1 , CL
√I, b2 , 3C0CL. (42)
The major result of this section is the generalized small gain
theorem, as stated next.
Theorem 23. (Generalized Small Gain Theorem) Given non-
negative sequences {uki }∞k=0, i = 1, . . . ,m, a non-negative
matrix T ∈ Rm×m, β ∈ Rm, and λ ∈ (0, 1) such that
uλ,N 4 Tuλ,N + β, ∀N ∈ N, (43)
where uλ,N , [|u1|λ,N , . . . , |um|λ,N ]⊤. If ρ(T) < 1, then
|ui|λ is bounded, for all i = 1, . . . ,m. That is, each ukivanishes at a R-linear rate O(λk).Proof. See Appendix C.
Then following results are instrumental to find a sufficient
condition for ρ(T) < 1.
Lemma 24. Consider a polynomial p(z) = zm − a1zm−1 −a2z
m−2 − . . .− am−1z − am, with z ∈ C and ai ∈ R+, i =1, . . .m. Define zp , max
{|zi|
∣∣ p(zi) = 0, i = 1, . . . ,m}
.
Then, zp < 1 if and only if p(1) > 0.
Proof. See Appendix F.
Step 4: Linear convergence rate (proof of Theorem 9)
Our path to prove linear convergence rate passes through
Theorem 23: we first cast the set of inequalities (41) into a
system in the form (43), and then study the spectral properties
of the resulting coefficient matrix.
Given γ < 1/L, define L(γ) , 1−τη2γ; and choose λ ∈ R
such thatmax (ρ,L(γ)) < λ < 1. (44)
Note that L(γ) < 1, as γ < 1/L; hence (44) is nonempty.
Applying Lemma 21 and Lemma 22 to the set of inequalities
(41) with γk ≡ γ, we obtain the system (42) at the top of the
page. By Theorem 23, to prove the desired linear convergence
rate, it is sufficient to show that ρ(K) < 1. The characteristic
polynomial pK(t) of T satisfies the conditions of Lemma 24;
hence ρ(K) < 1 if and only if pK(1) > 0, that is,((
1 +Lγ
λ− L(γ)
)b2
λ− ρ + b1 +Lb2γ
λ− L(γ)
)C2γ
λ− ρ
+
(1 +
Lγ
λ− L(γ)
)b2γ
λ− ρ , B(λ; γ) < 1.
(45)
By the continuity of B(λ; γ) and (44), B(1; γ) < 1 is suffi-
cient to claim the existence of some λ ∈ (max (ρ,L(γ)) , 1)such that B(λ; γ) < 1. Hence, setting B(1; γ) < 1, yields
0 < γ < γ1, with
γ1 ,τη2(1− ρ)2
(τη2 + L)b2(C2 + 1− ρ) + (b1τη2 + Lb2)C2(1− ρ).
(46)
It is easy to check that γ1 < 1/L. Therefore, 0 < γ < γ1 is
sufficient for Ekc , E
kt , E
kz , E
ko to vanish with an R-Linear rate.
The desired result,∣∣xki − x⋆
∣∣ = O(λk), i ∈ V , follows readily
from Ekc = O(λk) and Ek
o = O(λk). The explicit expression
of the rate λ, as in (13), is derived in Appendix D.
VIII. ASY-SONATA: PROOF OF THEOREMS 10 AND 11
Through the section, we use the same notation as in Sec.VII.
A. Preliminaries
We begin establishing a connection between the merit
function MF defined in (14) and the error quantities Ekc , Ek
t ,
and Ekz , defined in (38), (39), and (40) respectively.
Lemma 25. The merit function MF satisfies
MF (xk) ≤ C3 (E
kc )
2 + 3 η−2((Ek
t )2 + (Ek
z )2), (47)
with C3 , 3C2LI +
3L2
I + 6CLL+ 4.
Proof. Define J , (1/I) · 11⊤ and xk , (1/I) · 1⊤xk; and
recall the definition of ξki (cf. Lemma 15) and that xk+1ψ =
xkψ − γkψkikz
kik . [cf. (37)]. We have
MF (xk) ≤
∣∣∇F (xk)∣∣2 +
∥∥xk − 1xk∥∥2 (48)
≤∣∣∇F (xk)
∣∣2 + 2∥∥xk − 1xkψ
∥∥2 + 2∥∥1xkψ − 1xk
∥∥2 (49)
≤∣∣∇F (xk)
∣∣2 + 2∥∥xk − 1xkψ
∥∥2 + 2∥∥J(1xkψ − xk
)∥∥2
≤∣∣∇F (xk)
∣∣2 + 4∥∥xk − 1xkψ
∥∥2 . (50)
We bound now∣∣∇F (xk)
∣∣; we have
∣∣∇F (xk)∣∣ ≤
∣∣∇F (xkψ)∣∣+ L
∣∣xk − xkψ∣∣
≤∣∣∇F (xkψ)− gk
∣∣+∣∣gk − (ξk−1
ik)−1zkik
∣∣ + (ξk−1ik
)−1∣∣zkik∣∣
+L√I
∥∥J(xk − 1xkψ
)∥∥
≤(CL
√I +
L√I
)Ek
c + η−1 Ekt + η−1Ek
z ,
(51)
where in the last inequality we used ξkik ≥ η for all k (cf.
Lemma 15) and ‖J(xk − 1xkψ)‖ ≤ Ekc .
Eq. (47) follows readily from (50) and (51).
Our ultimate goal is to show that the RHS of (47) is
summable. To do so, we need two further results, Proposi-
tion 26 and Lemma 27 below. Proposition 26 establishes a
connection between F (xk+1ψ ) and Ek
c , Ekt , and Ek
z .
Proposition 26. In the above setting, there holds: k ∈ N0,
F (xk+1ψ ) ≤ F (x0ψ) +
1
2
(L+ α−1 + β−1
) k∑
t=0
(Etz)
2(γt)2
− ηk∑
t=0
(Etz)
2γt +α
2C2
LI
k∑
t=0
(Etc)
2 +β
2η−2
k∑
t=0
(Ett)
2,
(52)
where α and β are two arbitrary positive constants.
Proof. By descent lemma, we get
15
F (xk+1ψ ) ≤
F (xkψ) + γkψkik
⟨∇F (xkψ),−zkik
⟩+L(γkψk
ik)2
2
∣∣zkik∣∣2
≤ F (xkψ) +Lγk
2
2
∣∣zkik∣∣2 + γkψk
ik
⟨(ξk−1
ik)−1zkik ,−zkik
⟩
+ γkψkik
⟨∇F (xkψ)− gk,−zkik
⟩
+ γkψkik
⟨gk − (ξk−1
ik)−1zkik ,−zkik
⟩
≤ F (xkψ) +Lγk
2
2
∣∣zkik∣∣2 − γkη
∣∣zkik∣∣2
+ γkCL
I∑
j=1
∣∣xkψ − xkj∣∣ ∣∣zkik
∣∣+ γkη−1Ekt
∣∣zkik∣∣
≤ F (xkψ) +Lγk
2
2
∣∣zkik∣∣2 − γkη
∣∣zkik∣∣2
+ γkCL
√IEk
c
∣∣zkik∣∣+ γkη−1Ek
t
∣∣zkik∣∣
≤ F (xkψ) +Lγk
2
2
∣∣zkik∣∣2 − γkη
∣∣zkik∣∣2 + α
2C2
LI(Ekc )
2
+α−1
2
∣∣zkik∣∣2 γk2 + β
2η−2(Ek
t )2 +
β−1
2
∣∣zkik∣∣2 (γk)2
≤ F (xkψ) +1
2
(L+ α−1 + β−1
)(Ek
z )2(γk)2
− η(Ekz )
2γk +α
2C2
LI(Ekc )
2 +β
2η−2(Ek
t )2.
Applying the above inequality inductively one gets (52).
The last result we need is a bound of∑k
t=0(Etc)
2 and∑kt=0(E
tt)
2 in (52) in terms of∑k
t=0(Etz)
2(γt)2.
Lemma 27. Define
c ,2C2
2
(1− ρ)2 and t ,36 (C0CL)
2 (2C22 + (1− ρ)2
)
(1− ρ)4 .
The following holds: k ∈ N,k∑
t=0
(Etc)
2 ≤ cc + c
k∑
t=0
(Etz)
2(γt)2,
k∑
t=0
(Ett)
2 ≤ ct + t
k∑
t=0
(Etz)
2(γt)2,
(53)
where cc and ct are some positive constants.
Proof. The proof follows from Proposition 19 and Lemma 28
below, which is a variant of [28] (its proof is thus omitted).
Lemma 28. Let {uk}∞k=0, {vki }∞k=0, i = 1, . . . ,m, be nonneg-
ative sequences; λ ∈ (0, 1); and R0 ∈ R+ such that
uk+1 ≤ Rλk +
k∑
l=0
λk−lvl.
Then, there holds: k ∈ N,k∑
l=0
(ul)2 ≤ (u0)2 +2R2
1− λ2 +2
(1− λ)2k∑
l=0
(vl)2.
Using (53) in (52), we finally obtain
k∑
t=0
(Etz)
2γt(η − γtC4(α, β)) ≤ F (x0ψ)− F inf + C5(α, β)
(54)
with C4(α, β) , (1/2) (L+α−1+β−1+C2LIαc+ η
−2βt)and C5(α, β) = (1/2)
(C2
LIαcc + η−2βct); and F inf > −∞
is the lower bound of F .
We are now ready to prove Theorems 10 and 11.
B. Proof of Theorem 10
Set γk ≡ γ, for all k ∈ N0. By (54), one infers that∑∞t=0 E
tz
2< ∞ if γ satisfies 0 < γ < γ2(α, β), with
γ2(α, β) , η/C4(α, β).Note that γ2(α, β) is maximized
setting α = α⋆ =(CL
√Ic)−1
and β = β⋆ = η−1/2t ,
resulting in
γ2(α⋆, β⋆) = (2η)/(L+ 2CL
√Ic + 2η−1√t). (55)
Let 0 < γ < γ2(α⋆, β⋆). Given δ > 0, let Tδ be the first
iteration k ∈ N0 such that MF (xk) ≤ δ. Then we have
Tδ · δ <Tδ−1∑
k=0
MF (xk) ≤
∞∑
k=0
MF (xk)
(47)
≤ C3
∞∑
k=0
(Ekc )
2 + 3η−2∞∑
k=0
((Ek
t )2 + (Ek
z )2)
(53),(54)
≤F (x0ψ)− F inf + C5(α
⋆, β⋆)
γ(η − γC4(α⋆, β⋆))· C6 + C7 <∞
where C6 , C3c(γ)2 + 3η−2
(t(γ)
2 + 1)
and C7 is some
constant. Therefore, Tδ = O(1/δ).
C. Proof of Theorem 11.
We begin showing that the step-size sequence {γt}t∈N0
induced by the local step-size sequence {αt}t∈N0and the asyn-
chrony mechanism satisfying Assumption 6 is nonsummable.
The proof is straightforward and is thus omitted.
Lemma 29. Let {γt}t∈N0be the global step-size sequence
resulted from Algorithm 2, under Assumption 6. Then, there
hold: limt→∞ γt = 0 and∑∞
t=0 γt =∞.
Since limt→∞ γt = 0, there exists a sufficiently large k ∈N, say k, such that η − γkC4(α
⋆, β⋆) ≥ η/2 for all k > k.
It is not difficult to check that this, together with (54), yields∑∞k=0(E
kz )
2γk <∞. We can then write
∞∑
k=0
MF (xk)γk
(47)
≤ C3
∞∑
k=0
(Ekc )
2γk + 3η−2∞∑
k=0
((Ek
t )2 + (Ek
z )2)γk < C8,
(56)
for some finite constant C8, where in the last inequality we
used (53),∑∞
k=0(Ekz )
2γk <∞ and limt→∞ γt = 0.
Let Nδ , inf{k ∈ N0 :
∑kt=0 γ
t ≥ C8/δ}
. Note that Nδ
exists, as∑∞
k=0 γk =∞ (cf. Lemma 29). Let Tδ , inf
{k ∈
N0 :MF (xk) ≤ δ
}. It must be Tδ ≤ Nδ. In fact, suppose by
contradiction that Tδ > Nδ; and thus MF (xk) > δ, for 0 ≤
k ≤ Nδ. It would imply∑Nδ
k=0MF (xk)γk > δ
∑Nδ
k=0 γk ≥
δ · (C8/δ) = C8, which contradicts (56). This proves (15).
16
IX. CONCLUSIONS
We proposed ASY-SONATA, a distributed asynchronous
algorithmic framework for convex and nonconvex (uncon-
strained, smooth) multi-agent problems, over digraphs. The
algorithm is robust against uncoordinated agents’ activa-
tion and (communication/computation) (time-varying) delays.
When employing a constant step-size, ASY-SONATA achieves
a linear rate for strongly convex objectives–matching the rate
of a centralized gradient algorithm–and sublinear rate for
(non)convex problems. Sublinear rate is also established when
agents employ uncoordinated diminishing step-sizes, which
is more realistic in a distributed setting. To the best of our
knowledge, ASY-SONATA is the first distributed algorithm
enjoying the above properties, in the general asynchronous
setting described in the paper.
APPENDIX
A. Proof of Lemma 14
We study any entry Ak+K1−1hm with m ∈ V and h ∈ V . We
prove the result by considering the following four cases.
(i) Assume h = m ∈ V . It is easy to check that Akhh ≥ m,
for any k ∈ N0 and h ∈ V . Therefore, Ak+s−1:khh ≥∏k+s−1
t=k Athh ≥ ms, for all k, s ∈ N0 and h ∈ V .
(ii) Let (m,h) ∈ E ; and let s be the first time m wakes up in
the interval [k, k + T − 1]. We have As(m,h)0,m = ahm. The
information that node m sent to node (m,h)0 at iteration s is
received by node h when the information is on some virtual
node (m,h)d. We discuss separately the following three sub-
cases for d: 1) 1 ≤ d ≤ D − 1; 2) d = 0; and 3) d = D.
1) 1 ≤ d ≤ D − 1: We have
As+d:s+1(m,h)d,(m,h)0
= As+d(m,h)d,(m,h)d−1 · · · As+1
(m,h)1,(m,h)0 = 1,
As+d+1h,(m,h)d
= ahh.
Therefore, As+d+1:shm = As+d+1
h,(m,h)dAs+d:s+1
(m,h)d,(m,h)0As
(m,h)0,m =
ahhahm ≥ m2.
2) d = 0: We simply have
As+1:shm = As+1
h,(m,h)0As(m,h)0,m = ahhahm ≥ m2.
Therefore, for 0 ≤ d ≤ D − 1,
Ak+2T+D−1:khm = Ak+2T+D−1:s+d+2
hh As+d+1:shm As−1:k
mm
≥ mk+2T+D−s−d−2m2ms−k ≥ m2T+D.
3) d = D: Before agent j wakes up at time s + D + τ ,
where 1 ≤ τ ≤ T , the information will stay on virtual nodes
(m,h)D. Once agent j wakes up, nodes (m,h)D will send all
its information to it. Then we have
As+D:s+1(m,h)D,(m,h)0 = 1, As+D+τ :s+D+1
h,(m,h)D = ahh.
Similarly, we have
Ak+2T+D−1hm = Ak+2T+D−1:s+D+τ+1
hh As+D+τ :s+D+1h,(m,h)D
· As+D:s+1(m,h)D ,(m,h)0A
s(m,h)0,mA
s−1:kmm ≥ m2T+D.
To summarize, in all of the three sub-cases, we have
Ak+K1−1hm ≥ Ak+K1−1:k+2T+D
hh Ak+2T+D−1:khm
≥ mK1−2T−Dm2T+D = mK1 .
(iii) Let m 6= h and (m,h) ∈ V × V \ E . Since
the graph (V , E) is connected, there are mutually dif-
ferent agents i1, . . . , ir, with r ≤ I − 2, such that
(m, i1), (i1, i2), . . . , (ir−1, ir), (ir, h) ⊂ E , which is actually
a directed path from m to h in the graph (V , E). Then, by
result proved in (ii), we have
Ak+(I−1)(2T+D)−1:khm ≥ Ak+(I−1)(2T+D)−1:k+(r+1)(2T+D)
hh
· Ak+(r+1)(2T+D)−1:k+r(2T+D)hir
· · · Ak+2T+D−1:ki1m
≥ m(I−r−2)(2T+D)m(r+1)(2T+D) = m(I−1)(2T+D).
We can then easily get Ak+K1−1:khm =
Ak+K1−1:k+(I−1)(2T+D)hh A
k+(I−1)(2T+D)−1:khm ≥ mK1 .
(iv) If m is a virtual node, it must be associated with an edge
(j, i) ∈ E and there exists 0 ≤ d ≤ D such that m = (j, i)d. A
similar argument as in (ii) above shows that any information
on m will eventually enter node i taking 1 ≤ τ ≤ D + T .
That is, Ak+τ−1:kim = aii, for some 1 ≤ τ ≤ D + T . On the
other hand, by the above results, we know
Ak+T+D+(I−1)(2T+D)−1:k+T+Dhi ≥ m(I−1)(2T+D).
Therefore,
Ak+K1−1:khm ≥ Ak+K1−1:k+T+D
hi Ak+T+D−1:k+τii Ak+τ−1:k
im
≥ m(I−1)(2T+D)mT+D−τ m ≥ mK1
B. Proof of Lemma 21
Fix N ∈ N, and let k such that 1 ≤ k + 1 ≤ N . We have:
uk+1
λk+1≤ R0
λ
(λ0λ
)k
+m∑
i=1
Ri
λ
k∑
l=0
(λiλ
)k−lvliλl
≤ R0
λ+
m∑
i=1
Ri
λ|vi|λ,N
k∑
l=0
(λiλ
)k−l
≤ R0
λ+
m∑
i=1
Ri
λ− λi|vi|λ,N .
Hence,
|u|λ,N ≤ max
(u0,
R0
λ+
m∑
i=1
Ri
λ− λi|vi|λ,N
)
≤ u0 + R0
λ+
m∑
i=1
Ri
λ− λi|vi|λ,N .
C. Proof of Theorem 23
From [46, Ch. 5.6], we know that if ρ(T) < 1, then
limk→∞ Tk = 0, the series∑∞
k=0 Tk converges (wherein we
define T0 , I), I−T is invertible and∑∞
k=0 Tk = (I−T)−1.
Given N ∈ N, using (43) recursively, yields: uλ,N ≤Tuλ,N +β ≤ T
(Tuλ,N + β
)+β = T2uλ,N +(T+ I)β ≤
· · · ≤ Tℓuλ,N +∑ℓ−1
k=0 Tkβ, for any ℓ ∈ N. Let ℓ → ∞,
we get uλ,N ≤ (I − T)−1β. Since this holds for any given
N ∈ N, we have uλ ≤ (I − T)−1β. Hence, uλ is bounded,
and thus each uki vanishes at an R-linear rate O(λk).
17
D. Proof of the rate decay (13) (Theorem 9)
Let λ ≥ L(γ)+ǫγ, with ǫ > 0 to be properly chosen. Then,
B(λ; γ) ≤(1 +
L
ǫ
)b2γ
λ− ρ
+
((1 +
L
ǫ
)b2
λ− ρ + b1 +Lb2ǫ
)C2γ
λ− ρ .(57)
Using λ− ρ < 1, a sufficient condition for Eq. (45) is [RHS
less than one](b1C2 +
Lb2C2
ǫ+
(1 +
L
ǫ
)b2(1 + C2)
)γ ≤ (λ− ρ)2 .
(58)
Now set ǫ = (τη2)/2. Since the RHS of the above inequality
can be arbitrarily close to (1− ρ)2, an upper bound of γ is
γ2 ,
(1− ρ)2/(
b1C2 +2Lb2C2
τη2+
(1 +
2L
τη2
)b2(1 + C2)
)
︸ ︷︷ ︸,J1
.
According to λ ≥ L(γ) + ǫγ and (58), we get
λ = max
(1− τη2γ
2, ρ+
√J1γ
). (59)
Notice that when γ goes from 0 to γ2, the first argument inside
the max operator decreases from 1 to 1−(τη2γ2)/2, while the
second argument increases from ρ to 1. Letting 1− τη2γ2 = ρ+
√J1γ, we get the solution as γ1 =
(√J1+2τη2(1−ρ)−√
J1
τη2
)2
.
The expression of λ as in (13) follows readily.
E. Proof of Corollary 7.1
We know that mkz =
∑Ii=1 z
0i +
∑k−1t=0 ǫ
t. Clearly m0z =∑I
i=1 z0i =
∑Ii=1 u
0i . Suppose for k = ℓ, we have that mℓ
z =∑Ii=1 u
ℓi . Then we have that
mℓ+1z = m
ℓz + ǫℓ =
(I∑
i=1
uℓi
)+ uℓ+1
iℓ− uℓiℓ
=∑
j 6=iℓ
uℓj + uℓ+1iℓ
=I∑
i=1
uℓ+1i .
Thus we have that mkz =
∑Ii=1 u
ki , ∀k ∈ N0.
Now we assume that limk→∞∑I
i=1
∣∣uk+1i − uki
∣∣ = 0.Notice that for k ≥ T,
∣∣ǫk∣∣ =∣∣uk+1
ik− ukik
∣∣ ≤k∑
t=k−T+1
∣∣ut+1ik− utik
∣∣
≤k∑
t=k−T+1
I∑
i=1
∣∣ut+1i − uti
∣∣ .
Therefore we have that limk→∞∣∣ǫk∣∣ = 0. According to
Theorem 7 and [29, Lemma 7(a)], we have that
limk→∞
∣∣∣∣∣yk+1i − (1/I) ·
I∑
i=1
uk+1i
∣∣∣∣∣ = 0.
On the other hand, we have that
∣∣∣∣∣I∑
i=1
uk+1i −
I∑
i=1
uk+1i
∣∣∣∣∣ ≤I∑
i=1
∣∣uk+1i − uk+1
i
∣∣
≤I∑
i=1
k∑
t=k−T+1
∣∣ut+1i − uti
∣∣ k→∞−→ 0.
By triangle inequality, we get that
limk→∞
∣∣∣∣∣yk+1i − (1/I) ·
I∑
i=1
uk+1i
∣∣∣∣∣ = 0.
F. Proof of Lemma 24
“⇐=:” From p(1) > 0, we know that∑m
i=1 ai < 1. We
prove by contradiction. Suppose there is a root z∗ of p(z)satisfying |z∗| ≥ 1, then we have
zm∗ = a1zm−1∗ + a2z
m−2∗ + . . .+ am−1z∗ + am.
Clearly z∗ 6= 0, so equivalently
1 = a11
z∗+ a2
1
z2∗+ . . .+ am−1
1
zm−1∗
+ am1
zm∗.
Further,
1 =
∣∣∣∣a11
z∗+ a2
1
z2∗+ . . .+ am−1
1
zm−1∗
+ am1
zm∗
∣∣∣∣
≤ a11
|z∗|+ a2
1
|z∗|2+ . . .+ am−1
1
|z∗|m−1 + am1
|z∗|m
≤ a1 + a2 + . . .+ am−1 + am
< 1.
This is a contradiction.
“=⇒:” If p(1) = 0, we clearly have that zp ≥ 1. Now
suppose p(1) < 0. Because limz∈R,z→+∞ p(z) = +∞ and
p(z) is continuous on R, we know that p(z) has a zero in
(1,+∞) ⊂ R. Thus zp > 1.
G. Proof of Lemma 17
We interpret the dynamical system (33) over an augmented
graph. We begin constructing the augmented graph obtained
adding virtual nodes to the original graph G = (V , E). We
associate each node i ∈ V with an ordered set of virtual nodes
i[0], i[1], . . . , i[D]; see Fig. 6. We still call the nodes in the
original graph G as computing agents and the virtual nodes as
noncomputing agents. We now identify the neighbors of each
agent in this augmented system. Any noncomputing agent i[d],d = D,D − 1, · · · , 1, can only receive information from the
previous virtual node i[d−1]; i[0] can only receive information
from the real node i or simply keep its value unchanged;
computing agents cannot communicate among themselves.
18
ℓ i j
j[2]ℓ[0]ℓ[1]ℓ[2] i[2] i[1] i[0] j[0]j[1]
ℓ i j
(a) Snapshot of the original graph
(b) Augmented graph associated with (a)
Figure 6. Example of augmented graph, when the maximum delay is D = 2;three noncomputing agents are added for each node i ∈ V .
At the beginning of each iteration k, every computing agent
i ∈ V will store the information xki ; whereas every noncom-
puting agent i[d], with d = 0, 1, · · · , D, will store the delayed
information vk−di . The dynamics over the augmented graph
happening in iteration k is described by (33). In words, any
noncomputing agent i[d] with i ∈ V and d = D,D− 1, · · · , 1receives the information from i[d−1]; the noncomputing agent
ik[0] receives the perturbed information xkik+δk from node ik;
the values of noncomputing agents j[0] for j ∈ V\{ik} remain
the same; node ik sets its new value as a weighted average
of the perturbed information xkik + δk and vk−dk
j
j ’s received
from the virtual nodes j[dkj ]’s for j ∈ N inik ; and the values of
the other computing agents remain the same. The dynamics is
further illustrated in Fig. 7. The following Lemma shows that
the product of a sufficiently large number of any instantiations
of the matrix Wk, under Assumption 6, is a scrambling matrix.
ℓ ik j
ℓ[0]ℓ[1]ℓ[2] ik[2] ik[1] ik[0] j[0]j[1]
j[2]
1
wikik
wikj
1 1
1
1 1
1
1 11
1
Figure 7. The dynamics in iteration k. Agent ik uses the delayed information
vk−1
j from the virtual node j[1].
Lemma 30. Let {Wk}k∈N0be the sequence of augmented
matrices generated according to the dynamical system (31),
under Assumption 6, and with W satisfying Assumption 4
(i), (ii). Then, for any k ∈ N0, Wk is row stochastic and
Wk+K1−1:k has the property that all entries of its first Icolumns are uniformly lower bounded by η.
Proof. We study any entry W k+K1−1hm with m ∈ V and h ∈ V.
We prove the result by considering the following four cases.
(i) Assume h = m ∈ V . Since W khh ≥ m for any k ∈ N0 and
any h ∈ V , we have W k+s−1:khh ≥ ∏k+s−1
t=k W thh ≥ ms for
∀ k ∈ N0, ∀ s ∈ N and ∀h ∈ V .
(ii) Assume that (m,h) ∈ E . Suppose that the first time when
agent h wakes up during the time interval [k + T + D, k +2T +D− 1] is s, and agent h uses the information vs−d
m from
the noncomputing agent m[d]. Then we have
W s:s−dh,m[0] ≥ W s
h,m[d] · · · W s−dm[1],m[0] = whm ≥ md+1.
Then suppose that the last time when agent m wakes up during
the time interval [s − d − T, s − d − 1] is s − d − t. The
noncomputing agent m[0] receives some perturbed information
from agent m at iteration s − d − t and then performs self-
loop (i.e., keep its value unchanged) during the time interval
[s− d− t+ 1, s− d− 1]. Thus we have
W s−d−1:s−d−tm[0],m = W s−d−1:s−d−t+1
m[0],m[0] W s−d−tm[0],m = 1 · 1 ≥ mt−1.
Therefore we have
W k+2T+D−1:khm ≥ W k+2T+D−1:s+1
hh W s:s−dh,m[0]
· W s−d−1:s−d−tm[0],m W s−d−t−1:k
mm
≥ mk+2T+D−s−1md+1mt−1ms−d−t−k ≥ m2T+D.
Further we have
W k+K1−1hm ≥ W k+K1−1:k+2T+D
hh W k+2T+D−1:khm
≥ mK1−2T−Dm2T+D = mK1 .
(iii) Assume that m 6= h and (m,h) ∈ V ×V \E . Because the
graph (V , E) is connected, there are mutually different agents
i1, . . . , ir with r ≤ I − 2 such that
(m, i1), (i1, i2), . . . , (ir−1, ir), (ir, h) ⊂ E ,which is actually a directed path from m to h. Then, by result
proved in (ii), we have
Wk+(I−1)(2T+D)−1:khm = W
k+(I−1)(2T+D)−1:k+(r+1)(2T+D)hh
Wk+(r+1)(2T+D)−1:k+r(2T+D)hir
· · · W k+2(2T+D)−1:k+2T+Di2i1
· W k+2T+D−1:ki1m
≥ m(I−r−2)(2T+D)m(r+1)(2T+D) = m(I−1)(2T+D).
Then we can easily get
W k+K1−1:khm = W
k+K1−1:k+(I−1)(2T+D)hh W
k+(I−1)(2T+D)−1:khm
≥ mK1−(I−1)(2T+D)m(I−1)(2T+D) = mK1 .
(iv) If h is a noncomputing node, it must be affiliated with a
computing agent j ∈ V , i.e., there exists 0 ≤ d ≤ D such that
h = j[d]. Then we have
W k+K1−1:k+K1−dh,j[0] = W k+K1−1
j[d],j[d−1] · · · Wk+K1−dj[1],j[0] = 1.
Suppose that the last time when agent j wakes up during the
time interval [k+K1 − d− T, k+K1 − d− 1] is s. We have
W k+K1−d−1:sj[0],j = W k+K1−d−1:s+1
j[0],j[0] W sj[0],j = 1.
By results proved before, we have
W k+K1−1:khm ≥ W k+K1−1:k+K1−d
h,j[0] W k+K1−d−1:sj[0],j
· W s−1:k+(I−1)(2T+D)jj W
k+(I−1)(2T+D)−1:kjm
≥ 1 · 1 · ms−k−(I−1)(2T+D)m(I−1)(2T+D) ≥ mK1 .
Based on Lemma 30, we get the following result according
to the discussion in [31].
Lemma 31. In the setting above, there exists a sequence of
stochastic vectors {ψk}k∈N0such that for any k ≥ t ≥ 0,
∥∥∥Wk:t − 1ψt⊤∥∥∥∞≤ 2(1 + m−K1)
1− m−K1
ρk−t.
19
Furthermore, ψki ≥ η = mK1 for all k ≥ 0 and i ∈ V .
The above result leads to Lemma 17 by noticing that∥∥∥Wk:t − 1ψt⊤
∥∥∥ ≤√(D + 2)I
∥∥∥Wk:t − 1ψt⊤∥∥∥∞≤ C2ρ
k−t.
REFERENCES
[1] Y. Tian, Y. Sun, and G. Scutari, “Asy-sonata: Achieving linear conver-gence for distributed asynchronous optimization,” in Proc. of Allerton
2018, Oct. 3-5.[2] ——, “Asy-sonata: Achieving geometric convergence for distributed
asynchronous optimization,” arXiv:1803.10359, Mar. 2018.
[3] D. P. Bertsekas and J. N. Tsitsiklis, Parallel and distributed computation:numerical methods. Englewood Cliffs, NJ: Prentice-Hall, 1989.
[4] A. Nedic, “Asynchronous broadcast-based convex optimization over anetwork,” IEEE Trans. Auto. Contr., vol. 56, no. 6, pp. 1337–1351, 2011.
[5] X. Zhao and A. H. Sayed, “Asynchronous adaptation and learning overnetworks–Part I/Part II/Part III: Modeling and stability analysis/Per-formance analysis/Comparison analysis,” IEEE Trans. Signal Process.,vol. 63, no. 4, pp. 811–858, 2015.
[6] S. Kumar, R. Jain, and K. Rajawat, “Asynchronous optimization overheterogeneous networks via consensus admm,” IEEE Trans. Signal Inf.
Process. Netw., vol. 3, no. 1, pp. 114–129, 2017.[7] M. Eisen, A. Mokhtari, and A. Ribeiro, “Decentralized quasi-newton
methods,” IEEE Trans. Signal Process., vol. 65, no. 10, pp. 2613–2628,2017.
[8] N. Bof, R. Carli, G. Notarstefano, L. Schenato, and D. Varagnolo,“Newton-raphson consensus under asynchronous and lossy communi-cations for peer-to-peer networks,” arXiv:1707.09178, 2017.
[9] Z. Peng, Y. Xu, M. Yan, and W. Yin, “Arock: an algorithmic frameworkfor asynchronous parallel coordinate updates,” SIAM J. Sci. Comput.,vol. 38, no. 5, pp. A2851–A2879, 2016.
[10] T. Wu, K. Yuan, Q. Ling, W. Yin, and A. H. Sayed, “Decentralizedconsensus optimization with asynchrony and delays,” IEEE Trans. Signal
Inf. Process. Netw., vol. PP, no. 99, 2017.[11] J. N. Tsitsiklis, D. P. Bertsekas, and M. Athans, “Distributed asyn-
chronous deterministic and stochastic gradient optimization algorithms,”IEEE Trans. Auto. Contr., vol. 31, no. 9, pp. 803–812, 1986.
[12] J. Liu and S. J. Wright, “Asynchronous stochastic coordinate descent:Parallelism and convergence properties,” SIAM J. Optim., vol. 25, no. 1,pp. 351–376, 2015.
[13] L. Cannelli, F. Facchinei, V. Kungurtsev, and G. Scutari, “Asynchronousparallel algorithms for nonconvex optimization,” Math. Prog., June 2019.
[14] F. Niu, B. Recht, C. Re, and S. J. Wright, “Hogwild: a lock-free approachto parallelizing stochastic gradient descent,” in Proc. of NIPS 2011, pp.693–701.
[15] X. Lian, Y. Huang, Y. Li, and J. Liu, “Asynchronous parallel stochasticgradient for nonconvex optimization,” in Proc. of NIPS 2015, pp. 2719–2727.
[16] I. Notarnicola and G. Notarstefano, “Asynchronous distributed optimiza-tion via randomized dual proximal gradient,” IEEE Trans. Auto. Contr.,vol. 62, no. 5, pp. 2095–2106, 2017.
[17] J. Xu, S. Zhu, Y. C. Soh, and L. Xie, “Convergence of asynchronousdistributed gradient methods over stochastic networks,” IEEE Trans.
Auto. Contr., vol. 63, no. 2, pp. 434–448, 2017.[18] F. Iutzeler, P. Bianchi, P. Ciblat, and W. Hachem, “Asynchronous
distributed optimization using a randomized alternating direction methodof multipliers,” in Proc. of CDC 2013, pp. 3671–3676.
[19] E. Wei and A. Ozdaglar, “On the o(1/k) convergence of asynchronousdistributed alternating direction method of multipliers,” in Proc. of
GlobalSIP 2013, pp. 551–554.[20] P. Bianchi, W. Hachem, and F. Iutzeler, “A coordinate descent primal-
dual algorithm and application to distributed asynchronous optimiza-tion,” IEEE Trans. Auto. Contr., vol. 61, no. 10, pp. 2947–2957, 2016.
[21] H. Wang, X. Liao, T. Huang, and C. Li, “Cooperative distributedoptimization in multiagent networks with delays,” IEEE Trans. Syst.
Man Cybern. Syst., vol. 45, no. 2, pp. 363–369, 2015.[22] J. Li, G. Chen, Z. Dong, and Z. Wu, “Distributed mirror descent method
for multi-agent optimization with delay,” Neurocomputing, vol. 177, pp.643–650, 2016.
[23] K. I. Tsianos and M. G. Rabbat, “Distributed dual averaging for convexoptimization under communication delays,” in Proc. of ACC 2012, pp.1067–1072.
[24] ——, “Distributed consensus and optimization under communicationdelays,” in Proc. of Allerton 2011, pp. 974–982.
[25] P. Lin, W. Ren, and Y. Song, “Distributed multi-agent optimization sub-ject to nonidentical constraints and communication delays,” Automatica,vol. 65, pp. 120–131, 2016.
[26] T. T. Doan, C. L. Beck, and R. Srikant, “Impact of communicationdelays on the convergence rate of distributed optimization algorithms,”arXiv:1708.03277, 2017.
[27] P. Di Lorenzo and G. Scutari, “Distributed nonconvex optimization overnetworks,” in Proc. of IEEE CAMSAP 2015, Dec. 2015.
[28] J. Xu, S. Zhu, Y. C. Soh, and L. Xie, “Augmented distributed gradientmethods for multi-agent optimization under uncoordinated constantstepsizes,” in Proc. of CDC 2015, Dec., pp. 2055–2060.
[29] P. Di Lorenzo and G. Scutari, “Next: In-network nonconvex optimiza-tion,” IEEE Trans. Signal Inf. Process. Netw., vol. 2, no. 2, pp. 120–136,2016.
[30] C. N. Hadjicostis, N. H. Vaidya, and A. D. Dominguez-Garcia, “Robustdistributed average consensus via exchange of running sums,” IEEE
Trans. Auto. Contr., vol. 31, no. 6, pp. 1492–1507, 2016.[31] A. Nedic and A. Ozdaglar, “Convergence rate for consensus with
delays,” J. Glob. Optim., vol. 47, no. 3, pp. 437–456, 2010.[32] P. Lin and W. Ren, “Constrained consensus in unbalanced networks
with communication delays,” IEEE Trans. Auto. Contr., vol. 59, no. 3,pp. 775–781, 2013.
[33] A. Nedic, A. Olshevsky, and W. Shi, “Achieving geometric convergencefor distributed optimization over time-varying graphs,” SIAM J. Optim.,vol. 27, no. 4, pp. 2597–2633, 2017.
[34] Y. Sun, G. Scutari, and D. Palomar, “Distributed nonconvex multiagentoptimization over time-varying networks,” in Proc. of Asilomar 2016.IEEE, pp. 788–794.
[35] G. Scutari and Y. Sun, “Distributed nonconvex constrained optimizationover time-varying digraphs,” Math. Prog., vol. 176, no. 1-2, pp. 497–544, July 2019.
[36] Y. Sun, A. Daneshmand, and G. Scutari, “Convergence rate of distributedoptimization algorithms based on gradient tracking,” arXiv:1905.02637,2019.
[37] D. Kempe, A. Dobra, and J. Gehrke, “Gossip-based computation ofaggregate information,” in Proc. of FOCS 2003. IEEE, pp. 482–491.
[38] N. Bof, R. Carli, and L. Schenato, “Average consensus with asyn-chronous updates and unreliable communication,” in Proc. of the IFACWord Congress 2017, pp. 601–606.
[39] T. S. Rappaport, Wireless Communications: Principles & Practice.Prentice Hall, 2002.
[40] S. M. Kay, Fundamentals of Statistical Signal Processing–Estimation
Theory. Prentice Hall, 1993.[41] L. Cannelli, F. Facchinei, and G. Scutari, “Multi-agent asynchronous
nonconvex large-scale optimization,” in Proc. of IEEE CAMSAP 2017,pp. 1–5.
[42] L. Zhao, M. Mammadov, and J. Yearwood, “From convex to nonconvex:a loss function analysis for binary classification,” in 2010 IEEE ICDM
Workshops, pp. 1281–1288.[43] D. Dua and C. Graff, “UCI machine learning repository,” 2017.
[Online]. Available: http://archive.ics.uci.edu/ml[44] M. Assran and M. Rabbat, “Asynchronous subgradient-push,”
arXiv:1803.08950, 2018.[45] J. Zhang and K. You, “Asyspa: An exact asynchronous algorithm for
convex optimization over digraphs,” arXiv:1808.04118, 2018.[46] R. A. Horn and C. R. Johnson, Matrix analysis. Cambridge university
press, 1990.