Stabilization of continuous-time Markov/semi-Markov jump

linear systems via finite data-rate feedback 1

JingyiWang a Jianwen Feng a Chen Xu a XiaoqunWu b Jinhu Lu c

aCollege of Mathematics and Statistics, Shenzhen University, Shenzhen 518060, PR China

bSchool of Mathematics and Statistics, Hubei Key Laboratory of Computational Science, Wuhan University, Wuhan, China

cState Key Laboratory of Software Development Environment, School of Automation Science and Electrical Engineering,Beijing Advanced Innovation Center for Big Data and Brain Computing, Beihang University, Beijing, China

Abstract

This paper investigates almost sure exponential stabilization of continuous-time Markov jump linear systems (MJLSs) undercommunication data-rate constraints by introducing sampling and quantization into the feedback control. Different fromprevious works, the sampling times and the jump times are independent of each other in this paper. The quantization isrecursively adjusted on the sampling time, and its updating strategy does not depend on the switching in a sampling interval.In other words, the explicit value of the switching signal in a sampling interval is not necessary. The numerically testablecondition is developed to ensure almost sure exponential stabilization of MJLSs under the proposed communication and controlprotocols. We also drop the assumption of stabilizability of all individual modes required in previous works about the switchedsystems. Moreover, we extend the result to the case of continuous-time semi-Markov jump linear systems (semi-MJLSs) via thesemi-Markov kernel approach. Finally, some numerical examples are presented to illustrate the effectiveness on stabilizationof the proposed communication and control protocols.

Key words: almost sure exponential stabilization, Markov jump linear systems, semi-Markov jump linear systems, finitedata-rate feedback, sampling and quantization

1 Introduction

In recent decades, the switched system has attractedconsiderable attention because of its strong engineeringbackground, see Liberzon & Morse (1999), Hespanha &Morse (1999) and Liberzon (2003c). In particular, theMarkov jump linear system has been considered as aspecial case of the switched system, when the switchingsignal is modeled as finite state Markov process, becauseit has been widely used to model many practical systemswith abrupt random changes, such as power systems(Li et al., 2007), freeway transportation system (Zhang& Prieur, 2017), networked control systems (Wang &Zhang, 2013), etc.

Email addresses: wangjingyi@szu.edu.cn (JingyiWang), fengjw@szu.edu.cn (Jianwen Feng),xuchen szu@szu.edu.cn (Chen Xu), xqwu@whu.edu.cn(Xiaoqun Wu), jhlu@iss.ac.cn (Jinhu Lu).1 This paper was not presented at any IFAC meeting.Corresponding author J. Feng. Tel. +86-755-26532674.

The stability and stabilization problems of MJLSshave received considerable attention in recent years (e.g., Bolzern et al. (2006); Li et al. (2012); Song et al.(2016); Zhang et al. (2016, 2009); Gabriel & Geromel(2017); Wu et al. (2018); Yang & Liberzon (2018)and the references therein). The MJLS can be usedto model practical systems with random changes. Thecontinuous-time Markov process is used to describethe switching mechanism, including the switchingpatterns, the jump (switching) times, and the sojourn(holding) times of the active mode. The stabilityanalysis of continuous-time MJLSs was consideredin the almost sure sense and a sufficient conditionwas derived depending on the transition matrix inBolzern et al. (2006), while the discrete-time case waspresented in Song et al. (2014). Moreover, the switchingMJLS was studied and its almost sure exponentialstability was obtained, in which the transition ratematrix for the random Markov process was variedwhen a deterministic switching occurred in Song et al.(2016). The stability and stabilization of discrete-timesemi-MJLSs were considered via the semi-Markov

Preprint submitted to arXiv 29 October 2021

arX

iv:2

110.

1493

1v1

[m

ath.

DS]

28

Oct

202

1

kernel approach in Zhang et al. (2016). The secondorder stabilization problems of MJLSs were studiedvia explicitly constructing the stabilizing logarithmicquantizer and controller in Zhang et al. (2009). Thestability problem of semi-Markov and Markov switchedsystems was investigated by using the probabilityanalysis method in Wu et al. (2018). However, it is oftena key restriction that the jump times of Markov processand the sampling times of the sampler are the sametime series, which leads to the fact that the samplingand switching are simultaneous, but the switching isusually stochastic. When the sampler don’t know theswitches occurring, can the system achieve stabilizationvia the controller?

Understanding control over communication networkswas listed as a major challenge for the controls field(Murray et al., 2003). In engineering systems, thetotal communication capacity in bits per second maybe large in the overall system, but each componentis effectively allocated only a small portion (Nairet al., 2007). The finite data-rate feedback here meansthat measurement information transferred though acommunication channel with finite bandwidth from thesensor to the controller (see Fig. 1). The finite data-rate feedback, which can balance the communicationcapacity and control performance, combines the reliabletransmission of information in communication theoryand feedback control of information in control theory.Sampling and quantization are fundamental tools todeal with finite data-rate feedback problems in themodern control systems. Actually, sampling is thereduction of a continuous-time signal to a discrete-timesignal at the sampling time, and quantization is a kindof mapping from continuous signals to discrete sets bythe prescribed rules. With this motivation, stabilizationof the control systems via finite data-rate feedbackcontrol was studied in the continuous-time (Brockett &Liberzon, 2000; Liberzon, 2003b; Liberzon & Hespanha,2005; Berger & Jungers, 2021), discrete-time (Elia &Mitter, 2001; Liberzon, 2003a; Zhang et al., 2019) andswitching (Wakaiki & Yamamoto, 2017) settings, whichmay be subject to external disturbances (Sharon &Liberzon, 2012) or in nonlinear systems (Zhang et al.,2019; Shi & Shen, 2017). Moreover, the techniques canbe used to deal with the stabilization of systems withadditive Gaussian white noise (Liberzon, 2003b).

To the best of our knowledge, few studies have beenconducted on the stabilization problem of Markovjump linear systems under finite data-rate feedback.In this paper, we consider the stabilization problem ofthe Markov/semi-Markov jump linear systems underquantized state feedback subject to communicationdata-rate constraints. The main contributions of thispaper are listed as follows. First, we give the methodto design the communication and control protocolsand update the parameters of the protocol undercommunication data-rate constraints. Second, we

derive testable sufficient conditions for the almostsure exponential stabilization of the MJLS underthe proposed protocol, and drop the assumption ofstabilizability of all individual modes. Third, the resultsare extended to the almost sure exponential stabilizationof the semi-MJLSs.

The structure of this article is listed as follows: Section2 introduces some preliminaries, including the modeldescription of MJLSs, information patterns of thesystem, and some related concepts of almost sureexponential stabilization. In Section 3, we obtainthe sufficient conditions of almost sure exponentialstabilization, which are dependent on the generatorof the Markov process. Next, the updating rule ofthe quantization parameters is designed. Moreover,the results are extended to the case of semi-MJLSs.Some examples are given in Section 4 to illustrate theeffectiveness of our results.

2 Preliminaries

2.1 Notations

First, we write down the notations that will be usedthroughout this paper. Let R = (−∞,+∞) be the set ofreal numbers, R+ = [0,+∞) be the set of non-negativereal numbers, N = 0, 1, 2, 3, . . . be the set of naturalnumbers, N+ = 1, 2, 3, . . . be the positive integers,N[m,n] = m,m + 1,m + 2, . . . , n where m < n form,n ∈ N, Rn be the n−dimensional Euclidean spaceand Rn×n be the set of all n × n real matrices. Let1n = [1, 1, . . . , 1]> ∈ Rn be the one vector, In be then−dimensional identity matrix, O be the zero matrix.For A,B ∈ Rm×n, let [A,B] ([A;B]= [A>, B>]>)denote the horizontal (vertical) concatenation of A andB. Let the superscript > denote the transpose of a

matrix. Let∏ki=1Bi = BkBk−1 · · ·B1 denote the left

product of matrix Bi (i = 1, 2, . . . , k). Let λ(·) andλ(·) denote the smallest and the largest eigenvalue of asymmetric matrix, respectively. Let ‖·‖ denote l∞ norm,i.e., ‖x‖ = max1≤i≤n |xi| on Rn and the correspondinginduced matrix norm ‖A‖ = max1≤i≤n

∑mj=1 |Aij | on

Rn×m. Let a triple (Ω,F ,P) be the complete probabilityspace, where Ω represents the sample space, F is theσ-algebra of subsets of the sample space, known as theevent space, and P is the probability measure on F andthe measure PE is known as the probability of theevent E ∈ F .

2.2 Model description

We will consider the stabilization problem for continuous-time Markov jump linear systems as follows:

x(t) = Aσ(t)x(t) +Bσ(t)u(t), x(0) = x0, (1)

2

where x(t) ∈ Rn is the state vector, u(t) ∈ Rm is thecontrol input, A ∈ Rn×n, B ∈ Rn×m , and the switchingsignal σ(t) : R+ → M is a finite-state, homogeneousMarkov process taking value in a finite index set M =1, 2, . . . ,M with generator Γ = [γij ] ∈ RM×M givenby

Pσ(t+ h) = j|σ(t) = i =

γijh+ o(h), i 6= j,

1 + γiih+ o(h), i = j,

where h > 0 and limh→0 o(h)/h = 0, γij ≥ 0 (i 6= j)stands for the transition rate from mode i to mode j,

and γii = −∑Mj=1,j 6=i γij , which specifies the active

mode σ(t) at each time t. The set of the matrix pair(Ap, Bp), p ∈ M denotes a collection of matricesdefining the modes.

Define the sequence of jump times of σ(t), t ∈ R+ by

τ0 = 0, τk+1 = inf t : t ≥ τk, σ(t) 6= σ(τk) , k ∈ N,

where we adopt the convention inf ∅ = ∞. Define thesequence of sojourn times (holding times) of σ(t), t ∈R+ by

hk = τk − τk−1, k ∈ N+,

where hk =∞ if τk =∞. Let σ(∞) = σ(τk−1) if τk =∞.The jump chain induced by σ(t), t ∈ R+ is defined tobe

σk = σ(τk), k ∈ N.

The sequence σk, k ∈ N is called the embeddedchain of σ(t), t ∈ R+, which is the discrete-timeMarkov process with transition probability matrixΛ = [λij ] ∈ RM×M defined by λij = −γij/γiiif i 6= j, λii = 0 otherwise. The sojourn timesh1, h2, h3, . . . are independent exponential randomvariables with parameters γσ0

, γσ1, γσ2

, . . . , respectively,where γi = −γii. In other words, Pτk+1 − τk <t|σ(τk) = i, σ(τk+1) = j = 1 − e−γit is independent ofmode j for any t ∈ R+ (see, e.g., Privault (2018)).

Assumption 1 (Markov process) The Markovprocess σ(t), t ∈ R+ is irreducible and aperiodic.

Assumption 1 implies that the Markov process isergodic and has a unique stationary distributionπ = [π1, π2, . . . , πM ] which can be calculated by πΛ = π

and∑Mi=1 πi = 1 (see, e.g., Privault (2018)).

The objective is to stabilize the Markov jump linearsystems with the controller u(t) under communicationdata-rate constraints. In the sequel, the concept ofstability is given as follows:

Definition 2 (Mao, 2008) The solution of system (1)is said to be almost surely exponentially stable if thereexists a positive constant ε such that

P

lim supt→∞

1

tln ‖x(t)‖ ≤ −ε

= 1,

for any x0 ∈ Rn.

Our another goal is to drop the assumption ofstabilizability of all individual modes, which is needed inthe context of stabilization of finite data-rate feedback,e.g., Liberzon (2014); Yang & Liberzon (2018). LetMs

be the index set of the stabilizable pair (Ap, Bp), i.e.,there exists a state feedback gain matrix Kp such thatAp +BpKp is Hurwitz, and LetMu be the index set ofthe unstabilizable pair (Ap, Bp), i.e., there are not anymatricesKp such thatAp+BpKp is Hurwitz. Obviously,M =Ms ∪Mu andMs ∩Mu = ∅.

If the pair (Ap, Bp) is stabilizable, we assume that thesuitable stabilizing gain matrixKp has been selected andfixed such that Ap + BpKp is Hurwitz, the matrix Kp

can be obtain, e.g., by solving some algebraic Riccatiequations, and if the pair (Ap, Bp) is unstabilizable, weassume matrix Kp = O with suitable dimension.

2.3 Information Patterns

In this paper, the controller is separated from theactuator and the sensor used to measure the systemstate, and the communication channel is noiseless. Thestate information is processed and transferred similarlyto Liberzon (2003a); Tatikonda & Mitter (2004) andYang & Liberzon (2018) as shown in Fig. 1 in thefollowing standard way.

Markov Jump Linear Systemx(t) = Aσ(t)x(t) +Bσ(t)u(t)

Controller Quantizer

Decoder Communication Channel Encoder

Samplingσ(t)

x(t)

tk = kτ

x(tk), σ(tk)

ck, σ(tk)

ik, σ(tk)ik, σ(tk)

ck, σ(tk)

u(t)

Fig. 1. Block diagram of the MJLS and information pattern

(1) Sampling: State measurements are taken at timetk = kτ , k = 0, 1, 2, . . ., where τ is a fixed samplingperiod.

(2) Quantizing and encoding: Each state measurementx(tk) is quantized and encoded by an integer ik from1 to Nn by some rule given in below, where N is an

3

odd positive integer. In addition, the pair (ik, σ(tk))is encoded as a sequence of bits, and sent to thedecoder by the digital communication channel.

(3) Decoding: The state ck and σ(tk) are decoded fromthe bitstream of the pair (ik, σ(tk)) by the rulesgiven in advance.

(4) Controlling: The control signal u(t) is thendetermined solely from the decoder’s state (ck, σ(tk))according to the control protocol.

Remark 3 The sequences σ(tk), k ∈ N and σ(τk), k ∈N are different. In particular, σ(tk), k ∈ N isdependent on the sampling time while σ(τk), k ∈ Nis dependent on the switching. A connection betweenσ(tk), k ∈ N and σ(τk), k ∈ N is σ(tk) = σ(τk′)where τk′ = maxτi : τi ≤ tk, i ∈ N. An illustration ofsuch a switching pattern for the case M = 1, 2, 3 isdepicted in Fig. 2.

Fig. 2. Sample path of a Markov process σ(t), t ∈ R+ withjump times and switching modes (M = 3), and samplingtimes.

Processing similarly to Liberzon (2003a); Nair et al.(2007); Tatikonda & Mitter (2004); Yang & Liberzon(2018), the data-rate (also known as bit-rate) betweenthe encoder and the decoder

R = (log2(Nn + 1) + log2 |M|)/τ

is bits per unit of time, where |M| is the cardinality ofthe index set M (i.e., the number of modes). R is usedto characterize the capacity of communications.

Assumption 4 (finite data-rate) (Liberzon, 2014)The sampling period τ satisfies Λp < N for any p ∈ M,where Λp = ‖ exp(Apτ)‖.

Remark 5 The assumption is viewed as a constraint ofdata-rate, since the inequality ‖ exp(Apτ)‖ < N requiresτ to be sufficiently small with respect to N . Combiningthe definition of R, one can know that the data-rate Rhas a lower bound.

3 Main Results

The control objective is to stabilize the system defined inSection 2.2 in the sense of Definition 2 while respecting

the communication data-rate constraints described inSection 2.3. Our results were inspired by the work ofLiberzon (2014) and Yang & Liberzon (2018), where allindividual modes are stabilizable and switches actuallyoccur less often than once per sampling period.

3.1 Communication and control protocols

In this subsection, we describe the communication andcontrol strategy similarly to Liberzon (2014) and Yang& Liberzon (2018).

The initial state x0 is unknown. At t0 = 0, the sensorand the controller are both provided with x∗0 = 0 andarbitrarily selected initial estimates E0 > 0 and δ0 > 0.Starting from t0 = 0, at each sampling time tk, the sensordetermines if the state x(tk) is inside the hypercube ofradius Ek centered at x∗k denoted by

Sk = x ∈ Rn : ‖x− x∗k‖ ≤ Ek (2)

The hypercube Sk is the approximation of the reachableset at tk, which is also used as the range of quantization.How to update x∗k and Ek such that (2) holds i.e.,

‖x(tk)− x∗k‖ ≤ Ek, (3)

will be given in Section 3.2.

At each sampling time tk, the quantizer divides thehypercube Sk into Nn equal hypercubic boxes, N perdimension, each box is encoded by a unique integer indexfrom 1 to Nn, and the index ik of the box containingx(tk) and the active mode σ(tk) are transmitted tothe decoder. The decoder follows the same predefinedindexing protocol as the encoder, so that it is ableto reconstruct the center ck of the hypercubic boxcontaining x(tk) from ik. The controller then generatesthe control input

u(t) = Kσ(tk)x(t) (4)

for t ∈ [tk, tk+1), where Kσ(tk) is the feedback gainmatrix, and x(t) is the state of the auxiliary systemdescribed by

˙x(t) = Aσ(tk)x(t) +Bσ(tk)u(t) (5)

with the boundary condition x(tk) = ck. The auxiliarysystem is to design the feedback controller with theestimated state of x(t), which is frequently unobservable.Simple calculation shows that

‖x(tk)− ck‖ ≤1

NEk and ‖x∗k − ck‖ ≤

N − 1

NEk. (6)

Let ∆k = x∗k − ck.

4

3.2 Updating the parameters of the quantization

In the following, we will give the rules to update x∗k andEk such that equation (3) holds.

Depending on whether switches occur, two case need tobe considered respectively. Consider the sample interval[tk, tk+1), without loss of generality, let σ(tk) = p andσ(tk+1) = q throughout this paper.

Case 1: Sampling interval without switches occurring.Let e(t) = x(t) − x(t). Obviously, e(t) = Ape(t) ande(t−k+1) = exp(Apτ)e(tk) from equation (1) and (5). Onecan obtain that

‖e(t−k+1)‖ = ‖ exp(Apτ)e(tk)‖ ≤ ‖ exp(Apτ)‖‖e(tk)‖

So we can update x∗k+1 and Ek+1 by using

x∗k+1 := x(t−k+1) = exp((Ap +BpKp)τ

)ck, (7a)

Ek+1 :=ΛpNEk. (7b)

Case 2: Sampling interval with switches occurring. Letτ1k , τ2

k , . . ., τsk be the switching times of the Markovprocess, where s is the number of switches in [tk, tk+1).Obviously, τ ik and s are unknown. Let τ0

k = τk and

τs+1k = τk+1. From (1) (4) and (5), one can obtain that

x(t−k+1) = [In,On]

s∏i=0

exp(Ap,σ(τ i

k)(τ

i+1k − τ ik)

)[xk; ck],

(8)

where

Ap,q =

[Aq BqKp

On Ap +BpKp

].

In order to estimate x∗k+1 and Ek+1, one can select some

times τ1k , τ

2k , . . . , τ

M−1k as expected switching times.

σ(τ ik) is the expected switching mode corresponding tothe switching time τ ik. Let τ0

k = τk and τMk = τk+1.Processing similarly to (8), one can obtain that

x(t−k+1) = [In,On]

M∏i=0

exp(Ap,σ(τ i

k)(τ

i+1k −τ ik)

)[c>k , c

>k ]>,

(9)

where set σ(τ0k ), . . . σ(τM−1

k ) = M and τ i+1k − τ ik =

τ(1/γp)/(∑mi=1 1/γi) if σ(τ ik) = p. Moreover, one can

select some instants τ1k , τ

2k , . . . , τ

wk as the worst switching

times. Let τ0k = τk and τw+1

k = τk+1. Processingsimilarly to (8), one can obtain that

x(t−k+1) = [In,On]

w∏i=1

exp(Ap,σ(τ i

k)(τ

i+1k −τ ik)

)[c>k , c

>k ]>,

(10)

where

w, τ ik, σ(τ ik), i ∈ N[1,w]

= arg max∥∥ w∏i=1

exp(Ap,σ(τi)(τ

i+1k − τ ik)

)∥∥can be seen as the outliers of the switching times.

Let Sk = [In,On]∏Mi=0 exp

(Ap,σ(τ i

k)(τ

i+1k − τ ik)

)and

Sk = [In,On]∏wi=1 exp

(Ap,σ(τ i

k)(τ

i+1k − τ ik)

). From (3)

and (6), one can get that

‖x(tk+1)− x(t−k+1)‖ ≤ ‖Sk‖(2‖x∗k‖+

2N − 1

NEk). (11)

Moreover, from (6) and (11), by the triangle inequalityone can get that

‖x(tk+1)− x(tk+1)‖≤(2‖Sk‖+ ‖Sk − Sk‖)‖x∗k‖

+(N − 1)‖Sk − Sk‖+ (2N − 1)‖Sk‖

NEk. (12)

So we can update x∗k+1 and Ek+1 when switches occurby using

x∗k+1 :=Sk[c>k , c>k ]>, (13a)

Ek+1 :=χk‖x∗k‖+ ψkEk, (13b)

where

χk =2‖Sk‖+ ‖Sk − Sk‖,ψk =((N − 1)‖Sk − Sk‖+ (2N − 1)‖Sk‖)/N.

Remark 6 How to select τ ik, i ∈ N[1,M ] is challengingbut computable. It is easy to see that

∥∥ w∏i=1

exp(Ap,σ(τi)(τ

i+1k − τ ik)

)∥∥ ≤ exp(

maxq∈M

‖Ap,q‖τ).

The above inequality can be used to choose the worstswitching times.

Remark 7 Matrix Sk is not uniquely determinedfrom (9), because Ap,q, p, q ∈ M may be not

5

commute. Nevertheless, one can guarantee that xk ∈ Skholds for any k ∈ N+ because inequality (12) holds.Matrix Sk is computable, because the modes arefinite, and τ is bounded. It it easy to verify that∥∥∏w

i=1 exp(Ap,σ(τi)(τ

i+1k −τ ik)

)∥∥ ≤ exp(

maxq∈M ‖Ap,q‖τ).

xtk+1is used to estimate the center of quantization and

is dependent on the mean value of the sojourn time ofthe mode, which can be seen as the “expected value”.x(tk+1) is used to estimate the range of quantization,which can be seen as the “worst value”.

3.3 Stability Analysis of the MJLS

In the subsection, the sufficient condition will be givento ensure the stabilization of the MJLSs under the abovecommunication and control protocol.

For convenience, let

νp = max

1− α1,p,β1,p

ρp+

Λ2p

N2

,

α1,p =λ(Qp)

λ(Pp)− αpλ(S>p QpSp)

λ(Pp),

β1,p =(

1 +1

αp

)nλ(S>p QpSp)

(N − 1

N

)2

,

υp = maxα2,p,

β2,p

ρp+

Λ2p

N2

,

α2,p = (1 + βp)λ(S>p PpSp)

λ(Pp),

β2,p =(

1 +1

βp

)nλ(S>p PpSp)

(N − 1

N

)2

,

µp = maxq∈M

µpq,

µpq = maxα3,pq,

β3,pq

ρp

,

α3,pq = 2(1 + βpq)λ(S>p PqSp)

λ(Pp)+ ρqχ

2p(1 + αpq)

1

λ(Pp),

β3,pq = 2(

1 +1

βpq

)nλ(S>p PqSp)

(N − 1

N

)2

+ ρqψ2p(1 +

1

αpq),

where ρp, αp, βp, αpq and βpq are positive constants, andPp are positive definite matrices, which will be defined.We arrive at the following result.

Theorem 8 Consider Markov jump linear system (1).Suppose that Assumptions 1 and 4 hold. If the inequality

∑p∈M

πppµ,p lnµp+∑p∈Ms

πppν,p ln νp+∑p∈Mu

πppυ,p ln υp < 0

(14)

holds, where pν,p = e−2γpτ , pυ,p = e−γpτ , andpµ,p = 1 − e−2γpτ , then system (1) reaches almost sureexponential stabilization under the communication andcontrol protocol described in Section 3.1.

PROOF. Define the Lyapunov function

Vp(x∗k, Ek) = (x∗k)>Ppx

∗k + ρpE

2k, p ∈M

which depends on the active mode p (p = σ(tk)), wherePp is a positive definite matrix, and ρp > 0. By thedefinition of the quantization in Section 3.1, it is obviousthat the sequences x∗k, k ∈ N and Ek, k ∈ N can beused to characterize the stability of x(t). The proof isdivided into 5 steps as follows:

Step 1: Sampling interval without switches occurring.Two scenarios are needed to be considered as follows:

a) (Ap, Bp) is stabilizable, i.e., p ∈ Ms. Let Sp =

e(Ap+BpKp)τ , there exists Pp > 0 and Qp > 0 such thatS>p PpSp − Pp = −Qp < 0. One can obtain that

(x∗k+1)>Ppx∗k+1 − (x∗k)>Ppx

∗k

=(Spx∗k + Sp∆k)>Pp(Spx

∗k + Sp∆k)− (x∗k)>Ppx

∗k

≤− (x∗k)>Qpx∗k + α(x∗k)>S>p PpSpx

∗k

+1

α∆>k S

>p PpSp∆k + ∆>k S

>p PpSp∆k

for any αp > 0, because Pp is positive definite, whichhas the Cholesky factorization, and 2x>Ppy ≤ x>Ppx+y>Ppy for any x, y ∈ Rn and α > 0,

Vp(x∗k+1, Ek+1)

≤(

1− λ(Qp)

λ(Pp)+

αλ(S>p QpSp)

λ(Pp)

)(x∗k)>Ppx

∗k

+(

1 +1

α

)nλ(S>p QpSp)

(N − 1

N

)2

E2k +

Λ2p

N2ρpE

2k

≤νpVp(x∗k, Ek), (15)

One can choose ρp and αp such that νp < 1 for anyp ∈Ms.

b) (Ap, Bp) is unstabilizable, i.e., p ∈Mu. Let Kp = Oand Sp = exp(Apτ), one can obtain that

(x∗k+1)>Ppx∗k+1

=(x∗k)>S>p PpSpx∗k + 2 (x∗k)

>S>p PpSp∆k + ∆>k S

>p PpSp∆k

Processing similarly to (15), one can obtain that

Vp(x∗k+1, Ek+1)

≤(

(1 + βp)λ(S>p PpSp)

λ(Pp)

)(x∗k)>Ppx

∗k

+(

1 +1

βp

)nλ(S>p PpSp)

(N − 1

N

)2

E2k +

Λ2p

N2ρpE

2k

≤υpVp(x∗k, Ek), (16)

for any βp > 0. Obviously, υp > 1 for any ρp and βp,p ∈Mu

6

Step 2: Sampling interval with switches occurring. Noticethat (13), one can obtain that

Vq(x∗k+1, Ek+1) =

(x∗k+1

)>Pqx

∗k+1 + ρqE

2k+1

≤(Sp[(x∗k)>, (x∗k)>]> + Sp[∆

>k ,∆

>k ]>)>

× Pq(Sp[(x∗k)>, (x∗k)>]> + Sp[∆>k ,∆

>k ]>)

+ ρq(χp‖x∗k‖+ ψpEk)2

Processing similarly to (15), one can obtain that

Vq(x∗k+1, Ek+1)

≤(

2(1 + βpq)λ(S>p PqSp)

λ(Pp)

)(x∗k)>Ppx

∗k

+ 2(

1 +1

βpq

)nλ(S>p PqSp)

(N − 1

N

)2

E2k

+ ρq

(χ2p(1 + αpq)

1

λ(Pp)(x∗k)>Ppx

∗k + ψ2

p(1 +1

αpq)E2

k

)≤µpqVp(x∗k, Ek), (17)

for any αpq > 0 and βpq > 0.

Step 3: Combined bound at sampling times. The samplingintervals divide the above three types. And combiningthem, from (15) (16) and (17), one can get that

Vσ(tk)(x∗k, Ek) ≤

∏p,q∈M

µNpqpq

∏p∈Ms

νNpp

∏p∈Mu

υNpp

× Vσ(t0)(x∗0, E0), (18)

where Npq denotes the number of the sampling intervalswith switches occurring and σ(tk) = p and σ(tk+1) = q,and Np denotes the number of the sampling intervalswithout switches occurring and σ(tk) = p.

Step 4: State bound in sampling intervals. Consider bothswitches occurring and no switch occurring scenarios inan interval [tk, tk+1). When no switch occurs, one canobtain that

x(t) = [In On] exp(Ap,p(t− tk)

)[x>k , c

>k ]>

for t ∈ [tk, tk+1). One can obtain that

‖x(t)‖ ≤ ‖[In On] exp(Ap,p(t− tk)

)[x>k , c

>k ]>‖

≤ exp(‖Ap,p‖τ)

)(‖xk‖+ ‖ck‖). (19)

When switches occur, letτ1k , τ2

k , . . ., τsk be the switchingtimes of the Markov process, where s is the number ofswitches in [tk, tk+1). Let τ0

k = τk and τs+1k = τk+1, one

can obtain that

x(t) = [In On] exp(Ap,σ(τj

k)(t− τ

jk))

j−1∏i=1

exp(Ap,σ(τ i

k)(τ

i+1k − τ ik)

)[x>k , c

>k ]>

for any t ∈ [τ jk , τj+1k ) ⊂ [tk, tk+1), j ∈ N[0,s]. Processing

similarly to (19), one can obtain that

‖x(t)‖ ≤ maxq∈M,p6=q

exp(‖Ap,q‖τ

)(‖xk‖+ ‖ck‖), (20)

for any t ∈ [tk, tk−1). From (19) and (20), one can obtainthat

‖x(t)‖ ≤ maxq∈M

exp(‖Ap,q‖τ

)(‖xk‖+ ‖ck‖). (21)

Step 5: Almost sure exponential stabilization. Let pswitch

be the probability that switches occur in a samplinginterval, and 1− pswitch be the probability that not any

switch occurs. Let pν,p :=∫ +∞

2τγpe−γpωdω = e−2γpτ be

the probability of the sojourn time more than 2τ of mode

p for p ∈ Ms. Let pυ,p :=∫ +∞τ

γpe−γpωdω = e−γpτ be

the probability of the sojourn time more than τ of mode

p for p ∈ Mu. Let pµ,p :=∫ 2τ

0γpe−γpωdω = 1 − e−2γpτ

be the probability of the sojourn time less than 2τ ofmode p for p ∈ M. Obviously, pswitch <

∑p∈M πppµ,p.

By using ergodic law of large numbers, from (18) and(21), one can have that

lim supt→∞

1

tln ‖x(t)‖ ≤

( ∑p∈M

πppµ,p lnµp

+∑p∈Ms

πppν,p ln νp +∑p∈Mu

πppυ,p ln υp)k(‖x∗0‖+E0)

in the almost sure sense, where k is a positive constant.So, if condition (14) holds, thenPlim supt→∞

1t ln ‖x(t)‖ ≤

−ε = 1. This completes the proof of Theorem 8.

Remark 9 The condition of the MJLS is computable,which is independent of the explicit evolution of theMarkov process. The generator Γ of the Markov process,which encodes all properties of the process in a singlematrix, is important in the conditions of stabilization.Different generators result in the different stabilizationfor the same modes.

Remark 10 The concept of dwell time and averagedwell-time have become standard assumption in the studyof stability and stabilizability of switched and hybridsystems (Yang & Liberzon, 2018; Berger & Jungers,2021). Liberzon (2014), Yang & Liberzon (2018) andWakaiki & Yamamoto (2017) assume that the samplingperiod is no larger than the dwell time, that is, switchesactually occur less often than once per sampling period.The dwell-time assumption of switching is droppedby using the sojourn time of Markov process. Theassumption of stabilizability of all individual modes isnot required in this paper.

7

3.4 Extend to the semi-MJLS case

The semi-MJLS is more general than the MJLS inmodeling some practical systems. In the subsection,we will extent our result to the semi-MJLS case. Theswitching signal σ(t), t ∈ R+ is a homogeneous semi-Markov process. The discrete-time process σk, k ∈ Nis the embedded Markov chain of σ(t), t ∈ R+ withtransition probability matrix Λ = [λij ] ∈ RM×Mdefined by λij = Pσk+1 = j|σk = i > 0 if i 6= j,λii = 0 otherwise. The function Fij(t) is a cumulativedistribution function of a sojourn time in mode i beforemoving to mode j of σ(t), t ∈ R+, defined by Fij(t) :=Phk+1 ≤ t|σk = i, σk+1 = j for any i, j ∈ M,t ∈ R+. The function fij(t) is the probability densityfunction corresponding to Fij(t). The semi-Markovkernel Θ(t) = [θij(t)] ∈ RM×M of σ(t), t ∈ R+ isdefined by θij(t) := Pσk+1 = j, hk+1 ≤ t|σk = i,for any i, j ∈ M, t ∈ R+. It it easy to check thatθij(t) = λijFij(t).

In this paper, the cumulative distribution function ofthe sojourn time depends on both the current and nextsystem mode.

Assumption 11 (semi-Markov process) The semi-Markov process σ(t), t ∈ R+ is irreducible andaperiodic.

Similarly to the Markov process, Assumption 11 impliesthe semi-Markov process is ergodic and has a uniquestationary distribution π = [π1, π2, . . . , πM ] which can

be calculated by πΛ = π and∑Mi=1 πi = 1 (see, e.g.,

Grabski (2015)).

Theorem 12 Consider semi-Markov jump linearsystem (1). Suppose that Assumptions 4 and 11 hold. Ifthe inequality∑p∈M

πpp′µ,p lnµp+

∑p∈Ms

πpp′ν,p ln νp+

∑p∈Mu

πpp′υ,p ln υp < 0

holds, where p′µ,p =∑q∈M

λpq∫ 2τ

0fpq(ω)dω, p′ν,p =∑

q∈Mλpq

∫ +∞2τ

fpq(ω)dω and p′υ,p =∑q∈M

λpq∫ +∞τ

fpq(ω)dω,

then system (1) reaches almost sure exponentialstabilization under the communication and controlprotocol described in Section 3.1.

Remark 13 The difference between the Markov processand the semi-Markov process is the probability densityfunction of the sojourn times. The sojourn times ofsemi-Markov process are random variables with anydistribution. The proof is the same as the proof ofTheorem 8 except Step 5. One can deal with Step5 by computing the probability of sojourn times via

the semi-Markov kernel approach, thus obtain thatfi(t) =

∑j∈M λijfij(t)) and Fi(t) =

∑j∈M θij(t) =∑

j∈M λijFij(t). The proof is omitted for brevity. TheMarkov process can be treated as a special case of thesemi-Markov process, where the probability densityfunction fi(t) = γie

−γit only depends on the currentsystem mode (see, e.g., Grabski (2015)). Theorem 12 ismore general than Theorem 8.

4 Numerical Simulation

4.1 Evolution algorithm

In this subsection, the algorithm of the state evolutionis given. Let ∆t be the time step, end be the ending timeof the simulation. ∆t can be designed depending on thesampling period τ . Algorithm 1 shows the logic of thedesigned protocol to compute the states of the MJLS.

Algorithm 1 Control and state evolution

Initial: System matrices Ai, Bi, Ki for i ∈ M, Γ, N ,∆t and endOutput: State evolution: x(t), x(t)Compute τ by using condition (14)for t = 0 : ∆t : end do

Update σ(t), x(t) and x(t) by using (1), (4) and (5)if t = tk tk = kτ, k ∈ N then

if no switch occurs in [tk−1, tk) thenUpdate Ek, x∗k by using definition (7)

elseUpdate Ek, x∗k by using definition (13)

end ifUpdate ck according to the quantization.

end ifend for

4.2 Numerical Examples

In this subsection, some numerical examples areprovided to demonstrate the validity of the obtainedtheoretical results.

Consider a MJLS with three modes with systemmatrices:

A1 =

[1 0

0 −1

], B1 =

[1

0

],K1 =

[−2 0

],

A2 =

[1 0

0 −1

], B2 =

[0

1

],K2 =

[0 0],

A3 =

[0 1

−1 0

], B3 =

[0

1

],K3 =

[0 −4

],

8

and generator of the Markov process

Γ =

−0.050 0.010 0.040

0.075 −0.15 0.075

0.035 0.005 −0.045

.

First, consider the MJLS under the protocol definedin Section 3.1 and 3.2. Choose P1 = In, P2 = In, forp ∈ 1, 2, 3, τ = 0.1, N = 10 and ∆t = 0.0001 suchthat the conditions of Theorem 8 are satisfied (thedetailed results are shown in Table 1). The evolution ofthe states of the MJLS is shown in Fig. 3 for the choseninitial values [−5, 8.9]>. One can observe from Fig. 3that stabilization can be reached.

p stabilizability νp υp µp πp

1 Yes 0.8766 / 5.5734 0.4386

2 No / 1.3435 4.8951 0.1404

3 Yes 0.9883 / 7.7885 0.4211

Table 1The detailed computation results.

Next, the 20 realizations of the MJLS are given and Fig.4 illustrates the |x1(t)| and |x2(t)| starting with the sameinitial value [−5, 8.9]> and generator Γ. Apparently, theMJLS under the designed communication and controlprotocol achieves almost sure exponential stabilization.

5 Conclusion

In this paper, we consider the stabilization problemof the MJLSs under the communication data-rateconstraints, where the switching signal is a continuous-time Markov process. Sampling and quantization areused as fundamental tools to deal with the problem.Under the proposed communication and controlprotocol, the sufficient conditions are given to ensurethe almost sure exponential stabilization of the MJLSs.The conditions depend on the generator of the Markovprocess. The sampling times and the jump time is alsoindependent. We extend the result to the semi-MJLSscase.

In future, we will extend our results to Markov/semi-Markov jump nonlinear systems. The communicationand control protocols will applied to networked controlsystems. Typical communication channels are noisy andhave delays. The noise and delays will be considered instabilization of MJLSs/semi-MJLSs.

Acknowledgements

This work was supported in part by the National NaturalScience Foundation of China under Grant 61873171,

61872429 and 61973241, in part by the Natural ScienceFoundation of Guangdong Province, China under Grant2019A1515012192.

References

Berger, G. O., & Jungers, R. M. (2021). Quantizedstabilization of continuous-time switched linear systems.IEEE Control Systems Letters, 5 (1), 319–324.

Bolzern, P., Colaneri, P., & De Nicolao, G. (2006). Onalmost sure stability of continuous-time Markov jumplinear systems. Automatica, 42 (6), 983–988.

Brockett, R. W., & Liberzon, D. (2000). Quantized feedbackstabilization of linear systems. IEEE Transactions onAutomatic Control , 45 (7), 1279–1289.

Elia, N., & Mitter, S. K. (2001). Stabilization of linearsystems with limited information. IEEE Transactions onAutomatic Control , 46 (9), 1384–1400.

Gabriel, G. W., & Geromel, J. C. (2017). Performanceevaluation of sampled-data control of markov jump linearsystems. Automatica, 86 , 212–215.

Grabski, F. (2015). Semi-Markov Processes: Applications inSystem Reliability and Maintenance. Elsevier.

Hespanha, J. P., & Morse, A. S. (1999). Stability of switchedsystems with average dwell-time. In Proceedings of the38th IEEE Conference on Decision and Control (pp. 2655–2660). volume 3.

Li, C., Chen, M. Z. Q., Lam, J., & Mao, X. (2012). Onexponential almost sure stability of random jump systems.IEEE Transactions on Automatic Control , 57 (12), 3064–3077.

Li, L., Ugrinovskii, V. A., & Orsi, R. (2007). Decentralizedrobust control of uncertain markov jump parametersystems via output feedback. Automatica, 43 (11), 1932–1944.

Liberzon, D. (2003a). Hybrid feedback stabilization ofsystems with quantized signals. Automatica, 39 (9), 1543–1554.

Liberzon, D. (2003b). On stabilization of linear systems withlimited information. IEEE Transactions on AutomaticControl , 48 (2), 304–307.

Liberzon, D. (2003c). Switching in Systems and Control .Boston: Birkhauser.

Liberzon, D. (2014). Finite data-rate feedback stabilizationof switched and hybrid linear systems. Automatica, 50 (2),409–420.

Liberzon, D., & Hespanha, J. P. (2005). Stabilizationof nonlinear systems with limited information feedback.IEEE Transactions on Automatic Control , 50 (6), 910–915.

Liberzon, D., & Morse, A. S. (1999). Basic problems instability and design of switched systems. IEEE ControlSystems Magazine, 19 (5), 59–70.

Mao, X. (2008). Stochastic Differential Equations andApplications. (2nd ed.). Cambridge: WoodheadPublishing.

Murray, R. M., Astrom, K. J., Boyd, S. P., Brockett, R. W.,& Stein, G. (2003). Future directions in control in aninformation-rich world. IEEE Control Systems Magazine,23 (2), 20–33.

Nair, G. N., Fagnani, F., Zampieri, S., & Evans, R. J. (2007).

9

(a) Time evolution of state x1(t) and x1(t) (b) Time evolution of state x2(t) and x2(t)

(c) The quantization x1(t) (d) The quantization x2(t)

Fig. 3. The states and quantization of the MJLS with a Markov process sample path.

Fig. 4. The states of 20 realizations with x(0) = [−5, 8.9]>.

Feedback control under data rate constraints: an overview.Proceedings of the IEEE , 95 (1), 108–137.

Privault, N. (2018). Understanding Markov chains: Examplesand Applications. (2nd ed.). Springer Singapore.

Sharon, Y., & Liberzon, D. (2012). Input to state stabilizingcontroller for systems with coarse quantization. IEEETransactions on Automatic Control , 57 (4), 830–844.

Shi, P., & Shen, Q. K. (2017). Observer-based leader-following consensus of uncertain nonlinear multi-agentsystems. International Journal of Robust and NonlinearControl , 27 (17), 3794–3811.

Song, Y., Dong, H., Yang, T., & Fei, M. (2014). Almost surestability of discrete-time Markov jump linear systems. IETControl Theory Applications, 8 (11), 901–906.

Song, Y., Yang, J., Yang, T., & Fei, M. (2016). Almost surestability of switching Markov jump linear systems. IEEETransactions on Automatic Control , 61 (9), 2638–2643.

Tatikonda, S., & Mitter, S. (2004). Control undercommunication constraints. IEEE Transactions onAutomatic Control , 49 (7), 1056–1068.

Wakaiki, M., & Yamamoto, Y. (2017). Stabilizationof switched linear systems with quantized output andswitching delays. IEEE Transactions on AutomaticControl , 62 (6), 2958–2964.

Wang, B.-C., & Zhang, J.-F. (2013). Distributed outputfeedback control of markov jump multi-agent systems.Automatica, 49 (5), 1397–1402.

Wu, X., Tang, Y., Cao, J., & Mao, X. (2018). Stabilityanalysis for continuous-time switched systems withstochastic switching signals. IEEE Transactions onAutomatic Control , 63 (9), 3083–3090.

Yang, G., & Liberzon, D. (2018). Feedback stabilization ofswitched linear systems with unknown disturbances underdata-rate constraints. IEEE Transactions on AutomaticControl , 63 (7), 2107–2122.

Zhang, C., Kan Chen, & Dullerud, G. E. (2009).Stabilization of Markovian jump linear systems withlimited information - a convex approach. In 2009American Control Conference (pp. 4013–4019).

Zhang, L., Leng, Y., & Colaneri, P. (2016). Stability

10

and stabilization of discrete-time semi-Markov jumplinear systems via semi-Markov kernel approach. IEEETransactions on Automatic Control , 61 (2), 503–508.

Zhang, L., & Prieur, C. (2017). Stochastic stability of markovjump hyperbolic systems with application to traffic flowcontrol. Automatica, 86 , 29–37.

Zhang, M., Shi, P., Ma, L., Cai, J., & Su, H. (2019). Network-based fuzzy control for nonlinear Markov jump systemssubject to quantization and dropout compensation. FuzzySets and Systems, 371 , 96–109.

Zhang, M., Shi, P., Ma, L., Cai, J., & Su, H. (2019).Quantized feedback control of fuzzy Markov jumpsystems. IEEE Transactions on Cybernetics, 49 (9), 3375–3384.

11