JOURNAL OF LA Passivity Based Control of Stochastic Port ... · bilization of the control system,...

JOURNAL OF LATEX CLASS FILES, VOL. X, NO. X, JANUARY 20XX 1

Passivity Based Control ofStochastic Port-Hamiltonian Systems

Satoshi Satoh,Member, IEEE,and Kenji Fujimoto,Member, IEEE,

Abstract—This paper introduces Stochastic Port-HamiltonianSystems (SPHS’s), whose dynamics are described by Ito stochasticdifferential equations. SPHS’s are extension of the deterministicport-Hamiltonian systems which are used to express variouspas-sive systems. First, we show a necessary and sufficient condition topreserve the stochastic port-Hamiltonian structure of thesystemunder a class of coordinate transformations. Second, we derive acondition for the system to be stochastic passive. Third, weequipStochastic Generalized Canonical Transformations (SGCT’s),which are pairs of coordinate and feedback transformationspreserving the stochastic port-Hamiltonian structure. Finally, wepropose a stochastic stabilization framework based on stochasticpassivity and SGCT’s.

Index Terms—Nonlinear stochastic control, Stochastic Hamil-tonian systems, Stochastic passivity, Stochastic stability.

I. I NTRODUCTION

There exist various disturbances such as measurementnoises, modeling errors and so on in controlling real plants.Since they sometimes cause performance degradation or desta-bilization of the control system, it is important to considerthem. Stochastic control theory is one of the efficient controlmethods which can take such disturbances into account. Theo-ries and techniques for the deterministic dynamical systems areapplied to stochastic ones described by stochastic differentialequations [1]. Conserved quantities and symmetry for thestochastic systems are formulated in [2]. Lyapunov functionapproaches to stochastic stability are introduced in [3], [4].Nonnegative supermartingales are used as stochastic Lyapunovfunctions, and asymptotic convergence of sample trajectories isproven by the martingale convergence theorem (see also [5]).The notion of stochastic passivity is introduced in [6]. As thedeterministic passivity-based control [7], asymptotic stabilityin probability can be achieved for a stochastic nonlinearsystem by the unity feedback of a passive output. For otheruseful stabilization methods, the literature [8] deals with astochastic version of a control Lyapunov function approachfor a class of input-affine nonlinear stochastic systems. Itprovides a sufficient condition for asymptotic stabilizabilityin probability. In [9], a stabilization method based on themaximal Lyapunov exponent and the stochastic averaging isproposed. It provides a necessary and sufficient condition for

S. Satoh is with the Division of Mechanical Systems and Ap-plied Mechanics, Faculty of Engineering, Hiroshima University, 1-4-1,Kagamiyama, Higashi-Hiroshima 739-8527, Japan e-mail: [email protected](see http://home.hiroshima-u.ac.jp/satoh/index.html).

K. Fujimoto is with the Department of Aerospace Engineering, GraduateSchool of Engineering, Kyoto University, Yoshida-Honmachi, Sakyo-ku, Ky-oto 606-8501, Japan e-mail: [email protected]

Manuscript received XXX; revised XXX.

asymptotic stability in probability of the original systemasthe maximal Lyapunov exponent of a reduced order averagedsystem becomes negative.

The aim of this paper is to introduce Stochastic Port-Hamiltonian Systems (SPHS’s). They are extension of de-terministic port-Hamiltonian systems [10], [11], which areinput-affine Hamiltonian systems. Although they intrinsicallyhave useful properties for control such as invariance andpassivity, SPHS’s do not always possess similar properties.So, we clarify the corresponding properties of a SPHS andpropose a stabilization method based on them. The resultsof the paper grew out of our previous reports [12], [13]. Inthis paper, we first define a SPHS and show a necessary andsufficient condition for the SPHS structure to be preservedunder a class of coordinate transformations. Second, we derivea condition for the system to be stochastic passive. Third, weintroduce Stochastic Generalized Canonical Transformation(SGCT) which is an extension of the generalized canoni-cal transformation for deterministic port-Hamiltonian systems[14]. This transformation consists of a pair of coordinateand feedback transformations preserving the SPHS structure.Finally, we propose a stabilization method based on stochasticpassivity and SGCT’s, and provide an optimality conditionfor the proposed controller. This method is summarized asfollows: we transform a SPHS to another stochastic passiveone by a SGCT, and then stabilize the transformed system bythe output feedback based on stochastic passivity. Here wecompare the proposed method to other existing ones. Sincewe utilize stochastic passivity, and we can show a conditionunder which a transformed Hamiltonian becomes a stochasticcontrol Lyapunov function, our method can be a special classof [6], [8]. However, we provide a systematic procedure basedon the SGCT for possessing passivity and for constructinga Lyapunov function. Passivity of a system and existenceof an appropriate Lyapunov function are pre-assumed in [6],[8], respectively. Besides, compared to [9], we only providea sufficient condition for asymptotic stability in probability.However, our method is applicable to a wider class of the plantsystems, since the literature [9] has particular restrictions dueto the averaging. Although we provide explicit conditions forconstructing a stabilizing controller, it is difficult to analyti-cally obtain the maximal Lyapunov exponent, and numericalcomputations are necessary for designing their controller.

The present paper is also motivated by the authors’ formerstudy, where we proposed a learning optimal control schemebased on a symmetric property of deterministic Hamiltoniansystems [15], [16], [17]. This learning method allows oneto obtain an optimal feedforward control input minimizing a


cost function by iteration of laboratory experiments withoutusing precise knowledge of the plant model. However, notonly this method but also many conventional iterative learningcontrol methods, e.g., [18] do not consider any environmentaldisturbance and measurement noise during experiments. Tosolve this problem, we consider the plant system with theabove uncertainties as a stochastic system, i.e., a SPHS andwe focus on stochastic control theory. Since the proposedSPHS’s can represent a wide class of important systems inthe presence of noise such as mechanical systems, electro-mechanical systems, nonholonomic systems and so on, theresults here can be applied not only to an extension ofthe learning control method but also to general nonlinearstochastic control problems.

The remainder of the paper is organized as follows. Insection II, a SPHS is defined and conditions for invarianceand stochastic passivity of the system are shown. In sectionIII, a SGCT is introduced. In section IV, a stabilizationmethod based on stochastic passivity and SGCT’s is proposed.Finally, in section V, numerical examples demonstrate theeffectiveness of the proposed method. In the sequel, for atwice differentiable scalar functionH(x, t) : Rn×R → R, wedefine∂H(x,t)

∂xas an-dimensional row vector such that itsi-th

column is given by∂H(x,t)∂xi , where the superscript(·)i denotes

the i-th row of (·). We also define∂∂x

(∂H(x,t)

∂x

)⊤

as an× n

matrix such that its(i, j)-th element is given by∂2H(x,t)∂xj∂xi , and

∂∂t

(∂H(x,t)

∂x

)⊤

as an-dimensional column vector such that its

i-th row is given by∂2H(x,t)∂t∂xi .

II. SOME PROPERTIES OF STOCHASTIC

PORT-HAMILTONIAN SYSTEMS

This section proposes Stochastic Port-Hamiltonian Systems(SPHS’s) which are extension of port-Hamiltonian systems[10], [11], and clarifies some of their properties.

A. Stochastic Port-Hamiltonian Systems (SPHS’s)

A port-Hamiltonian system is described by the followingform of (1) and it is utilized to express various deterministicpassive systems:

x = (J(x, t)−R(x, t))∂H(x, t)

∂x

⊤

+ g(x, t)u

y = g(x, t)⊤∂H(x, t)

∂x

⊤ . (1)

We extend this system to a stochastic dynamical systemwhich is described by the following Ito stochastic differentialequation:

dx = (J(x, t) −R(x, t))∂H(x, t)

∂x

⊤

dt

+g(x, t)u dt+ h(x, t) dw

y = g(x, t)⊤∂H(x, t)

∂x

⊤

. (2)

Herex(t) ∈ Rn, u(t), y(t) ∈ R

m describe the state, the inputand the output, respectively. In this paper, althoughx, u, and

y basically represent functions of time andx(t), u(t) andy(t)represent their values evaluated for a fixedt, we sometimesdrop “(t)” for notational simplicity.

The structure matrixJ(x, t) ∈ Rn×n and the dissipation

matrix R(x, t) ∈ Rn×n are skew-symmetric and symmetric

positive semi-definite for allx and t ≥ 0, respectively. Thematrix R(x, t) represents dissipative elements such as frictionof mechanical systems and resistance of electric circuits.Matrix valued functionsg(x, t) ∈ R

n×m andh(x, t) ∈ Rn×r

represent the control and the noise ports, respectively. Here wesuppose thath(0, t) = 0n×r for all t ≥ 0, where0i×j denotesthe i × j zero matrix. The signalw(t) ∈ R

r is a standardWiener process defined on a probability space(Ω,F ,P),whereΩ is a sample space,F is the sigma algebra of theobservable random events andP is a probability measureon Ω. We suppose that a HamiltonianH(x, t) ∈ R, whichdescribes the total energy of the system, is a sufficientlysmooth function, and that the inputu is a R

m-valued mea-surable function and satisfiesE[

∫ t

0‖u(s)‖2 ds] < ∞ with the

expectation with respect to the measureP , denoted byE[·]. BysettingfH(x, u, t) := (J−R)∂H

∂x

⊤+gu, we also suppose that

there exist positive constantskL1, kL2 and kG such that thefollowing inequalities hold for allξ(t), η(t) ∈ R

n, u(t) ∈ Rm

and t ≥ 0:

‖fH(ξ, u, t)− fH(η, u, t)‖ ≤ kL1‖ξ − η‖

‖h(ξ, t)− h(η, t)‖ ≤ kL2‖ξ − η‖

‖fH(ξ, u, t)‖+ ‖h(ξ, t)‖ ≤ kG(1 + ‖ξ‖+ ‖u‖).

We define a stochastic system described by the form of (2) asa Stochastic Port-Hamiltonian System (SPHS).

Remark 1:Suppose that the noise port does not exist, i.e.,h(x, t) ≡ 0n×r for any x and t. Then SPHS reduces to theconventional deterministic port-Hamiltonian system (1) as aspecial case.

In this paper, according to [4], [19], [6], [20], we introducethe notion of stability in probability for the system (2) asfollows.

Definition 1: The origin of the system (2) is stable inprobability if and only if for anyǫ > 0 andδ > 0, there existsd(ǫ, δ) > 0 such that if the initial conditionx(0) satisfies‖x(0)‖ < d(ǫ, δ), thenP

supt≥0 ‖x(t)‖ > ǫ

< δ holds.

Definition 2: The origin of the system (2) is asymptoticallystable in probability if and only if it is stable in probability andfor any ǫ > 0, limT→∞ P

supt≥T ‖x(t)‖ > ǫ

= 0 holds.

B. Coordinate transformations preserving SPHS structure

First, we reveal a property of SPHS’s with respect tocoordinate transformations. We consider the following time-varying coordinate transformation

t = tx = Φ(x, t) , (3)

such that for anyt, a mapΦt : Rn → R

n defined asΦt(x) :=Φ(x, t) = x is invertible. In the sequel, we utilize the notationΦ−1(x, t) asΦ−1

t(x), that is,Φ−1(x, t) = Φ−1

t (x) = x. Thefollowing lemma is obtained in [14] for the deterministic port-Hamiltonian system in (1).


Lemma 1: [14] The deterministic port-Hamiltonian systemin (1) is transformed into another one by the time-varyingcoordinate transformation (3), if and only if there exist a skew-symmetric matrixK(x, t) and a symmetric matrixS(x, t) suchthatR(x, t)+S(x, t) is positive semi-definite and they satisfy

∂Φ(x, t)

∂x(K(x, t)−S(x, t))

∂H(x, t)

∂x

⊤

−∂Φ(x, t)

∂t= 0. (4)

It follows that the system is always transformed into anotherone by any time-invariant coordinate transformationx = Φ(x).Lemma 1 characterizes the class of coordinate transformationswhich preserves a deterministic port-Hamiltonian structure.However, the following lemma implies that the same class oftransformations does not always preserve a SPHS structure:

Lemma 2:The stochastic port-Hamiltonian system (2) istransformed into another one by the time-varying coordinatetransformation (3), if and only if there exist a skew-symmetricmatrix K(x, t) and a symmetric matrixS(x, t) such thatR(x, t) + S(x, t) is positive semi-definite and they satisfy

1

2tr

∂

∂x

(∂Φi(x, t)

∂x

)⊤

h(x, t)h(x, t)⊤

=∂Φi(x, t)

∂x(K(x, t)−S(x, t))

∂H(x, t)

∂x

⊤

−∂Φi(x, t)

∂t(i = 1, 2, . . . , n). (5)

Proof: First, the necessity of Eq. (5) is shown. Byutilizing Ito’s formula [21], [22], the dynamics of the systemin the new coordinatexi, i = 1, 2, . . . , n is calculated as

dxi =∂Φi

∂tdt+

∂Φi

∂xdx+

1

2tr

∂

∂x

(∂Φi

∂x

)⊤

hh⊤

dt

=

[

∂Φi

∂t+

∂Φi

∂x(J−R)

∂H

∂x

⊤

+1

2tr

∂

∂x

(∂Φi

∂x

)⊤

hh⊤

]

dt

+∂Φi

∂xgu dt+

∂Φi

∂xh dw. (6)

Suppose that the system (2) is transformed into another oneby the transformation. Then, the following equation holds forall u andw:

The Right Hand Side (R.H.S.) of Eq. (6)

≡

[

(J(x, t)− R(x, t))∂H(Φ−1(x, t), t)

∂x

⊤]i

dt

+[g(x, t)u

]idt+

[h(x, t) dw

]i. (7)

Eq. (7) and thatt is identical tot due to Eq. (3) imply that

∂Φ

∂xg ≡ g,

∂Φ

∂xh ≡ h. (8)

In what follows, the symbol·−1 represents the inverse functionof the argument function and if no confusion arises,[·]−1

represents the inverse matrix of the argument. The first termof

the right hand side of the identity (7) is calculated as follows:[

(J(x, t)− R(x, t))∂H(Φ−1(x, t), t)

∂x

⊤]i

=

[

∂Φ

∂x

[∂Φ

∂x

]−1

(J−R)

[∂Φ

∂x

]−⊤∂Φ

∂x

⊤ ∂H(Φ−1(x, t), t)

∂x

⊤]i

=∂Φi

∂x

[∂Φ

∂x

]−1

(J−R)

[∂Φ

∂x

]−⊤∂H(x, t)

∂x

⊤

. (9)

It follows from Eqs. (6), (7) and (9) that

∂Φi

∂t+

∂Φi

∂x(J −R)

∂H

∂x

⊤

+1

2tr

∂

∂x

(∂Φi

∂x

)⊤

hh⊤

=∂Φi

∂x

[∂Φ

∂x

]−1

(J − R)

[∂Φ

∂x

]−⊤∂H

∂x

⊤

.

Consequently, we have

1

2tr

∂

∂x

(∂Φi

∂x

)⊤

hh⊤

=∂Φi

∂x

[([∂Φ

∂x

]−1

J

[∂Φ

∂x

]−⊤

−J)

−([∂Φ

∂x

]−1

R

[∂Φ

∂x

]−⊤

−R)]

∂H⊤

∂x−

∂Φi

∂t. (10)

From Eq. (10) we define the matricesK(x, t) andS(x, t) as

K(x, t) :=

[∂Φ

∂x

]−1

J(Φ(x, t), t)

[∂Φ

∂x

]−⊤

− J(x, t),

S(x, t) :=

[∂Φ

∂x

]−1

R(Φ(x, t), t)

[∂Φ

∂x

]−⊤

−R(x, t).

Then, K(x, t) is skew-symmetric, sinceJ(x, t) andJ(Φ(x, t), t) are so. ForR(x, t) is symmetric andR(Φ(x, t), t)is symmetric positive semi-definite,S(x, t) is symmetric andR(x, t) + S(x, t) is symmetric positive semi-definite.

The outputy(t) is well-defined, since

y = g(x, t)⊤∂H(Φ−1(x, t), t)

∂x

⊤

=

(∂Φ(x, t)

∂xg(x, t)

)⊤(∂H(x, t)

∂x

∂Φ−1(x, t)

∂x

)⊤

=

(∂Φ(x, t)

∂xg(x, t)

)⊤(

∂H(x, t)

∂x

[∂Φ(x, t)

∂x

]−1)⊤

= g(x, t)⊤∂H(x, t)

∂x

⊤

.

This proves the necessity of Eq. (5).Second, the sufficiency of Eq. (5) is shown. The outputy

in the new coordinatex can be calculated by using Eq. (8) as

y = g(x, t)⊤[∂Φ(x, t)

∂x

]⊤ [∂Φ(x, t)

∂x

]−⊤∂H(x, t)

∂x

⊤

= g(x, t)⊤∂H(Φ−1(x, t), t)

∂x

⊤

.

Therefore, the outputy has the same form in the coordinatex as inx. Now suppose that Eq. (5) holds. Then, by utilizing


Eqs. (5) and (6), the dynamics of the system can be calculatedin the new coordinatex as

dxi =

[

∂Φi

∂x(J −R)

∂H

∂x

⊤

+∂Φi

∂x(K − S)

∂H

∂x

⊤]

dt

+∂Φi

∂xgu dt+

∂Φi

∂xh dw

=

[(∂Φ

∂x(J +K)

∂Φ

∂x

⊤

︸︷︷︸

=: J

−∂Φ

∂x(R + S)

∂Φ

∂x

⊤

︸︷︷︸

=: R

)

×∂H(Φ−1(x, t), t)

∂x

⊤]i

dt+∂Φi

∂xg

︸︷︷︸

=: gi

u dt+∂Φi

∂xh

︸︷︷︸

=: hi

dw

=

[

(J(x, t)− R(x, t))∂H(Φ−1(x, t), t)

∂x

⊤]i

dt

+[g(x)u

]idt+

[h(x) dw

]i. (11)

Then, J(x, t) is skew-symmetric, sinceJ(Φ−1(x, t), t) andK(Φ−1(x, t), t) are so.R(x, t) is symmetric positive semi-definite because of the assumption thatR(Φ−1(x, t), t) +S(Φ−1(x, t), t) is so. This proves the sufficiency of Eq. (5).

Remark 2:Consider a deterministic port-Hamiltonian sys-tem (1) and apply Lemma 2 to the system. In this case,h(x, t) ≡ 0n×r holds for allx and t. Then, Eq. (5) yields

∂Φi(x, t)

∂x(K(x, t)− S(x, t))

∂H(x, t)

∂x

⊤

−∂Φi(x, t)

∂t= 0

(i = 1, 2, . . . , n). (12)

It implies the condition (4) in Lemma 1. Here suppose thatΦ is a time-invariant coordinate transformation. Then, thecondition (12) holds for anyH(x, t) andΦ(x) if we select thematricesK(x, t) ≡ 0n×n and S(x, t) ≡ 0n×n, respectively.This implies that any time-invariant coordinate transformationalways preserves the deterministic port-Hamiltonian structure.It shows that Lemma 2 implies the conventional result for thedeterministic port-Hamiltonian system as a special case.

C. Stochastic passivity of SPHS’s

In [14], a passivity condition was given, under which adeterministic port-Hamiltonian system (1) becomes passive. Inthis subsection, second, we investigatestochastic passivityofSPHS’s (2). The notion of stochastic passivity was introducedin [6]. However, since the literature [6] deals with the propertyfor only time-invariant stochastic systems, let us extend theconcept to time-varying stochastic ones in a manner analogousto the deterministic time-varying case [14].

Definition 3: Consider the following nonlinear time-varying stochastic system:

dx = f(x, u, t) dt+ h(x, t) dwy = s(x, u, t)

, (13)

wheref : Rn × Rm × R → R

n, h : Rn × R → Rn×r and

s : Rn × Rm × R → R

m are sufficiently smooth functions

and they satisfyf(0, 0, t) = 0n×1, h(0, t) = 0n×r ands(0, 0, t) = 0m×1, respectively. Then the system (13) is said tobe stochastic passive with respect to a storage functionV (x, t),if there exists a non-negative functionV (x, t) : Rn × R → R

such thatV (x, t) ≥ V (0, t) = 0 and, for any(x, u, t) ∈R

n × Rm × R, it satisfies

LV (x, t) ≤ s(x, u, t)⊤u. (14)

Here L(·) represents the infinitesimal generator for time-varying functions, defined by

L(·) :=∂(·)

∂t+

∂(·)

∂xf +

1

2tr

∂

∂x

(∂(·)

∂x

)⊤

hh⊤

. (15)

The following lemma characterizes stochastic passivity ofSPHS’s in (2).

Lemma 3:Consider the stochastic port-Hamiltonian systemof the form (2). Suppose that a HamiltonianH(x, t) is a non-negative function such thatH(x, t) ≥ H(0, t) = 0. Then, thesystem is stochastic passive with respect toH(x, t) if and onlyif the following inequality holds:

∂H(x, t)

∂t+

1

2tr

∂

∂x

(∂H(x, t)

∂x

)⊤

h(x, t)h(x, t)⊤

≤∂H(x, t)

∂xR(x, t)

∂H(x, t)

∂x

⊤

. (16)

Proof: First, the necessity of the condition (16) is shown.According to the definition (15),LH(x, t) is calculated as

LH(x, t) = (17)

∂H

∂t+∂H

∂x

(

(J−R)∂H

∂x

⊤

+ gu

)

+1

2tr

∂

∂x

(∂H

∂x

)⊤

hh⊤

=∂H

∂t−

∂H

∂xR∂H

∂x

⊤

+ y⊤u+1

2tr

∂

∂x

(∂H

∂x

)⊤

hh⊤

.

The last equality follows from∂H(x,t)∂x

J(x, t)∂H(x,t)∂x

⊤= 0

with the skew-symmetric matrixJ(x, t). Suppose that thesystem (2) is stochastic passive. Then, Eqs. (14) and (17) provethe necessity.

Second, the sufficiency is shown. From inequality (16)and Eq. (17), we obtainLH(x, t) ≤ y⊤u. According tothe definition of stochastic passivity (14), this proves thesufficiency.

Remark 3:Consider that we apply Lemma 3 to the deter-ministic port-Hamiltonian system (1) with the HamiltonianH(x, t) satisfyingH(x, t) ≥ H(0, t) = 0. For this system,h(x, t) ≡ 0n×r holds. Then the passivity condition (16)reduces to

∂H(x, t)

∂t≤

∂H(x, t)

∂xR(x, t)

∂H(x, t)

∂x

⊤

. (18)

The condition (18) reduces to Lemma 14 in [14] for thedeterministic port-Hamiltonian system. Moreover, suppose thatthe system is time-invariant. Then the condition (18) alwaysholds. It implies the result in [11] that any time-invariantdeterministic port-Hamiltonian system with a positive definiteHamiltonian is passive. This shows that Lemma 3 implies


the conventional results for deterministic port-Hamiltoniansystems as a special case.

Remark 4:We discuss a physical interpretation of Lemma3. First, we investigate the following two quantities withrespect to the stochastic system (13) and a scalar functionV (x(t), t): E[V (x(t + dt), t + dt)|x(t)] and V (E[x(t +dt)|x(t)], t + dt). The first quantity is the expected value ofV at the timet + dt conditioned onx(t). According to Ito’sformula, we have

E[V (x(t+ dt), t+ dt)|x(t)] = V (x(t), t) +

(

∂V

∂t+

∂V

∂xf

+1

2tr

∂

∂x

(∂V

∂x

)⊤

hh⊤

)

dt+ o(dt), (19)

where o(·) denotes the landau notation. SinceE[x(t +dt)|x(t)] = x(t)+f(x, u, t) dt, the second quantity representsa realization ofV at t + dt along the system without noise,and we have

V (E[x(t + dt)|x(t)], t+ dt)

= V (x(t) + f(x, u, t) dt+ o(dt), t + dt)

= V (x(t), t) +

(∂V

∂t+

∂V

∂xf(x, u, t)

)

dt+ o(dt). (20)

Since it follows from Eqs. (19) and (20), that

E[V (x(t+ dt), t+ dt)|x(t)] − V (E[x(t+ dt)|x(t)], t + dt)

=1

2tr

∂

∂x

(∂V

∂x

)⊤

hh⊤

dt+ o(dt),

the term12 tr

∂∂x

(∂V∂x

)⊤hh⊤

represents the expected varia-tion of V caused by noise.

Consequently, we can interpret the inequality (16) in Lemma3 as the expected energy increment due to the time-dependencyof V and the noise effect should be dissipated by the frictionin order to achieve stochastic passivity.

III. R ECOVERY OF PASSIVITY VIA STOCHASTIC

GENERALIZED CANONICAL TRANSFORMATIONS

(SGCT’S)

Stochastic passivity is one of the useful concepts for sta-bilizing nonlinear stochastic systems. Since SPHS’s are notalways stochastic passive, as shown in Lemma 3, we proposea technique to render the system stochastic passive based onaspecial class of coordinate and feedback transformations calledthe generalized canonical transformations [14]. This techniqueplays an important role in our passivity based stochasticstabilization method proposed in the next section.

The generalized canonical transformations transform boththe input and the output of the deterministic port-Hamiltoniansystem (1) into those of another port-Hamiltonian systemwith another Hamiltonian function. They are useful to designfeedback controllers for deterministic Hamiltonian systems.This section, first, extends such transformations to stochasticversions for the SPHS (2). We clarify conditions for thetransformations under which the transformed system preservesthe SPHS structure. Second, we investigate an extra condition

under which the transformed SPHS obtains stochastic passiv-ity.

Let us defineStochastic Generalized Canonical Trans-formations (SGCT’s), which are natural extension of thedeterministic generalized canonical transformations [14].

Definition 4: A set of transformations

t = t

x = Φ(x, t)

H(x, t) = H(x, t) + U(x, t) |x=Φ−1(x,t), t=t

y = y + α(x, t) |x=Φ−1(x,t), t=t

u = u+ β(x, t) |x=Φ−1(x,t), t=t (21)

is said to be a stochastic generalized canonical transformation(SGCT) for the stochastic port-Hamiltonian system (2), if ittransforms the system into another one which is also of theform (2) with the HamiltonianH(x, t). Here the coordinatetransformationt = t and x = Φ(x, t) is the same as Eq. (3),andU : Rn ×R → R, α : Rn ×R → R

m andβ : Rn ×R →R

m are functions, respectively.Now we show conditions which these transformations

should satisfy.Theorem 1:Consider the stochastic port-Hamiltonian sys-

tem in (2). A set of transformations defined by (21) with thefunctionsΦ(x, t), U(x, t), α(x, t) andβ(x, t) yields a SGCTif and only if there exist a skew-symmetric matrixP (x, t), asymmetric matrixQ(x, t) such thatR(x, t)+Q(x, t) is positivesemi-definite, and the functionsΦ(x, t), U(x, t) and β(x, t)satisfy

∂Φi

∂t+1

2tr

∂

∂x

(∂Φi

∂x

)⊤

hh⊤

=∂Φi

∂x

[

(J−R)∂U

∂x

⊤

+ gβ

+ (P−Q)∂(H + U)

∂x

⊤]

, (i = 1, 2, . . . , n). (22)

Further, a functionα(x, t) is given by

α(x, t) = g(x, t)⊤∂U(x, t)

∂x

⊤

. (23)

Proof: First, the necessity of the theorem is shown. Thedynamics of the transformed system in the new coordinate(x, t) is calculated as (see, also Eq. (6))

dxi=

[

∂Φi

∂t+∂Φi

∂x(J−R)

∂H

∂x

⊤

+1

2tr

∂

∂x

(∂Φi

∂x

)⊤

hh⊤

]

dt

+∂Φi

∂xgu dt+

∂Φi

∂xh dw. (24)

Suppose that the SPHS in (2) is transformed into another oneusing a SGCT withΦ, U andβ. Then, the following equationholds for allu andw

R.H.S. of Eq. (24)

≡

[

(J − R)∂H(x, t)

∂x

⊤]i

dt+[gu]idt+

[h dw

]i

=∂Φi

∂x

[∂Φ

∂x

]−1

(J − R)

[∂Φ

∂x

]−⊤∂(H(x, t) + U(x, t))

∂x

⊤

dt

+[g(u+ β)

]idt+

[h dw

]i, (25)


where J(x, t) ∈ Rn×n and R(x, t) ∈ R

n×n are skew-symmetric and symmetric positive semi-definite matrices, re-spectively. Eq. (25) and thatt is identical tot, imply that

∂Φ

∂xg ≡ g,

∂Φ

∂xh ≡ h. (26)

From Eqs. (24), (25) and (26), we have

∂Φi

∂t+1

2tr

∂

∂x

(∂Φi

∂x

)⊤

hh⊤

=∂Φi

∂x

[[∂Φ

∂x

]−1

(J−R)

×

[∂Φ

∂x

]−⊤∂(H + U)

∂x

⊤

− (J−R)∂H

∂x

⊤

+ gβ

]

. (27)

Here we define the matricesP (x, t) andQ(x, t) as

P (x, t) :=

[∂Φ

∂x

]−1

J(Φ(x, t), t)

[∂Φ

∂x

]−⊤

− J(x, t),

Q(x, t) :=

[∂Φ

∂x

]−1

R(Φ(x, t), t)

[∂Φ

∂x

]−⊤

−R(x, t). (28)

Then, P (x, t) is skew-symmetric, sinceJ(x, t) andJ(Φ(x, t), t) are so. ForR(x, t) is symmetric andR(Φ(x, t), t)is symmetric positive semi-definite,Q(x, t) is symmetricand R(x, t) + Q(x, t) is symmetric positive semi-definite.By substituting Eq. (28) for Eq. (27), Eq. (22) is obtainedimmediately.

The change of the outputα(x, t) which yields a SGCT (21)can be calculated as

α = y − y = g⊤∂H(x, t)

∂x

⊤

− g⊤∂H(x, t)

∂x

⊤

= g⊤∂Φ(x, t)

∂x

⊤ [∂Φ(x, t)

∂x

]−⊤∂(H(x, t) + U(x, t))

∂x

⊤

− g⊤∂H(x, t)

∂x

⊤

= g⊤∂U(x, t)

∂x

⊤

.

This proves the necessity of the theorem.

Second, the sufficiency of the theorem is shown. Nowsuppose the assumption of the theorem holds. Then, by sub-stituting Eq. (22) for Eq. (24), the dynamics of the system canbe calculated in the new coordinate as

dxi =

[

∂Φi

∂x(J −R)

∂H

∂x

⊤

+∂Φi

∂x

[

(J −R)∂U

∂x

⊤

+ gβ

+ (P −Q)∂(H + U)

∂x

⊤]]

dt+∂Φi

∂xgu dt+

∂Φi

∂xh dw

=

[

∂Φ

∂x(J+P−R−Q)

∂Φ

∂x

⊤ ∂(H + U)

∂x

⊤]i

dt

+∂Φi

∂xg(u+ β) dt+

∂Φi

∂xh dw. (29)

Here, J(x, t), R(x, t), g(x, t) and h(x, t) are given by

J(x, t) =∂Φ(x, t)

∂x(J(x, t)+P (x, t))

∂Φ(x, t)

∂x

⊤∣∣∣∣∣ x = Φ−1(x, t)

t = t

R(x, t) =∂Φ(x, t)

∂x(R(x, t)+Q(x, t))

∂Φ(x, t)

∂x

⊤∣∣∣∣∣ x = Φ−1(x, t)

t = t

g(x, t) =∂Φ(x, t)

∂xg(x, t)

∣∣∣∣∣ x = Φ−1(x, t)

t = t

h(x, t) =∂Φ(x, t)

∂xh(x, t)

∣∣∣∣∣ x = Φ−1(x, t)

t = t

. (30)

Then, J(x, t) is skew-symmetric, sinceJ(Φ−1(x, t), t) andP (Φ−1(x, t), t) are so.R(x, t) is symmetric positive semi-definite because of the assumption thatR(Φ−1(x, t), t) +Q(Φ−1(x, t), t) is so. Consequently, the dynamics of thesystem in the new coordinatex is given by

dxi =

[

(J(x, t)− R(x, t)

) ∂H(x, t)

∂x

⊤]i

dt+[g(u + β)

]idt

+[h dw

]i, (i = 1, 2,. . ., n) (31)

The output in the new coordinatey is obtained by Eq. (23) as

y = y + α(x, t)

= g⊤∂H(x, t)

∂x

⊤

+ g⊤∂U(x, t)

∂x

⊤

= g⊤∂Φ

∂x

⊤ [∂Φ

∂x

]−⊤∂(H + U)

∂x

⊤

= g⊤∂H(x, t)

∂x

⊤

. (32)

Eqs. (31) and (32) imply the sufficiency of the theorem.Finally, the following lemma states a condition for a trans-

formed SPHS by a SGCT to be stochastic passive.Lemma 4:Consider the stochastic port-Hamiltonian system

in (2) and transform it by an appropriate stochastic generalizedcanonical transformation such thatH(x, t) ≥ H(0, t) = 0.Then, the transformed system becomes stochastic passive withthe new HamiltonianH(x, t) as a storage function if and onlyif the following inequality holds:

−∂(H+U)

∂x

[∂Φ

∂x

]−1∂Φ

∂t+

∂(H+U)

∂t

+1

2tr

∂

∂x

(

∂(H+U)

∂x

[∂Φ

∂x

]−1)⊤

h(x)h(x)⊤∂Φ

∂x

⊤

≤∂(H+U)

∂x(R+Q)

∂(H+U)

∂x

⊤

. (33)

Proof: Due to Lemma 3, the necessary and sufficient con-dition for stochastic passivity is that the following inequalityholds in the new coordinate transformed by the SGCT:

∂H(x, t)

∂t+

1

2tr

∂

∂x

(∂H(x, t)

∂x

)⊤

hh⊤

≤∂H

∂xR∂H

∂x

⊤

.

(34)


The first term on the left hand side of the inequality (34) iscalculated as

∂H(x, t)

∂t=

∂(H + U)

∂x

∂Φ−1(x, t)

∂t+

∂(H + U)

∂t. (35)

Since the Jacobian of the pair of the coordinate transformationsx = Φ(x, t) and t = t, denoted byJ , is given by

J =

(∂Φ(x, t)

∂x

∂Φ(x, t)

∂t01×n 1

)

,

the Jacobian of the inverse coordinate transformation whichcoincides withJ −1 is obtained as

J−1 =

[∂Φ(x, t)

∂x

]−1

−

[∂Φ(x, t)

∂x

]−1∂Φ(x, t)

∂t01×n 1

≡

∂Φ−1(x, t)

∂x

∂Φ−1(x, t)

∂t

01×n 1

. (36)

It follows from the identity (36) that

∂Φ−1(x, t)

∂t= −

[∂Φ(x, t)

∂x

]−1∂Φ(x, t)

∂t. (37)

Here, the following equation holds:

∂

∂x

(∂H

∂x

)⊤

=∂

∂x

(∂(H + U)

∂x

∂Φ−1

∂x

)⊤∂Φ−1

∂x

=∂

∂x

(

∂(H + U)

∂x

[∂Φ

∂x

]−1)⊤[

∂Φ

∂x

]−1

.

(38)

Consequently, inequality (34) with Eqs. (30), (35), (37) and(38) imply that the condition (33) holds immediately. Thisproves the lemma.

Example 1:Consider the following simple mechanical sys-tem with random noise:

(dqdp

)

=

(0 1−1 0

)(∂H∂q∂H∂p

)

dt+

(01

)

u dt+

(0ap

)

dw

y = ∂H∂p

= pM

(39)

with the Hamiltonian H(x) = p2

2M , where x :=(q, p)⊤ ∈ R

2 denotes the state, a positive constantMdenotes the inertia andw(t) ∈ R is a standard Wienerprocess. Here we suppose that the noise portap with

a > 0. Since 12 tr

∂∂x

(∂H(x)∂x

)⊤

h(x)h(x)⊤

= a2p2

2M and

∂H(x)∂x

R(x)∂H(x)∂x

⊤= 0, it follows from Lemma 3 that this

system is not stochastic passive. We assign any positivedefinite scalar functionU(q) so that H becomes positivedefinite. By using Theorem 1, we obtain a SGCT asq = q,p = p, α = 0, β = dU(q)

dq − Q22

Mp, whereP := 02×2 and

Q := diag0, Q22. Then by using Lemma 4, the conditionunder which the transformed system becomes stochastic pas-sive isMa2 ≤ 2Q22.

Proposition 1: Suppose that the conditions in Lemma 4are satisfied, and thatH(x, t) is positive definite for allt.

Moreover, suppose that the inequality (34) holds strictly forall x\0 andt, if ∂H(x,t)

∂xg(x, t) = 0 holds for allx\0 and

t. Then, the transformed SPHS satisfies a stochastic Lyapunovcondition proposed in [8] andH(x, t) is a control Lyapunovfunction.

Proof: According to the definition of the control Lya-punov function in [8], it is sufficient to show that if∂H(x,t)

∂xg(x, t) = 0 holds for allx\0 andt, thenLH(x, t) <

0 holds.Under the condition, we have

LH(x, t) =∂H

∂t−

∂H

∂xR∂H

∂x

⊤

+1

2tr

∂

∂x

(∂H

∂x

)⊤

hh⊤

.

It follows from the assumptions in Proposition 1 thatLH(x, t) < 0.

IV. PASSIVITY BASED STOCHASTIC STABILIZATION

FRAMEWORK

This section proposes a stochastic stabilization frameworkfor SPHS’s based on stochastic passivity and the stochasticgeneralized canonical transformation (SGCT). For determinis-tic systems, LaSalle’s invariance principle plays an importantrole in passivity based stabilization [7], [23]. This principlewas originally applied to time-invariant systems. It is restrictedin applications for time-varying ones with some difficulties,for example, theω-limit set of a trajectory is itself notinvariant. In the literatures [24], [6], stochastic versions ofLaSalle’s invariance principle were proposed for time-invariantstochastic systems. Then, in [19], a generalization for time-varying case was proposed. Since a stability result for time-varying stochastic systems is more conservative than thatfor time-invariant ones, we consider both cases separately.Finally, an optimality condition for the proposed controlleris presented.

A. Stability theory for time-invariant SPHS’s

By utilizing Lemmas 3 and 4, the following theorem statesthe proposed stabilization method.

Theorem 2:Consider a time-invariant stochastic port-Hamiltonian system of the form (2). Suppose that the system istransformed by an appropriate stochastic generalized canonicaltransformation such thatH(x) := H(Φ−1(x)) + U(Φ−1(x))is positive definite and thatH(x) is a d times differentiablefunction with d ≥ 1. Then the statex of the transformedsystem tends in probability to the largest invariant set whosesupport is contained in the locusLH(x) = 0 for any t ≥ 0,with the unity feedbacku = −y. Furthermore, if it holds thatΓ ∩ Π = 0, then the unity feedback renders the systemasymptotically stable in probability. Here, the distribution Λ,the setsΓ and Π are defined as

Λ = spanadkf0 gi(x)|0 ≤ k ≤ n− 1, 1 ≤ i ≤ m

Γ = x ∈ Rn|Lk

0H(x) = 0, k = 1, 2, . . . , d

Π = x ∈ Rn|Lk

0LλH(x)=0, ∀λ ∈ Λ, k = 0, 1, . . . , d− 1,

where f0 := (J(x) − R(x))∂H(x)∂x

⊤, and gi represents the

i-th column of g. The vector fieldadkf0 gi is defined as


adkf0 gi(x) = [f0, adk−1f0

gi](x), for any integerk ≥ 1, setting

ad0f0 gi = gi with the Lie bracket[·, ·], see e.g. [25], [23].L0

denotes the operator (15) in whichf is replaced byf0 andL(·) represents the Lie derivative along(·). Lk

0LλH can becalculated asLk

0LλH(x) = Lk0∂H∂x

λ = L0Lk−10

∂H∂x

λ for anyintegerk ≥ 1, settingL0

0LλH = LλH.Proof: Since the transformed system is stochastic passive

with respect toH(x), LH(x) ≤ −‖y‖2 ≤ 0 holds for theclosed loop system with the unity feedbacku = −y. Then thestochastic version of LaSalle’s theorem in [24], [6] impliesthat the statex tends in probability to the largest invariantset whose support is contained in the locusLH(x) = 0 forany t ≥ 0. Consequently, the rest of the theorem is shown bydirectly applying Corollary 4.7 in [6].

B. Stability theory for time-varying SPHS’s

We show the following stability theory for time-varyingSPHS’s. For the time-varying case, LaSalle’s invariance prin-ciple in [19] generally guarantees the output convergence only.

Theorem 3:Consider a time-varying stochastic port-Hamiltonian system of the form (2). Suppose that the sys-tem is transformed by a stochastic generalized canonicaltransformation such thatH(x, t) is positive definite for

all t and ∂H∂x

, ∂∂x

(∂H∂x

)⊤

and ∂H∂t

are continuous (matrixvalued) functions, respectively. Furthermore, suppose thatlim‖x‖→∞ inf0≤t<∞ H(x, t) = ∞, and one of the followingconditions holds:

(i) For each initial statex0 ∈ Rn, there is ad > 2 such that

sup0≤t<∞ E[‖x(t)‖d] < ∞.(ii) h(x, t) is bounded.(iii) Almost every sample path of

∫ t

0 h(x(τ), τ) dw is uni-formly continuous ont ≥ 0.

Then, for every initial statex0 ∈ Rn, the Hamiltonian function

of the feedback system with the unity feedbacku = −yconverges to a finite limit, that is,limt→∞ H(x, t) < ∞ existsalmost surely. Furthermore,

limt→∞

y(t) = 0m×1

holds almost surely.Proof: Since the transformed system is stochastic passive

with respect toH(x, t), LH(x, t) ≤ −‖y‖2 ≤ 0 holds for theclosed loop system with the unity feedbacku = −y. Then theclaim of the theorem is proven by directly applying Theorem2.1 and Theorem 2.5 in [19].

C. Optimality condition for the proposed controller

The following proposition shows an optimality condition forthe proposed controller.

Proposition 2: Suppose that the conditions in Lemma 4 aresatisfied. Moreover, suppose thatH(x, t) is positive definitefor all t, and that the equality holds in the inequality (33)in Lemma 4. Then, the proposed controlleru = −y − β =

−g⊤ ∂(H+U)∂x

⊤−β is the optimal control for the following cost

functionΓ(x, t) with anyT > 0:

Γ(x, t) =E

[∫ T

0

1

2

∂(H+U)

∂xgg⊤

∂(H+U)

∂x

⊤

+1

2(u + β)⊤(u + β) dt

]

.

Furthermore, the value function is given by the new Hamilto-nianH(x, t) + U(x, t).

Proof: The Stochastic Hamilton-Jacobi-Bellman (SHJB)equation with the value functionH + U associated with thisoptimal control problem is expressed as

−∂(H+U)

∂t= min

u

(

1

2

∂(H+U)

∂xgg⊤

∂(H+U)

∂x

⊤

+1

2(u+ β)⊤(u+ β) +

∂(H+U)

∂x

(

(J−R)∂H

∂x

⊤

+ gu

)

+1

2tr

∂

∂x

(∂(H+U)

∂x

)⊤

hh⊤

)

. (40)

By taking the gradient with respect tou of the inside of theparenthesis on the right hand side, and setting it to zero, the

optimal control is given byu = −g⊤ ∂(H+U)∂x

⊤− β, which

coincides with the proposed controller.By substituting the optimal control into the SHJB equation

(40), the right hand side of the equation is reduced to

∂(H+U)

∂x(J−R)

∂H

∂x

⊤

−∂(H+U)

∂x

⊤

gβ

+1

2tr

∂

∂x

(∂(H+U)

∂x

)⊤

hh⊤

.

Since the condition for the SGCT (22) in Theorem 1 issatisfied, by eliminatinggβ we have

R.H.S. of Eq. (40)= −∂(H+U)

∂x(R+Q)

∂(H+U)

∂x

⊤

−∂(H+U)

∂x

[∂Φ

∂x

]−1

∂Φ

∂t+1

2

tr

∂∂x

(

∂Φ1

∂x

)

⊤

hh⊤

.

.

.

tr

∂∂x

(

∂Φn

∂x

)

⊤

hh⊤

+1

2tr

∂

∂x

(∂(H+U)

∂x

)⊤

hh⊤

. (41)

Here we consider the dynamicsdx = (J−R)∂(H+U)∂x

⊤dt+

h dw, and the coordinate transformation (3). Since the in-finitesimal generator is invariant under the coordinate transfor-mation,L(H+U) = LH holds, whereL(H+U) is calculatedalong the above dynamics andLH is calculated along thetransformed dynamics by the transformation (3), respectively.L(H + U) andLH are calculated as follows:

L(H + U) =∂(H+U)

∂t+

∂(H+U)

∂x(J−R)

∂(H+U)

∂x

⊤

+1

2tr

∂

∂x

(∂(H+U)

∂x

)⊤

hh⊤


LH =∂H

∂t+

∂H

∂x

(

∂Φ

∂t+∂Φ

∂x(J−R)

∂H

∂x

⊤

+1

2

tr

∂∂x

(

∂Φ1

∂x

)

⊤

hh⊤

.

.

.

tr

∂∂x

(

∂Φn

∂x

)

⊤

hh⊤

+1

2tr

∂

∂x

(∂H

∂x

)⊤

hh⊤

.

It follows from L(H +U) = LH and Eqs. (35) and (37) that

1

2tr

∂

∂x

(∂(H+U)

∂x

)⊤

hh⊤

=1

2

∂(H+U)

∂x

[∂Φ

∂x

]−1

×

tr

∂∂x

(

∂Φ1

∂x

)

⊤

hh⊤

.

.

.

tr

∂∂x

(

∂Φn

∂x

)

⊤

hh⊤

+

1

2tr

∂

∂x

(∂H

∂x

)⊤

hh⊤

. (42)

From Eqs. (41) and (42), we have

R.H.S. of Eq. (40)= −∂(H+U)

∂x(R+Q)

∂(H+U)

∂x

⊤

−∂(H+U)

∂x

[∂Φ

∂x

]−1∂Φ

∂t+

1

2tr

∂

∂x

(∂H

∂x

)⊤

hh⊤

= −∂(H+U)

∂x

[∂Φ

∂x

]−1∂Φ

∂t−

∂H

∂t= −

∂(H + U)

∂t, (43)

where the second equality comes from the assumption that theequality holds in the inequality (33) in Lemma 4, and the lastequality comes from Eqs. (35) and (37). Eq. (43) implies thatthe proposed controller satisfies the SHJB equation with thevalue functionH + U .

V. NUMERICAL EXAMPLES

This section exhibits applications of the proposed stabiliza-tion method to a nonholonomic system in the presence ofnoise.

A. Time-invariant controller design

We apply Lemma 4 and Theorems 1 and 2 to control of arolling coin to an invariant set in probability in the presenceof noise. We consider the rolling coin on a horizontal plane[11], [26] depicted in Fig. 1. LetX-Y denote the orthogonalcoordinates of the point of contact of the coin. Letq1 be theheading angle of the coin, and(q2, q3) denote the position ofthe coin in theX-Y plane. Furthermore letp1 be the angular

X0

Y

u1u2 q1

q2

q3

Fig. 1. Rolling coin

velocity with respect to the heading angle,p2 be the rollingangular velocity of the coin,u1 and u2 be the accelerationswith respect top1 and p2, respectively. Finally, let all the

parameters be unity for simplicity, e.g., the radius of the coinand the moments of inertia. Then, this system is described bya SPHS of the form (2) withq = (q1, q2, q3)⊤, p = (p1, p2)⊤,x = (q⊤, p⊤)⊤, H(x) = (1/2)p⊤p, R(x) = 05×5, g(x) =(02×3, I2)

⊤ and

J(x) =

0 0 0 1 00 0 0 0 cos q1

0 0 0 0 sin q1

−1 0 0 0 00 − cos q1 − sin q1 0 0

h(x) =

(03×2

diagh1(x), h2(x)

)

y = g(x)⊤∂H(x)

∂x

⊤

= p, (44)

whereIn represents then× n identity matrix, andh1(x) andh2(x) represent functions for the noise port. For the details ofnonholonomic Hamiltonian systems, see, e.g., [27], [28].

In [26], a stabilization technique of deterministic nonholo-nomic Hamiltonian systems is proposed, where a system isconverted into a canonical form of nonholonomic Hamiltoniansystems by a generalized canonical transformation with anypotential functionU such that it has the formU((q1)2 +(q2)2, q3) and is smooth and positive definite on(q1, q2) 6= 0and satisfies

(q1, q2) 6= 0 ⇒∂U

∂q6= 0, (45)

and with a special time-invariant coordinate transformationx = Φ(x). Then convergence of the state to the followingspecified invariant set is achieved:

Π0 :=x∣∣ q1 = q2 = 0 , p1 = p2 = 0

. (46)

The purpose of this subsection is to let the state converge tothe invariant setΠ0 in probability in the presence of noise byusing the same coordinate transformation proposed in [26].

Since the Hamiltonian of the system (44) is positive semi-definite, let us transform the system into anther stochas-tic passive system by the SGCT with a potential functionU((q1)2 + (q2)2, q3) and the following coordinate transfor-mation x = Φ(x) proposed in [26]:

q =(Φ1(x),Φ2(x),Φ3(x)

)⊤=(tan q1, q2, 2q3 − q2 tan q1

)⊤

p =(Φ4(x),Φ5(x)

)⊤=

(p1

1 + tan2 q1, p2√

1 + tan2 q1)⊤

.

(47)

In order to obtain the SGCT, we decide the other designparametersβ(x), P (x) andQ(x) according to Theorem1. The


following choice satisfies Eq. (22):P (x)=05×5 and

Q(x)=

03×3 03×2

02×3Q44(x) Q45(x)Q45(x) Q55(x)

=:

(03×3 03×2

02×3 Q(x)

)

β(x) =

(∂U∂q1

+ p1Q44(x) + p2Q45(x)

∂U∂q2

cos q1 + ∂U∂q3

sin q1 + p1Q45(x) + p2Q55(x)

)

α(x)=g⊤∂U

∂x

⊤

=

(0 0 0 1 00 0 0 0 1

)(∂U

∂q1,∂U

∂q2,∂U

∂q3, 0, 0

)⊤

= 02×1, (48)

where free parametersQ44(x), Q45(x) andQ55(x) should bechosen so thatQ(x) in Eq. (48) becomes symmetric positivesemi-definite. Furthermore, we derive another condition forQ44(x), Q45(x) and Q55(x) from Lemma 4 for stochasticpassivity. It follows from the inequality (33) that

1

2(h1(x)2 + h2(x)2)

≤ (p1)2Q44(x) + 2p1p2Q45(x) + (p2)2Q55(x). (49)

If there exist functionsQ44(x), Q45(x) andQ55(x) for a noiseport h(x) such that they satisfy the condition (49), and letQ(x) in Eq. (48) become symmetric positive semi-definite, thetransformed system obtains stochastic passivity. From anotherviewpoint, the inequality (49) implies a condition for thenoise porth(x) under which the system can be stabilizedbased on stochastic passivity. Let us note that the condition(48) is independent of the choice of a potential functionU((q1)2 + (q2)2, q3). In what follows, we suppose that thereexist functionsQ44(x), Q45(x) and Q55(x) such that theequality (49) holds andQ(x) becomes symmetric positivesemi-definite.

The transformed system is given by

J(x)=

0 0 0 1 00 0 0 0 10 0 0 −q2 q1

−1 0 q2 0 −p2q1/(1 + (q1)2)0 −1 −q1 p2q1/(1 + (q1)2) 0

R(x)=

03×3 03×2

02×3

cos4 q1Q44(x) cos q1Q45(x)

cos q1Q45(x)Q55(x)

cos2 q1

∣∣∣∣∣∣∣x=Φ−1(x)

g(x) =

(03×2

diag1/(1 + (q1)2),√

1 + (q1)2

)

(50)

h(x) =

(03×2

diagh1(x) cos2 q1, h2(x)/ cos q1

)∣∣∣∣∣x=Φ−1(x)

.

Eq. (50) implies that the transformed system has the form of(2). Since this system obtains stochastic passivity, it canbeeasily proven by Theorem 2 that with the unity feedbacku =−y, the statex tends in probability to the largest invariant setwhose support is contained in the locusLH(x) = 0. Now letus show that this largest invariant set coincides with our targetsetΠ0 in (46). SinceQ44(x), Q45(x) andQ55(x) are chosento satisfy the equality in the condition (49), the equation

12 tr

∂∂x

(∂H(x)∂x

)⊤

h(x)h(x)⊤

= ∂H(x)∂x

R(x)∂H(x)∂x

⊤holds.

Then, the passivity of the transformed system yields

LH(x) = y⊤u. (51)

Eq. (51) implies that we should show that the input/outputnulling set coincides withΠ0. The following calculationrequires thatx | p = 0 for the input/output nulling set:

0 ≡ y = g⊤ ∂H∂x

⊤=(

(1 + (q1)2)p1, p2/√

1 + (q1)2)⊤

.

Under the conditionx | p = 0, we can calculate

L0Lg1H(x)∣∣∣p=0

= −(1 + (q1)2)

(∂U

∂q1− q2

∂U

∂q3

)


= −

(∂U

∂q2+ q1

∂U

∂q3

)/√

1 + (q1)2.

From the conditions thatL0Lg1H(x)∣∣∣p=0

≡ 0 and


≡ 0, the input/output nulling set is obtained

as

x∣∣∣

∂U∂q

J12(q) = 0, p = 0

, where

J12(q) :=

(

1 0 −q2

0 1 q1

)⊤

.

From the condition of the potential functionU (see Eq. (45))and the coordinate transformationΦ in (47), it can be easilycalculated as

x∣∣∣∂U

∂qJ12(q) = 0, p = 0

=x∣∣ q1 = q2 = 0 , p1 = p2 = 0

=x∣∣ q1 = q2 = 0 , p1 = p2 = 0

≡ Π0. (52)

From Theorem 2 and Eq. (52), we obtain the controller whichachieves convergence toΠ0 as

u = −β(x) − (y + α(x))

=−

(∂U∂q1

+p1(1+Q44(x))+p2Q45(x)

∂U∂q2

cos q1+ ∂U∂q3

sin q1+p1Q45(x)+p2(1+Q55(x))

)

.

(53)

The purpose of this subsection has been achieved.Finally, let us show some simulation results. Here we

consider the noise port as

h(x) =

(0 0 0 k1p1 00 0 0 0 k2p2

)⊤

and we set a potential function asU = (1/2)q⊤q, designparameters asQ44 = (k1)2/2, Q45 = 0 andQ55 = k22/2 withk1 = k2 = 15 and the initial condition as(q1, q2, q3, p1, p2) =(0.1, 0.4, 0.2, 0, 0). We simulate a standard Wiener process inthe same manner as in [20].

First, we consider a scenario where there exists noise, anda feedback controller designed by a deterministic method in[26] is applied. The deterministic controller correspondstothe controller (53) withQ(x) = 0, and we verified that theobjective is achieved with the controller without noise. Fig.2 shows the motion of the coin in theX-Y plane and Fig.3 shows the time responses ofq and p, respectively. They


imply that the behavior of the system seems unstable withthe controller for the deterministic system in the presenceofnoise.

X

Y

Motion of the coin ( deterministic case with noise )

Π0

Fig. 2. Motion of the coin in theX-Y plane in the presence of noise witha deterministic controller

time

q1

time

q2

time

q3

time

p1

time

p2

Fig. 3. Responses ofq andp in the presence of noise with a deterministiccontroller

Finally, we consider a scenario where there exists the samenoise as in the previous scenario and the feedback controller(53) designed by the proposed method is applied. Fig. 4 showsthe motion of the coin in theX-Y plane and Fig. 5 shows thetime responses ofq and p, respectively. They imply that theproposed controller works well even in the presence of noise.

X

Y

Motion of the coin ( stochastic case )

Π0

Fig. 4. Motion of the coin in theX-Y plane with the proposed controllerwith the same noise signal to the previous case

These simulation results demonstrate the effectiveness ofthe proposed framework.

Remark 5:Due to Brockett’s condition [29], deterministicnonholonomic systems can not be asymptotically stabilizedaround a fixed point under any continuous static feedbackcontroller. In the case of the stochastic system (44), it is easily

time

q1

time

q2

time

q3

time

p1

time

p2

Fig. 5. Responses ofq and p with the proposed controller with the samenoise signal to the previous case

shown that the condition in Theorem 2 thatΓ∩ Π = 0 does

not hold. SinceL0H(x) = ∂H∂x

f0 +12 tr

∂∂x

(∂H∂x

)⊤

hh⊤

=

0, we haveΓ = R5. From the above argument, we have

Π =

x∣∣∣

∂U∂q

J12(q) = 0, p = 0

. ThereforeΓ ∩ Π 6= 0

is shown. In the literatures [26], [28], a discontinuous staticfeedback controller and a time-varying feedback controllerwhich render the origin of a deterministic nonholonomicport-Hamiltonian system asymptotically stable, are proposed,respectively. In the next subsection, we design a time-varyingfeedback controller which renders the origin of the samesystem in (44) asymptotically stable in probability.

B. Time-varying controller design

In this subsection, we again consider the rolling coin inthe presence of noise described in (44). In subsection V-A,we have designed a continuous static controller which makesthe state converge to an invariant set in probability in thepresence of noise. On the other hand, the literature [28] hasproposed time-varying asymptotically stabilizing controllersfor deterministic nonholonomic Hamiltonian systems. In thismethod, a special class of time-varying generalized canonicaltransformations is introduced in which a time-varying potentialfunction U(x, t) is parameterized by an arbitrary periodicfunction α(q, t) which is the parameter of the change ofthe output. The purpose of this subsection is to design atime-varying feedback controller which renders the originasymptotically stable in probability based on the techniquein [28].

First, we consider the following form of the new Hamilto-nian:

H=H+p⊤α+1

2α⊤α+V (q)=

1

2(p+α)⊤(p+α)+V (q),

whereα(q, t) is any periodic odd function and an appropriatefunctionV (q) should be chosen so thatH is non-negative inthe new coordinate. Here we utilize the following functionswhich are the same as those in [28]:

α(q, t) =(q3 sin t, 0

)⊤, V (q) =

1

2q⊤Kq, (54)

whereK is defined asK := diagk1, k2, k3 with appropriatepositive numbersk1, k2 andk3. Then let us construct a time-varying stochastic generalized canonical transformationwith


the following coordinate transformation utilized in [28]:

q =(q1 − q3 cos t, q2, q3

)⊤, p =

(p1 + q3 sin t, p2

)⊤.

(55)

In order to obtain a time-varying SGCT, let us decide the otherdesign parametersβ(x, t), P (x, t) and Q(x, t) by utilizingTheorem1. The following choice satisfies Eq. (22):

P (x, t)=05×5 , Q(x, t)=

(03×3 03×2

02×3Q44(x, t) 0

0 Q55(x, t)

)

(56)

β(x, t) =(q3 cos t+k1(q1−q3 cos t)+p2 sin t sin q1

+(p1+q3 sin t)Q44(x, t),

−k1(q1−q3 cos t) sin q1 cos t+k2q2 cos q1

+k3q3 sin q1+p2Q55(x, t))⊤, (57)

where the free parametersQ44(x, t) andQ55(x, t) should bechosen so thatQ(x, t) in Eq. (56) becomes symmetric posi-tive semi-definite. Furthermore, in order to obtain stochasticpassivity, let us derive another condition forQ44(x, t) andQ55(x, t) from Lemma 4. It follows from the inequality (33)that

1

2(h1(x, t)2 + h2(x, t)2)

≤ (p1 + q3 sin t)2Q44(x, t) + (p2)2Q55(x, t). (58)

In what follows, we suppose that there exist functionsQ44(x, t) andQ55(x, t) such that the equality in the inequality(58) holds andQ(x, t) in Eq. (56) becomes symmetric positivesemi-definite.

The transformed system is given byg(x) = (02×3, I2)⊤,

y = p and

J(x, t) =

0 0 0 1 − sin q1 cos t0 0 0 0 cos q1

0 0 0 0 sin q1

−1 0 0 0 sin q1 sin tsin q1 cos t − cos q1 − sin q1 − sin q1 sin t 0

∣∣∣∣∣∣

x = Φ−1(x, t)t = t

R(x, t) = Q(x, t)

∣∣∣∣ x = Φ−1(x, t)

t = t

(59)

h(x) =

(

03×2diagh1(x, t), h2(x, t)

)∣∣∣∣∣ x = Φ−1(x, t)

t = t

.

Eq. (59) implies that the transformed system has the form of(2). Since this system obtains stochastic passivity, it canbeeasily proven by Theorem 3 thatlimt→∞ y = 0 almost surelywith the unity feedbacku = −y. Generally, Theorem 3 onlyguarantees the convergence of the output. However, in thiscase, we can show that the unity feedback also renders theorigin of the system (44) asymptotically stable almost surely.Let u = y = p ≡ 0 of the system (59). Then it followsfrom Eq. (55) and the condition (58) thath(x, t) ≡ 05×2. Theliterature [28] has proven that the transformed system (59)has zero-state observability with respect tox without noise.These two facts prove the claim. Eventually, we obtain the

following time-varying feedback controller which renderstheorigin asymptotically stable almost surely as

u = −β(x, t)− (y + α(x, t)) (60)

(see, Eqs.(54) and (57) forα(x, t) andβ(x, t)).Finally, let us show some simulation results. Here we

consider the noise port ash1(x, t) = 0 andh2(x, t) = k2p2

and we set a matrixK in (54) asK = diag1, 1, 1 , designparameters asQ44 = 0 and Q55 = k22/2 with k2 = 8 andthe initial condition as(q1, q2, q3, p1, p2) = (0, 0, 1.0, 0, 0).Since convergence is slow and oscillatory in the case ofequipping time-varying feedback controllers [30], [28], weexecute subsequent simulations with different initial conditionand design parameters from those in subsection V-A.

First, we consider a scenario where there exists noise, anda feedback controller designed by a deterministic method in[28], which corresponds to the controller (60) withQ(x, t) =05×5, is applied. Although the convergence is slow and os-cillatory, we verified that the objective is achieved with thecontroller without noise. Fig. 6 shows the motion of the coinin theX-Y plane and Fig. 7 shows the time responses ofq andp. They imply that the behavior of the system seems unstablewith the controller for the deterministic system in the presenceof noise.

X

Y

Motion of the coin ( deterministic case with noise )

Fig. 6. Motion of the coin in theX-Y plane in the presence of noise witha time-varying deterministic controller

Time

q1

Time

q2

Time

q3

Time

p1

Time

p2

Fig. 7. Responses ofq and p in the presence of noise with a time-varyingdeterministic controller

Finally, we consider a scenario where there exists the samenoise as in the previous scenario and the feedback controller(60) designed by the proposed method is applied. Fig. 8 showsthe motion of the coin in theX-Y plane and Fig. 9 showsthe time responses ofq andp. They imply that the proposedcontroller works well even in the presence of noise, although


the convergence is slower and more oscillatory than the casewithout noise.

X

Y

Motion of the coin ( stochastic case )

Fig. 8. Motion of the coin in theX-Y plane with the proposed time-varyingcontroller with the same noise signal to the previous case

Time

q1

Time

q2

Time

q3

Time

p1

Time

p2

Fig. 9. Responses ofq andp with the proposed time-varying controller withthe same noise signal to the previous case

Those simulation results demonstrate the effectiveness ofthe proposed method.

VI. CONCLUSION

This paper has introduced stochastic port-Hamiltonian sys-tems (SPHS’s) and has clarified some of their properties.First, we have shown a necessary and sufficient conditionfor the SPHS structure to be preserved under coordinatetransformations. Second, we have derived a condition toachieve stochastic passivity of the system. Third, we haveintroduced stochastic generalized canonical transformations(SGCT’s). Fourth, we have provided a condition that thetransformed system by the SGCT becomes stochastic passiveand have proposed a stabilization method based on bothstochastic passivity and SGCT’s. Although stabilization ofgeneral nonlinear stochastic systems is very difficult, we focuson an important class of them and can provide a systematicprocedure with explicit conditions for a stabilizing controller.Finally, numerical simulations demonstrate the effectivenessof the proposed method.

Since the condition (22) is a second order partial differentialequation, we did not investigate how to solve it yet. However,since it is linear, we now try to parameterize the solution inthe case of mechanical systems. A systematic solution to thecondition is our future work.

In the proposed SPHS (2) as well as the deterministicHamiltonian system (1), the output is defined so that the

input and the corresponding output are the effort and theflow variables, whose product is always power. This definitionenables the system to possess an intrinsic property such aspassivity. If an extra structure is added to the output and/or thenoise port, it might induce a useful property between the noiseand the system such as the noise-to-state stability conceptproposed in [31]. Further investigation of another possiblestructure of the output and noise port is also our future work.

ACKNOWLEDGMENT

This work was supported in part by JSPS Grant-in-Aid forResearch Activity Start-up (No. 22860041).

The authors would like to thank Prof. Yoshikazu Hayakawaand Prof. Shinji Hara for their helpful comments and discus-sions on this work, and the anonymous reviewers for theirvaluable suggestions.

REFERENCES

[1] N. Ikeda and S. Watanabe,Stochastic differential equations and diffusionprocesses, 2nd ed. Amsterdam/Tokyo: North-Holland/Kodansha, 1989.

[2] T. Misawa, “Conserved quantities and symmetry for stochastic dynam-ical systems,”Phys. Lett. A, vol. 195, pp. 185–189, 1994.

[3] R. S. Bucy, “Stability and positive supermartingales,”J. DifferentialEquations, vol. 1, no. 2, pp. 151–155, 1965.

[4] H. J. Kushner,Stochastic Stability and Control. Academic Press, 1967.[5] X. Mao, Stability of Stochastic Differential Equations with Respect to

Semimartingales, ser. Pitman Res. Notes Math. Ser. 251. LongmanScientific and Technical, 1991.

[6] P. Florchinger, “A passive system approach to feedback stabilization ofnonlinear control stochastic systems,”SIAM J. Control Optim., vol. 37,no. 6, pp. 1848–1864, 1999.

[7] C. I. Byrnes, A. Isidori, and J. C. Willems, “Passivity, feedbackequivalence, and the global stabilization of minimum phasenonlinearsystems,”IEEE Trans. Autom. Contr., vol. 36, no. 11, pp. 1228–1240,1991.

[8] P. Florchinger, “Feedback stabilization of affine in thecontrol stochasticdifferential systems by the control lyapunov function method,” SIAM J.Control Optim., vol. 35, no. 2, pp. 500–511, 1997.

[9] W. Zhu and Z. Huang, “Stochastic stabilization of quasi-partially inte-grable Hamiltonian systems by using Lyapunov exponent,”NonlinearDynamics, vol. 33, pp. 209–224, 2003.

[10] B. Maschke and A. J. van der Schaft, “Port-controlled Hamiltoniansystems: modelling origins and system theoretic properties,” in Proc.2nd IFAC Symp. Nonlinear Control Systems, 1992, pp. 282–288.

[11] A. J. van der Schaft,L2-gain and Passivity Techniques in NonlinearControl. Berlin: Lecture Notes on Control and Information Science,1996, vol. 218.

[12] S. Satoh and K. Fujimoto, “On passivity based control ofstochasticport-Hamiltonian systems,” inProc. 47th IEEE Conf. on Decision andControl, 2008, pp. 4951–4956.

[13] ——, “Stabilization of time-varying stochastic port-Hamiltonian systemsbased on stochastic passivity,” inProc. IFAC Symp. Nonlinear ControlSystems, 2010, pp. 611–616.

[14] K. Fujimoto and T. Sugie, “Canonical transformation and stabilization ofgeneralized Hamiltonian systems,”Systems & Control Letters, vol. 42,no. 3, pp. 217–227, 2001.

[15] ——, “Iterative learning control of Hamiltonian systems: I/O basedoptimal control approach,”IEEE Trans. Autom. Contr., vol. 48, no. 10,pp. 1756–1761, 2003.

[16] S. Satoh, K. Fujimoto, and S. Hyon, “Gait generation viaunified learningoptimal control of Hamiltonian systems,” 2013, to appear inRobotica.

[17] S. Satoh, “Control of deterministic and stochastic Hamiltonian sys-tems application to optimal gait generation for walking robots,”Ph.D. dissertation, Nagoya Univ., Aichi, Japan, 2010, available :http://home.hiroshima-u.ac.jp/satoh/index.html.

[18] S. Arimoto, S. Kawamura, and F. Miyazaki, “Bettering operation ofrobotics,” J. Robotic Systems, vol. 1, no. 2, pp. 123–140, 1984.

[19] X. Mao, “Stochastic versions of the LaSalle theorem,”J. DifferentialEquations, vol. 153, pp. 175–195, 1999.


[20] A. Ohsumi, Introduction to Stochastic Systems. Tokyo: AsakuraPublishing, 2002, (in Japanese).

[21] K. Ito, “On a formula concerning stochastic differentials,” Nagoya Math.J., vol. 3, pp. 55–65, 1951.

[22] B. Øksendal,Stochastic differential equations, An introduction withapplications, 5th ed. Berlin Heidelberg New York: Springer-Verlag,1998.

[23] H. K. Khalil, Nonlinear Systems, 3rd ed. New York: MacmillanPublishing Company, 1996.

[24] H. J. Kushner,Stochastic stability, ser. Stability of Stochastic DynamicalSystems; Lecture Notes in Math. 294, R. Curtain, Ed. Berlin and NewYork: Springer-Verlag, 1972.

[25] A. Isidori, Nonlinear control systems, 3rd ed. London: Springer-Verlag,1995.

[26] K. Fujimoto and T. Sugie, “Stabilization of a class of Hamiltoniansystems with nonholonomic constraints via canonical transformations,”in Proc. European Control Conference, 1999.

[27] A. J. van der Schaft and B. M. J. Maschke, “On the Hamiltonian formu-lation of nonholonomic mechanical systems,”Reports on MathematicalPhysics, pp. 225–233, 1994.

[28] K. Fujimoto and T. Sugie, “Stabilization of Hamiltonian systems withnonholonomic constraints based on time-varying generalized canonicaltransformations,”Systems & Control Letters, vol. 44, no. 4, pp. 309–319,2001.

[29] R. B. Brockett, “Asymptotic stability and feedback stabilization,” inDifferential Geometric Control Theory, R. W. Brockett, R. S. Millmann,and H. J. Sussmann, Eds. Basel-Boston: Birkhauser, 1983, pp. 181–191.

[30] J.-B. Pomet, “Explicit design of time-varying stabilizing control lawsfor a class of controllable systems without drift,”Systems & ControlLetters, vol. 18, pp. 147–158, 1992.

[31] H. Deng, M. Krstic, and R. J. Williams, “Stabilizationof stochasticnonlinear systems driven by noise of unknown covariance,”IEEE Trans.Autom. Contr., vol. 46, no. 8, pp. 1237–1253, 2001.

PLACEPHOTOHERE

Satoshi Satohreceived his B.Sc., M.Sc. and Ph.D.degrees in engineering from Nagoya University,Japan, in 2005, 2007 and 2010, respectively.

He is currently an Assistant Professor of Facultyof Engineering, Hiroshima University, Japan. He hasheld a visiting research position at Radboud Univer-sity Nijmegen, The Netherlands, in 2011. His re-search interests include nonlinear control theory andits application to stochastic systems and robotics.

Dr. Satoh received the IEEE Robotics and Au-tomation Society Japan Chapter Young Award in

2008 and the SICE Young Author’s Award in 2010.

PLACEPHOTOHERE

Kenji Fujimoto received his B.Sc. and M.Sc. de-grees in engineering and Ph.D. degree in informaticsfrom Kyoto University, Japan, in 1994, 1996 and2001, respectively.

He is currently a Professor of Graduate Schoolof Engineering, Kyoto University, Japan. From 1997to 2004 he was a Research Associate of GraduateSchool of Engineering and Graduate School of Infor-matics, Kyoto University, Japan. From 2004 to 2012he was an Associate Professor of Graduate School ofEngineering, Nagoya University, Japan. From 1999

to 2000 he was a Research Fellow of Department of Electrical Engineering,Delft University of Technology, The Netherlands. He has held visiting researchpositions at the Australian National University, Australia and Delft Universityof Technology, The Netherlands in 1999 and 2002, respectively. His researchinterests include nonlinear control and stochastic systems theory.

Dr. Fujimoto received The IFAC Congress Young Author Prize in IFACWorld Congress 2005.

JOURNAL OF LA Passivity Based Control of Stochastic Port ... · bilization of the control system,...

Documents

Transcript of JOURNAL OF LA Passivity Based Control of Stochastic Port ... · bilization of the control system,...