~~~~~~~~O39 OS1 MASS*C~~~ ETTS INST OF TEcH cAMBRIDGE E~~~~ RONIC SYST~~~ TC F/G 12/tTHRESHOLD PRINCIPLE. (U

L*ICLASSIFIED ESL.P-727 AFOSR—TR—77—O5SO NL

ENDDATE IFILMED

• 5-77

IIII

p 44

10 ~‘ 2 8 ~j 2 .5

I~I)I~’~ ~~~3.5

111111 • 1 ~

______ L

• *lllllh.25 UD~?iii1L’~ w,~~

NATIONAL BUREAU OF STANDARDSMICROCOPY RESOLUTJON TEST CHART

i~~, 1977 — IR — 7 7 = 05 ~ 0 EsL p-727

0 — -~~

N

I— FURTHER RESULTS ON THE UNCERTAINTY THRESHOLD PRINCIPLE*

by

Richard T. Ku**Michael Athans**

Abstract

Additional. quantitative results are presented forthe existence of optimal decision rules and stochasticstability for linear systems with white random parameterswith respect to quadratic performance criteria , by exam-ining a specific version of a multivariable optimizationproblem.

I

/ i~:: ~~~~~‘;b0~~~e ;

* This research was conducted at the MIT Electronic Systems Laboratorywith support from AFOSR grant 77—3281 and NSF grant SOC-76—05837.

** Room 35—308, Electronic Systems Laboratory, Department of ElectricalEngineering and Computer Science, Massachusetts Institute of Techno—

• logy, Cambridge, Massachusetts, 02139.

C-,This paper has been submitted to the IEEE Transactions on Automatic

j Control.

F’— ~~ ~~~~~~~~~~~~~~~~~~~~~~~~ •~ — — _•

~~~~~;;.• —

~~~~~~~~~~~~~~~~~—- -

- ~~~ (~

iJSCI

~ ~~cr ::~ ~~ L~~C

12

~ted

~.~~~~~~~~~~~~ormat1°~ 0f~ t05r

-2- 1 I

II II

_ _ _

1. INTRODU~~ ION L~ ~ iThis paper considers the optimal stochastic control of a multi-

variable linear system, with a specific structure, with respect to a

quadratic index of performance. The system dynamics are described by

a linear vector difference equation in which both the system matrix (A)

and control matrix (B) are multiplied by white, possible correlated,

scalar random sequences.

A threshold condition involving the maximum eigenvalue of the

• system matrix A and the means, variances, and cross correlations of the

white parameters is obtained. If the threshold condition is violated,

• then there does not exist an optimal solution to the infinite horizon

optimization problem, and the resultant closed-loop system is not stable

in a mean square sense.

The results of this paper represent another manifestation of the

Uncertainty Threshold Principle (UTP) reported by Athans, Ku, and

Gershwin in (1] and represents a specific multivariable extension of the

scalar results reported in (1). It also generalizes the results of

Kutayama (2], which dealt with control—dependent white noise, to the case

of simultaneously, possibly correlated, state- and control—dependent white

noise parameters.

2. P~~ BLEM FORMULATION

Consider a linear stochastic discrete-time system whose dynamics are

described by the following vector difference equation

-~

-~ --~

—3—

x(t+l) = y ( t )A x (t ) + 6 ( t ) B u (t ) + ~~(t ) (1)

where x(t) is the n—dimensional state vector, u(t) is the rn-dimensional

control vector, and ~(t) is white gaussian process noise. Assume that

A and B are constant matrices of appropriate dimensions and that t~~, ~~is a controllable pair.

Further assume that the scalars 1(t) and 6(t) are gaussian white

random sequences (uncorrelated in time) with known stationary statistics.

More precisely, we assume

E{y(t)J = )~ ; E{(y(t) — y) ( y ( t) — y)}= r 6(t ,t) (2 )

E{6(t)} ~ ; E{(ô(t)—~)(6(t) — ~

)} = ~ 6(t ,T) (3)

E{(’y(t) — y) (6(T) — ~)} = A 6( t ,T) (4)

E{~~(t)} = 0 ; E{~~(t)~~’(T)) E ‘5(t,t) (5)

where 6(t,t) is the Kroenecker delta (6(t,T) 1 if t = 1, 6(t,t) = 0 if

t ~ T). Furthermofre assume that the process noise E(t) is mutually inde-

pendent of the random parameters 1(t) and 6( t ) .

We consider a standard quadratic cost functional

J = E{~ ~~~~x’(t)~~x(t) + u ’(t)R u(t)} (6)t0

where Q is positive semidefinite, R is positive definite, and ~ is

an observable pair .

Under th. assumptions that we can measure the entire state vector

x(t) exactly, at each instate of time, we wish to find the causal optimal

control sequence u(0), u(l), u(t), •.. which minimizes the quadratic

—4—

cost (6).

Remark 1 The problem considered by Katayama [2] corresponds to the

case

~~=i , r = o , A = o , : = o (7)

3. SOLUTION

The problem can be readily solved by stochastic dynamic progranmting

(3) for any fixed value of the planning horizon time P. The derivation

is straightforward and hence omitted. Only the results are stated.

The optimal control is obtained using linear state variable feedback ,

i.e.

• u*(t) = - G (t ) x ( t ) (8)

The optimal mXn feedback control gain matrix G(t ) is given by the formula

G( t ) = (B + (~2 + ~)B’K(t+l)B]~~~(~ ~ +A)B’K(t+l)A (9)

The nxn matrix K(t) satisfies a recursive matrix equation of the form

X(t) = ~~~ +r)A’xct+l)A +

- (y~~~+ A)2A’K(t+l)a(R + (~52 +

(10)

with K(T) = 0.

Remark 2 The recursion (10) will be referred to as the UTP matrix

equation; it is similar to a matrix Riccati equation. However, unlike

Riccati equations it cannot be related to a coupled set of linear equa-

tions. Under our assumption the positive definite matrix K(t) exists

— _______________________ ____ ~_~ .—.~. _,Mr

and is bounded for all finite planning horizon times T.

The optimal cost (6) is given by

J*(x(O),T) = ~~x ’( O ) K ( O ) x (O ) + ~ ~~~~K(t)E (11)t=o

4. THE INFINITE HORIZON CASE (T + co)

The interesting results occur as one analyzes the infinite horizon

case, T + ~~~, so as to examine the existence of an optimal solution and

the stabilizability of the stochastic system (1], [ 2 ] . Once more we

shall show that there exists a threshold condition which provides a

dividing line between existence and non—existence of optimal solutions

to the problem as P ~ ~~~~. We summarize the main result as follows.

Theorem 1 (Uncertainty Threshold Principle)

An opcimal solution exists for the problem stated in Section 2 as

T if and ~~~~ if

max ~~~~~.

< 1/B ; i = l,2,...,n (12)

where B is defined by

—— 2(1 6 + 1 k ) >~~~ (13)

and max IX~(A) I denotes the magnitude of the maximum eigenvalue of the

constant system matrix A in the system dynamics (1) .

Before we present the proof of the theorem it is important to make

some remarks.

-6-

Remark 3 In the case of non—random parameters (r = = A = 0), B = o.

This means that given our assumptions of (A, B) controllability and

~~~ ~‘/~j observability, one can always solve the infinite horizon opti-

mal control problem independent of the (open loop) eigenvalues of A.

On the other hand, as the variances F and ~ of the random parameters

increase, then 8 increases, and the value of I/B defines the radius of

a shrinking disc which must contain all the open-loop eigenvalues of A

in order for the problem to have a solution.

Remark 4 If the condition (12) is violated, i.e. if

max IA~(~ I > 1/8 (14)

then there is no solution to the optimal control problem, and one cannot

stabilize (in a mean square sense) the system (1). Under these condi-

tions the optimal cost, J~ , as defined by (1J3 undergoes exponential

growth as T increases

max I A~(8 A)ITJ*(T) > C e ; C = constant (15)

Because of the explosive growth of the optimal cost, then only short

term (small T) decisions make sense; see also (1].

As in the scalar case [1], even if condition (14) holds, the con-

trol gain matrix G(t), see eq. (9), remains well behaved and bounded

— (y + A)

[B’I((t+l)B] 1B’K(t+l)A (16)IIK t+i II-~

c3 ~~~+t~

—7—

Next we present the details of proving Theorem 1. We remark that the

proof essentially uses algebraic manipulations and known properties of

discrete Lyapunov and Riccati matrix equations.

The main idea of the proof is to examine the behavior of u r n K (t ) ,or the behavior “backward in time” of the UTP matrix equation. The

arguments are similar, but not identical, to those given in (2].

For the sake of notational convenience define the scalars

(17)

Then the UTP equation (10) can be written as

K(t) = a1A’K(t+l)A +

— t+l)B[R + ~~— B’K(t+l)B)~~B’K (t+l)A (18)a3

From Eqs . (13) and (17) one sees that

B2 — c~2

a3 (19)

By adding and subtracting

ct2a3A K(t+] (20)

to the right hand side of Eq. (18), and some algebra , Eq. (10) reduces

to

K(t) = B2~ ’~~(t+l)~ +

+ a2cL3A’ (K(t+l) - K(t+l)8(ct3R+ B’K(t+1)B]1B’K(t+l)]A

(2 1)

Attention is focused to the matrix

M(t+l) K(t÷l) — K(t+l)B[a~~ + B’K(t+l)B11B’x(t+l) (22)

_ _ _ _ -j

—8—

Such matrices arise naturally in the Riccati equation of standard linear

quadratic problems where the control weighting matrix is a3R. Under the

given assumptions of [A, B] controllability and ~~~~~,

~.1~’2] observability

it is known that that [4],[5]*

(a) M(t+l) = M’ (t+l) > 0 (23)

(b) There exists a bound

L>M(t) all t (24)

Since M(t+l) is positive definite, so is a2a3A’M(t+l)A. Hence we obtain

x(t) > 82~’~ (t÷u)~, + (25)

From (25) it is obvious that if any eigenvalue of (8 A) is greater than

unity then K(t ) grows without bound backward in time, that lint K(t)

does not exist and that the optimal cost undergoes exponential growth

as indicated by (15). On the other hand from (24) and (25) one obtains

that

K(t) < 82~ ’~~(t+l)~ , + + ct~ct~A’L A (26)

Hence if all eigenvalues of (B A) are less than unity, the right hand side

of the recursion (26 ) will approach a bounded constant solution matrix

and so will K(t ) . Hence , the limiting solution lint X(t) is well defined.

We remark that the above proof requires that B is flX~~ arid nonsingu—

lar, as required in the Corollary of (2). However we believe that this

is a sufficient, but by no means necessary, condition; it could probably

* The notation A > B (A > B) means that A-B is positive definite (A — Bis positive sentidefinite).

—9—

be ren~ ved by a more detailed analysis of the UTP difference equation.

5. ~ON~LUSIONS

The quantitative results of the Uncertainty Threshold Principle

have been extended to a special case of a ntultivariable control problem,

generalizing the results in [1] and [2J .

- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

— 10—

REFER~ I1CES

E l) N . Athans , R. Ku, and S.B. Gershwin, “The Uncertainty ThresholdPrinciple: Some Fundamental Limitations of Optimal Decision MakingUnder Dynamic Uncertainty” , Proc. 1976 IEEE Conference on Decisionand Control, Clearwater , Florida, Dec. 1976, pp. 1142-1145; also,IEEE Trans. on Automatic Control, Vol. AC-22, No. 3, June 1977.

[2] T. Katayania, “On the Matrix Riccati Equation for Linear Systemswith a Random Gain”, IEEE Trans. on Automatic Control, Vol. AC-2l,No. 5, October 1976, pp. 770—771.

[3] M. Aokj, Optimization of Stochastic Systems, N.Y., Academic Press,1967.

(4] A.H. Levis, “On the Optimal Sampled—Data Control of Linear Processes”,Sc.D. Thesis, M.I.T., Cambridge, Mass., June 1968.

• [5] G.A. Hewer, “Analysis of a Discrete Matrix Equation of Linear Controland Kaintan Filtering”, 3. Math. Anal. Appl., Vol. 42, No. 1, 1973,pp. 226—236 .

S

• - S

• .;~~ •

• -

UN CLASSI FIED -

SECURITY C L A S S I F I C A T I O N OF T i-it S P A G E (I+?,an Data ~ n?r~ed) .

D~~ OADT ~~f l r I Iuc~ IT AT I n~j °AGE READ I N ST RU C T I ON S~~~ mi_ t , ,i’ .’ BEFORE C OMP L E T I N G FORM

I. R~~EQ -pItJM~iER -• - - Lz~ GOVT ACCESSION NO. 3. R E CI P I E NT S C A T A L O G NUMBER1. AFOSR . -TR - 77” - 0 58 O 4. T ITLE (an d SubeItt.)

- S. TYP E OF REPORT & PERIOD COVERED

FURTHE R ~ESULTS ON THE UN~~ RTAINTY TH RESHOLD- PRINCIPLE . I Intertm

— - 6. PE~~FQRMJ c O ~~~~~REPORT NUMBER

- ,) ESL—P—727 1

7. A UTH OR(s) -

- è ’~~~O N T RA C 1~~~R ~~R AN.I JLlJbLBER(s)

‘ 1 ~~~~~~~~~ -

~~~7i,9i — AFO~R I~—3281—2;

MIcHAE r~/ATH ANs - ‘ ~- - /- - --

— — _! ‘~ i ‘ — — — /

9. PERFORMING O R G A N IZ A T I O N NAM E AND A DDRESS 10. PROGRA M ELEMENT . PROJ ECT S TA ~~~~~~ —AREA & W ORIC UNIT NUMBERSNASSACFIUSETTS INSTITUTE OF T~~~~~)LOGY / 6’l”2 ‘9

ELECTRONIC SYSTEMS LABORATORY / ~~~~~~ d

CA~3RIDGE , t~ASSACHUSETTS 02139 (~~~~~~~ ~ 23(~4/A1

i t . CONT ROLLI UG OFFICE NAME AND A D D R E S S f~ ~!~~~~AEQ9L.QAIE. - -

AIR FORCE OFFICE OF SCIENTIFIC RESEARCH/N H - -~~~~~~~~ ~A R Q ~~1~977 /

BOLLING AIR FORCE BASE , WASHINGTON , DC 20332 13. NUMBER OF PAGES

10IS. I4ONITORI, IL j AGEt ICY NA MC & A OORESS(II dill Steal leo,,, Controlling Office) IS. S E C U R I T Y CLASS. (ol this report)

~~~~. 1

~~ / UNCLASSIFIED

— / / / . / __________________________- / • IS. . O E C L A S S I F I C A T I O N / C O W i - i G R A D I NG

SCHEDULE

15. O ISTR IB UT!Oi-4 S T A T E M E N T (of t h i s Report)

APPROVED FOR PUBLIC RE LEASE; DISTRIBUTION UNLIMITED.

Ii. DISTRIB UTIO N STATEMENT (of the abstract ent ered I~ Block 20. ii different fro m Report)

IS. S U P P L EM E N T A R Y NOTES

This paper has been submitted to the IEEE Transactions on Automatic Control.

19. KEY WORDS (Continue on tee.,. . aid. if necessary and iden ify by block number)

20. A B S T R A C T (Continue on r.eeral. aid. if nec.$U,y and identity by block number)

~~ J~dditional quanti tat ive results are presented for the existence of optimaldecision rules and stochastic stability for linear systems with white xandomparameters with respect to quadratic performance criteria , by examining aspecific version of a multivariable optimization problem.

j ’:, . -oo ,:

_ _ _ _ _ _ __ _ _ _ _ --•- - - - _ _