SAFOR (68-0327A PROBABILISTIC APPROACH TO THE PARAMETER SENSITIVITY PROBLEM *

B. LIU and M. E. VAN VALKENBURG L Ditftbutlon of this docurwent Is u,,,.-Princeton University .... 1 t, ,],PrinceLon, New Jersey d M

ABSTRACT t , .

The deviation of a system function from its nominal valuedue to random variations or the inherent uncertainty of thesystem parameters is m~xmnined. u-r"ndp on various probabililait _measures of this deviation are discussed and examples are given.A comparison is made with the deterministic multiparametersensitivity function.

""0 The significance of parameter variation in linear L iworks and systems is well known for the case of deterministicmodels. [1-7]. In many cases, however, it is more appropriateto make use of a probabilistic approach in studying the sensi-tivity problem. This is particularly true when the number ofparameters in the network is not small. I

Considerable interest has been s)-( ,n in wt .6-vase vx-i-ation as a sensitivity measure of a system, and procedureshave been given for maximum (minimum) search by computer [8].An alternative approach to this problem has been that of de-riving bounds of the variations [9]. However, the boundsderived so far tend to be pessimistic [81. We g~vj here asimple way of calculating the approximate worst-case perfor-mance when the vari'ation of parameters is not large. A com-parison is made of this variation with the results obtain-ed using the probabilistic approach.

II. SINGLE PARAMETER CASE

Consider the system function which describes a linearnetwork, H(w,x) where W is frequency and x is an element inthe network. It is well known that H is a bilinear functionof x, i.e., 4

) A(W) + x (B)H (W X) = c(W) + x D(W)

Classical sensitivity SH, defined by

n x R H (2)sX='•n-x H •x

has been used in the study of the change of system performancewith respect to parameter variation. The real and imaginary

Hi parts of S are related separately to the amplitude and phase:;-- variationsof H caused by a small change Ax in x. The first-oX order effects are given byzE

-z AI..I=,•I, e SH &x (3)T _" Fx 1K

RV P•r.ented at tne Fi•Fth Annual, AZierton Conference on Cirouit andSSyo ter? Theory, October 4-6, Z967. To be pubZiohed in the Prooeedingo

of tne Conference.

0

a 6

a

-- - a A

-2-

and

A arg H Im SH Ax (4)xx

where iHj and 6Hfl and aarg H denote respectively the magnitudeof H. the change in 1HIr and the change in the argument of H.

Traditionally, the problem of system sensitivity and para-meter variation has been treated by using a deterministicapproach by assuming the variation 6x is a fixed quantity.However. strictly pec-tking, the e....ent values in a system areonly known probabilistically in view of inherent uncertainties.In addition, our knowledge of the variation of the variationof these element values due to aging, environment changes, etc.is conditioned by a certain degree of incompleteness. It istherefore more reasonable to take x as a random variable whosemean x is regarded as its nominal value and whose variance

a2 gives a measure of .to inherent inaccuracy.•x

At a fixed w, the expected value of H(w,x) as given inEq. 1 is given by

EjiW x) - iD ( p(x) dx (5)

where E denotes the expectation, and p(x) is the probabilitydensity of x. Since one normally takes

H(W,x) - A(M) + &(W) (6)C (W) + ; (w)

as the nominal H, it is of interest to compare the two nominalvalues of H as given by Eqs. 5 and 6.

We may write x as x + A, where the random variable A has

zero mean and variance a2 . If ax is small, H(w,x) may be

expaned in a power seriesH(W,x) - H(W,x) + - A + -1 2H A2 + (7)

a 2 2 a ..

and the first few terms can be taken as a good approximation.Since E(6] - 0, it can be seen that the difference betweenE(H(w,x)) and H(W,x) is a second-order effect.

This result can also be seen in terms of a graphicalrepresentation. The bilinear Eq. 1 maps the real x line intoa circle for a fixed w as illustrated in Fig. l(a). Supposex deviates from its nominal value i by a fixed 6. ThenH(W,x + 6) should be on the circle as shown in the slightlyexpanded illustration about H(W,x) of Fig. l(b). Thn approxi-

mation, H(wi) +A- 6, obtained by taking only the first two

terms in the series expansion, Eq. 7, lies on the tangent tothe circle at H(W x). Similarly, H(w,x-6) is on the circle

and H(wR) - _H 6 is on the tangent. Since the approximationax

-3

lies on a straight liic and since the random deviation isassumed to have zero mean, the resulting approximation toE(H(W,x)) coincides with H(W,x). The actual EtH(w,x)) wouldlie somewherp inside the circle as shown. By taking moreterma in the power series, the approximation to the expectedvalue of H would approach its true value.

If the excmt distribution of x is known, or a roaionableone can be assumed, then EIH1-1,,) -can be determined, thoughin most cases numerical calculation by a computer Is required.

2F'or example, if x is gaussian with mean ; and variance c2 Eq.5 can be written a31E[(•x) -B+ (E- B F)D .± 4 W(4•LD-F) (8) }

E (H (w, x)~ -A+m K ')- 8D D 2 a 2

where E - A+xB, F - C+iD and the function W(Z)is rejated tothe complementary error function by [101 W(J- W" erfc(-iz)If x is uniformly distributed between x E .30 a, the result is

B E +D - 13r F (9)EJHw~)) 2./-ax D 4nF - qra xD(9

Provided C/D is not real.

III. MULTIPLE PARAMETER CASE

Let xI and x 2 be two elements in a linear network. The

system function H(u., x1 , x2 ) is then bilinear in each of x,

and x 2 . We take x1 and x2 to be independent random variables- 2 2

with means R. and x 2 and variances al and 02 . Again,

H(W', Rs i2) is usually taken to be the nominal H and it can

be argued as before that H(w, .1, x2 ) is a good approximationof the actual mean E(H(w, xI, x2)).

At each w, and for each fixed x2 , the locus of H as a

function of x, is a circle, and for each xj, the locus of H

as a function of x 2 is also a circle. Suppose the value of xi

is to vary between Xl * 61 and the range of x2 is R2 ± 6 2

then the variation of H(w, x1 , x2) is as illustrated in Fig. 2.

The value of H is then limited to the region bounded by thefour dashed circular arcs. If none of these four arcs istangent to circles with center at the original, or to straightlines passing through the origin, it is seen that the extremesof IHI and arg H can occur only at the *corner points," a,b,c,or d. Even when this is not true because some of the arcsare tangent to circles with center at origin or to straight

1. For simplicity, the argument w is sometimes omitted.

-4-

lines through the origin, the extreme values evaluated at thecorners are still very good approximations to the true extremes.The above argument can be extended to situations withl more f•hantwo parameters. Figure 3 illustrates a 3-parameter case.

The significance of the problem of worst-case variationhas been recognized by many authors. Most of the results inthe past have concentrated along two lines. The first is todevice bounds which usually turn out to be rather pessimistic.The second Al to actually search for the extremes in the para-meter space and usually requires considerable computationaleffort. The observation made above can be used to determineapproximately the worst case variation of H by simply evalua-ting H at all of the corner points and comparing the results.Thus for a network of n parameters, x. with range X 6

i = 1,2,...n, one needs to calculate H for each of the 2combinations of the parameter values, xi = xi 1 6i. Theextremums obtained in this manner can be taken as approxima-tions of the actually extremums.

As an illustration, consider the third-order Chebyshevlow-pass filter shown in Fig. 4. The sensitivity problem ofI.;,is network has been studied by Kelly (8J who calculated theactual maximum and minimum of the amplitude characteristic.The result of calculation based upon the approach outlinedabove is sho%.r in Fig. 5 for k5% and *10% variations Qf allelements. Also shown is the actual maximum and minimum ascalculated by Kelly. For this order of parameter variation,it is seen that our method gives reasonably good approximationsexcept near the cut off frequency where the change is fast.However, if the range of variation is increased, one shouldexpect that our approximation procedures will not give reliableresults.

IV. MULTIPARAMETER SENSITIVITY AND STATISTICAL VARIATION

Let xi, i = 1,2,...n be element values. The first order

approximation to the variation in the system functionH(WXlX 2 #...,Xn) are related to the real and imagi.nary parts

of the sensitivities through the relationships

AIUI n H Ax(Re 6 X (10)i-I bi) xi

and n Ax.

A arg H P E (Im S ) -. &a (11)i-l xi xi

where the sensitivities SH are defined in the usual manner,i.e.,

S H - 8 Xi (12)Xj i x H

and the Axi's are the deviations from their nominal values xir

Wt take the xi's to be independent random vnriables with mean

i and variance oa. Thus

x, - , + 6. (13)

with2 2w[6i- M o, E(6121] = i2 (14)

Equations 10 and 11 may be written as*

HI 6.-'j~ 7H Owjl (Re SH x x15i-i (15)

and n 66atrg H Aj E (Is S (6i,-1 xi

with lii IH(w,x 1,x 2 ,...x,)I. in view of Eqs. 13 and 14, we

have f.Bs~L -0o (17) ,

E~n arg H) - 0 (18)

i- (R.S O12 (19)n 2T~HI.IJ X i

d arg 2 (20)

i-i x i i

If the range of 6. is restricted to i AV then it is clearthat the extrema of 'A IH/IHI are given by

1H n sHmaxLI - E IReS I Li (21)

•.nd A_'H H Limin - E IRe S I (22)

i-i xi xi

One may take *kai, with a proper choice of k, as the

range of the variation of parameter2 xi. Then the range ofva•-iation using the extrema is

n H .

RangeI - 2k E IRe SH I X (23)

2 That is, if k is taken as 3, then the element will be withinthis range with probability better than 0.997.

*In Leh discussion to follow, we replace w by - rather thandiscuss the precise meaning of the approximation.

-6-

and the range ubing ko where 7 i the variance of

A IHI/I I, and is T

n 2a, 2Range 1 1 - 2k 4 E (Re Sx) (.)

That Range1 I Range.I1 follows from an elementary inequality.

in fact, for most cases, one will find Range1 1 to be consid-

erably smaller than Range .

The third-order Chebyshev filter is used agan as anillustration. The sat of the absolute value Re S in calcu-K.

1

lated to be 1.3390 at w/27 = 0.1, 1.40.' at w/2n - 0.15, and2.8755 at w/2n - 0.2, where as the square root of the sum ofthe squares of the real parts of the individual sensitivitiesare 0.7756, 0.7310, and 1.6730 at these frequencies. That is,for this example, the extreme range of variation RangeI isalmost twice as large as a meaningful measure should be.Figure 6 shows the ranges as determined by the extrema and asdetermined by the variances of AIHI/IHI at w- 1.0 and w- 1.5,using k = 3. Also sketched in the figure are the probabilitydensity functions.

When the distributions of the elements are not gaussian,one can not draw the definitive result as given above. But ifthe number of terms. in Eq. 15 is not too small and if no termin the sum clearly dominates, then the final distribution oft1HI,/IHI will still be approximately gaussian [11). Supposethe elements in the example of the Chebyshev filter are uni-formly distributed, then the probability density of AIHI/IHIis that of the successive convolution of five rectangularshapes of proper width.

The result is also sketched in Fig. 6. If one takes *30as the range of variation, it is seen that with probabilitybetter than .92, AIHI/IHI will be within the range when theelements are unifomnly distributed.

ACKNOWLEDGEMENT

This research is sponsored by the Air Force Office ofScientific Research, Office of Aerospace Research, UnitedStates Air Force under AFOSR Grant 1333-67 and by the NationalScience Foundation under Grant GK-1439.

-7-

REFERENCES

I. Blostein, M.L., "On the Effect of Loss in Filter Networks,"Proc. 3rd Allerton Conference on Circuit and SystemTheory, 1965: p. 421-438.

2. Bihovski, M.L., "The Accuracy of Electric Network Analyzers"IZV an SSSR, Otd. Tekhn. Nauk, No. 8, 1948.

3. Cr"7. J.B., Jr. and W. R. Perkins, "The Role of Sensitivityin the Design of Multivariable Linear Systems," Proo- NEC,1964; pp. 7.12-745.

4, Cruz, J.3., Jr. and W. R. Perkins, "A New Approach to theSensitivity Problem in 9Itivariable Feedback SystemDesign," IEEE Trans. on Automatic Control, vol. AC-9, pp.216-223; July 1964,

5. Goldstein, A.J. and F. F. Kuo, "Multiparzeter Sensitivity"IRE Trans. on Circuit Theory, vol. CT-8, No. 7, pp. 177-178;June 1961.

6. Horowitz, I.M., Synthesis of Feedback Systems, AcademicPress, New York, 1963.

7. Lee, S.C.., "On Multiparameter Sensitivity," hirdAllerton Conf. on Circuit and System Theory, October 1965,pp. 407-420.

8. Kelly, Gerry L., "Nonlinear Programming and MultiparameterSensitivity in Linear Time-Invariant Networks," ReportR-301, Coordinated Science Laboratory, University ofIllinois, June 1966.

9. Hakimi, S.L. and J. B. Cruz, Jr., "Measures of Sensitivityfor Linear Systems with Large Multiple Parameter Variation"IRE WESCON Convention Record, Part 2, pp. 109-115; August1960.

10. Abramowitz, M. and 1. A. Stegun, Handbook of MathematicalFunctions, Dover, Ne'- York, 1965, pp. 297.

11. Papoulis, Athanasios, Probability, Random Variables, andStochastic Processes, McGraw-Hill Book Co., 1965.

(i') (b)

A Z* I ( '•) * •xI H

HF(, i + a

IH(WXb)j

EI(Wx) a

Arg HRe-H f ixed

FigJ. 1

a 1" 1,c+6 1 ox2 varying

Im H X =x1 -o1 x 2 varying

• /d ,/ xl-1-6l'x2 varyingS//

/ 2w.2- 6 28 x varying

2 x 2 =x2X1 varying F

xXliX 2+x 1 varying,~varyingRe~j

X" f +6x x -6 ,x 3 varying

2 ~ ~~~ / 2X 1 1 ~nX,2 vdri.x r X ,6X+x varaiig2 2f3- , // 7 1

1 3 3vr2

3 varying x / 3\v2 /

,•2#rng -l., 1 / 3 I"V\ \ -x 3 varying

Fig. 30

g 3 varying

itt

I-9-

LI

i.l 1.7002T 2.5965 1.0Vout

.5-0. lx2n ,T xi gaussian

±10% varitio .��x uniformly distributed, (a)

0.

-. 5 - ".0 - .05 .5 of . 5 -rfrFig. 6

15u-O.15x21? x, gaussina

+5% variation 1.0x•i uniformly distributed

(b)

..15 p1 5 0 5 .10 .5 +

10

V..w.

00

00o i 00 4 In

0 4 24 uo to a >

+2 +I +I +

x x

OR~NAiNACI~TV~a~r~ee~ht)DOCUMENT CONTROL DATA.- R & D SEUTYCSSFAIO(Ser'Ifify or InP Or'• tt in of t IftI., h. , of ,P- fon t , t o, id~r mofCR rl, o•Anf lnA lll moot be entoered when tho veto tl report iq caIn)lfood)

I . ORIGINATING ACTIV¢ITY ('Catpotate attlhor) Af.aQ REPORT SE[CURITY CL.ASSItFICATI[ON

Princeton University UNCLASSIFIEDDepartment of Electrical Engineering 2b. GROUPPrinceton, New Jersey 08540

3. REPORT TITLE

A PROBABILISTIC APPROACH TO THE PARAMETER SENSITIVITY PROBLEM

4. OESCRIP TIVE NOTES (Typ* of report and Inctusiv, date,)

Scientific InterimS. AUTNORIS) (First name, middle initl•l, last name)

B. Liu 4 M. E. Van Valkenburg

. REPORT DATrE 7a. TOTAL. NO. OF PAGES Tb. NO. OP RaPs

R October 1967 10 116a. CUNTRACT OR GRANT NO. -30. ORIGINATOR*S REPORT NUMBERIS)

AF-AFOSR-1333-67b. PROJECT NO. 9749-01

c 61445010? o..-.-R PoRt 0,SI (Any othernumbers to. at ma,.h•- ab d

d. 681304 68-0tO. DISTRIB1UTION STATEMENT

1. Distribution of this document is unlimited.

If. SUPPLEMENTARY NOTES 12. SPONSORING MILITARY ACTIVITY

PROCEEDINGS Air Force Office of ScientificProc 5th Ann. Allerton Conf. on Research (SRMA)Circuit and. Sys. Theory, Ch rnstagn, 1400 Wilson Boulevard

*.. Arlington, virgini~a 22209

The deviation of a network function from its nominal value due torandom variations or the inherent uncertainty of network parameters isstudied in this paper.

Consider the network function H('x,x,,x7,..Q.Xn) where w is thefrequency variable and the x's are the p raffieters. It is well knownthat H is bilinear in each of the x's. Each xi is a random variablewith mean Sri and variance zi2. The mean'-i is usually taken to be thenomirnal value and the variance is related to the tolerance. It isshown that the mean of H, E(Hj, is approximately equal to H(W,' l,72,...the value of H when all the parameters take on their nominal va ues.For small c /., the variances of H are shown to be realted to theclassical single parameter sensitivity SH. When the x's are Gaussian,the distribution of H can be determined. When the x's are not Gaussian,various probabilistic bounds can be obtained by inequalities such asthat of Chebyshev.

This paper also presents a method of calculating approximately the* deterministic worst case parameter variation. A comparison is made of

this varLations with the results obtained using the probabilisticb approach, and it i-; shown that the conventional deterministic worst

case design criter Lon is in general too pessimistic, especially whenthe number of parrict.eters is not small.

DD . ...1473 ____________________W,",, 1

,. . ..,fl,-

- -, I n

Network SensitivityMultiparameter variationI To Aerance LimitRandom variation

f r. i

1'--

[ I