IMM

20.11.2002

Introduction to

Interval Analysis

Ole Caprani

Kaj Madsen

Hans Bruun Nielsen

0.25

0.5

0 0.25 0.5

0.25

0.5

0 0.25 0.5

Abstract. Interval analysis was introduced by Ramon Moore in 1959

as a tool for automatic control of the errors in a computed result, that

arise from input error, rounding errors during computation, and trunca-

tion errors from using a numerical approximation to the mathematical

problem. An important application is the enclosure of ranges of func-

tions, which enables us e.g. to solve equations with interval coeÆcients,

to prove existence of certain solutions, and to solve global optimization

problems. We discuss the basic concepts of interval analysis and present

methods for interval solution of some numerical problems.

Contents

1. Introduction 3

2. Interval Arithmetic 6

2.1. Real Intervals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.2. Interval Vectors and Matrices . . . . . . . . . . . . . 6

2.3. Real Interval Arithmetic . . . . . . . . . . . . . . . . . . 8

2.4. Floating Point Interval Arithmetic . . . . . . . . 9

3. Interval Algebra 11

4. Interval Functions 12

4.1. Interval Hulls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

4.2. Interval Extensions . . . . . . . . . . . . . . . . . . . . . . . 13

4.3. Interval Extensions for Rational Functions 14

5. The Set of Intervals as Metric Spaces 16

5.1. Continuity of Interval Functions . . . . . . . . . . . 19

5.2. Lipschitz Continuity . . . . . . . . . . . . . . . . . . . . . . 20

5.3. Width of an Interval . . . . . . . . . . . . . . . . . . . . . . 22

6. Linear and Quadratic Interval Extensions 24

6.1. Linear Interval Extensions . . . . . . . . . . . . . . . . 25

6.2. Mean Value Forms . . . . . . . . . . . . . . . . . . . . . . . . 27

6.3. Quadratic Interval Extensions . . . . . . . . . . . . . 30

7. Accurate Computation of Interval Hulls 32

8. Interval Integration 37

8.1. Basic Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

8.2. Interval Simpson . . . . . . . . . . . . . . . . . . . . . . . . . . 40

9. Roots of Functions 44

9.1. Basic Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

9.2. Interval Version of Newton's Method . . . . . . 46

9.3. Multidimensional Newton's Method . . . . . . . 52

9.4. Krawczyk's Version of Newton's Method . . 57

10. Linear Systems of Equations 61

10.1. Gaussian Elimination . . . . . . . . . . . . . . . . . . . . . 62

10.2. Sign-Accord Algorithm . . . . . . . . . . . . . . . . . . . . 67

11. Global Optimization 73

11.1. Basic Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

11.2. Use of Gradient Information . . . . . . . . . . . . . . 76

References 79

Notation 81

Index 82

1.

Introduction

Ingeneraltherearethreesourcesoferrorinconnectionwithnumerical

computations:roundingerrors,truncationerrorsandinputerrors.For

estimationofthee�ectofsucherrorsitmaybeadvantageoustocompute

withintervalsofnumbers.Inthisintroductionwegivesomesimple

examples.Inthefollowingchapterswegiveamorestringentpresentation

ofintervalcalculation.

Example1.1.

Firstconsiderroundingerrors.Assumethatweworkwith3

signi�cantdigitsandnormalrounding,andwishtocompute

y

=

1�0:531+

0:531

2

2

Weget 0

:469+

0:282

2

;

0:469+0:141;

0:610

Howlargeistheerrorinthiscomputedresult?

Inthissimpleexampleitiseasyto�ndtheanswer:allwehavetodois

touseasuÆcientnumberofdigitsinthecomputation,

0:469+

0:281961

2

;

0:469+0:1409805;

0:6099805

Sotheerrorinthe�rstresultis0:610�0:6099805

=

0:00001950.

Itisnotalwayspossible,however,tousethisapproach.Theremaybe

toomanydigitstokeeptrackofordivisionsmaygivein�nitedecimal

fractionsthatneedrounding.Insteadwemustestimatetheerroror

keeptrackofitduringcomputation.Thelatterisourchoicewhenwe

calculatewithintervals.

Example1.2.

Withtheproblemfromthe�rstexample,thwvalueof0:531

2

iscontainedintheinterval[0:281;0:282],andweget

0:469+

0:531

2

2

20:469+[0:281;0:282]�1 2

�0:469+[0:140;0:141]=

[0:609;0:610]

1.Introduction

4

Thus,weknowthatthetruevalueofyisbetween0:609and0:610.During

computationtheendpointsoftheintervalswereroundedinthedirections

thatensuredthattheresultingintervalcontainedthetrueresult.

By

monitoringintervalsinthisspecialwaywecankeeptrackofrounding

errors.

Example1.3.

Next,welookattruncationerrors.Wewishtocompute

y=e�

0:531

,andusetheTaylorserieswithremainderterm,

ex

=

1+x+

x2

2!

et;

wheretisbetween0andx

Weareinterestedinthecasex<0,andsinceet2[0;1]forallt<0,we

get

ex

21+x+

x2

2!

[0;1]:

(1.1)

Withx=�0:531we�nd

e�

0:531

21�0:531+

0:531

2

2

[0;1]

20:469+[:140;0:141]�[0;1]

20:469+[0;0:141]=

[0:469;0:610]

Againwefoundanintervalthatissuretocontainthecorrectresult.The

computationshowsthatintervalcalculationsallowsussimultaneouslyto

keeptrackofroundingerrorsandtruncationerrors.

Example1.4.

Finally,toillustrateinputerrors,supposethatinsteadof

x=

�0:531weonlyknowthatx2[�0:532;�0:531].Wecanestimate

thee�ectofthisuncertaintyinindata:Equation(1.1)gives

ex

21+[�0:532;�0:531]+

[�0:532;�0:531]2

2

�[0;1]

2[0:468;0:470]+

[0:280;0:284]2

2

�[0;1]

2[0:468;0:470]+[0;0:142]=

[0:468;0:612]

Thecomputationshowsthatallthreesourcesoferrorcanbeincluded,and

theresultisanintervalthatcontainsallvaluesfexgforxintheinterval

[�0:532;�0:531].

5

1.Introduction

Theresultsinthelasttwoexamplesarenotimpressivewithrespectto

accuracy.However,wecancopewiththatbyincreasingthenumberof

digitstorepresenttheintervalendpoints,byincreasingthenumberof

termsintheTaylorseries,andbyusingspecialtechniquestocompute

mappings.Thesesubjectsaretreatedinthefollowingchapters.

2.

IntervalArithmetic

Westartthischapterwithamoreformaldescriptionofintervalsand

thenwespecifywhatwemeanby\calculatingwithintervals".The

treatmentfallsintwoparts.Firstweassumethatthemathematically

correctversionofthefourarithmeticoperationsisavailable.Next,we

becomerealistic{weuse oatingpointnumberstorepresenttheinterval

endpointsand oatingpointarithmetictodothecalculationswiththese

endpoints.

2.1.

RealIntervals

A

realintervalA

isthebounded,closedsubsetoftherealnumbers

de�nedby A

=

[a;a]=

fx2Rja�x�ag;

wherea;a2Randa�a.WeuseI(R)todenotethesetofsuch

intervals.AsingletonAisadegenerateinterval:a=a.

Forlateruseweintroducefoursimpleconcepts,themidpointm(A),the

radiusr(A),theupperboundjAjandthewidthw(A)oftheintervalA,

m(A)=

1 2(a+a);

r(A)

=

1 2(a�a);

jAj=

maxa2

A

jaj=

maxfjaj;jajg;

w(A)=

a�a=

2r(A):

(2.1)

NotethatifX

isasingleton,thenm(X)=X

andr(X)=0.

2.2.

IntervalVectorsandMatrices

AnintervalvectorVinI(Rn)hasncomponents(coordinates),eachof

whichisaninterval,Vi2I(R);i=1;:::;n.

Theconceptsde�nedin(2.1)havestraightforwardextensionstointerval

vectors.The�rstthreearede�nedcomponentwise,

7

2.IntervalArithmetic

m(V) i=

m(Vi);

r(V) i=

r(Vi);

jVj i=

jVij:

(2.2)

Thus,eachoftheseisavectorinRn.Weshallfurthermakeuseof

thewidthandthenorm

ofanintervalvector.Bothofthesearescalars,

w(V);kVk2R+.

w(V)=

2kr(V)k1

=

maxi

fw(Vi)g;

kVk=

kjVjk1

=

maxi

jVij:

(2.3)

Example

2.1.

LetV

=

([�3;1];[3;5])�I(R

2).Then

m(V)=

(1;4);

r(V)=(2;1);

jVj=(3;5);

w(V)=4;

kVk=5:

AnintervalmatrixM

inI(Rm�

n)hasm

rowsandncolumns.Each

elementisaninterval,Mij2I(R);i=1;:::;m;j=1;:::n,andagain

theconceptsfrom(2.1)arede�nedelementwise,whilethescalarsfrom

(2.3)generalizeto

w(M)=

maxi;j

fw(Mi;j)g;

kMk=

kjMjk1

=

maxi

fX j

jMijjg:

(2.4)

Example2.2.

LetM

=� [4;6]

[0;2]

[�2;0][2;4]� .

Thenm(M)=

� 51

�13

� ,

r(M)=

� 11

1

1� ,

jMj=

� 62

2

4� ,

w(M)=2,

kMk=8.

Example2.3.

LetY2I(Rm

)denotetheintervalvectorobtainedasY=

MV,whereM

2I(Rm�

n)andV2I(Rn).Itcanbeshownthatwiththe

de�nitionsin(2.3)and(2.4)itholdsthat

kYk�kMk�kVk:

Weshallusethisappraisalinchapters9and10.

2.3.RealIntervalArithmetic

8

Notetheequivalencewiththein�nitynorm

ofrealvectorsandmatrices.

ForV2Rn

andM

2Rm�

n

thenormskVk 1andkMk 1arede�nedasin

(2.3)and(2.4),respectively,andkMVk 1�kMk 1�kVk 1.

Nowwearereadytospecifywhatwemeanby\calculatingwithin-

tervals".Thenextsectiondiscussesarithmeticoperationswithsimple

intervals.

2.3.

RealIntervalArithmetic

Let

�denoteoneofthefourarithmeticoperators,+,�,�or=.For

A;B2I(R)wede�ne

A�B=

fa�bja2A

^b2Bg;

(2.5)

withA=B

unde�nedif02B.Thefourarithmeticoperationsarecon-

tinuousmappingsofR

2ontoRandAandBareboundedandclosed.

Therefore,alsoA�Bisaninterval;A�B2I(R).Thus,Equation(2.5)

de�nesfouroperatorsonI(R).Thede�nitionensuresthattheresult-

ingintervalA�Bcontainsallpossibleoutcomesfromapplying�with

operandsfromAandB.Thede�nitionisnotoperational,however;it

doesnotshowhowto�ndtheendpointsoftheintervalA�B,butitis

easyto�ndmaximumandminimumforthefourarithmeticoperations,

andthisleadsto

A+B

=

[a+b;a+b]

A�B

=

[a�b;a�b]

A�B=

[min(ab;ab;ab;ab);max(ab;ab;ab;ab))]

A=B

=

A�[1=b;1=b]:

(2.6)

Example2.4.

[0;1]+[1;2]=

[1;3],

[1;2]�[1;2]=

[�1;1],

[�3;2]�[1;2]=

[�6;4],

[1;2]=[3;4]=

[1=4;2=3].

Identifyingsingletonswithrealnumbersweseethattheoperationsare

extensionsoftheusualoperationsinR.Thereforewepermitourselves

towritee.g.3+[1;2]asshortfor[3;3]+[1;2].

9

2.IntervalArithmetic

Fromthede�nition(2.5)itfollowsimmediatelythat

A1�A2

and

B1�B2

)

A1�B1�A2�B2

(2.7)

Thispropertyofthefourintervaloperationsisimportant.Wesaythat

theoperationsareinclusionmonotonic.

2.4.

FloatingPointIntervalArithmetic

Whenperformingintervalarithmeticonacomputerwehavetorepresent

intervals,andwechoosetouse oatingpointnumbersfortheendpoints.

LetFdenotethesetof oatingpointnumbersandde�nethe oating

pointintervaloperationsby

(A�B)=

smallestintervalwithendpointsinF,

whichcontainsA�B:

(2.8)

Again,thisde�nitionisnotoperational.Itdependse.g.onthe oating

pointprocessorwhetheritispossibleto�ndthetwoendpoints,butthis

problemisoutsidethescopeofthisintroduction.

Thede�nition(2.8)ensuresthatalso oatingpointintervaloperations

areinclusionmonotonic.

Inthefollowingwedescribeanumberofmethodsusingrealintervals

andrealintervalarithmetic.Suchintervalcomputationsaredonewith

thehelpof oatingpointintervalarithmetic.Theresultingintervalsare

guaranteedby(2.8)tocontaintheintervalswewouldhavefoundbythe

equivalentrealoperations.Intheformulationofthemethodsweonly

needtouserealintervals,andthepracticalcomputationleadstoawider

interval.Thus,wehaveautomatedestimationofroundingerrors,but

{asindicatedbythefollowingexample{westillhavetobecautious

whenweformulatethealgorithms.

Example2.5.

FollowingRis(1972)itholdsthat

p332[5:744;5:745];

p292[5:385;5:386];

andcalculatingwithfoursigni�cantdigitsgives

p33+

p292[11:12;11:14];

p33�

p292[0:3580;0:3600]:

2.4.FloatingPointIntervalArithmetic

10

The�rstresultisquitegood,buttheotheristerrible.Itisthewellknown

numericalproblemaboutcancellationinconnectionwithsubtraction,and

asusualthecureistoreformulatethecalculation

p33�

p29=

4

p33+

p29

2

4

[11:12;11:14]2[0:3590;0:3598];

whichisamuchbetterresult.

Finally,considertherelationbetweenordinary oatingpointcalculation

and oatingpointintervalarithmetic.Ifwemakea oatingpointcalcu-

lation,howdoestheresultrelatetothecorrespondingcalculationwith

oatingpointintervalarithmetic?Toanswerthatquestion,remember

that

(a�b)=

oneofthetwonumbersinF;

whichareclosesttoa�b;

(2.9)

unlessa�b2F,inwhichcase (a�b)=

a�b.Combiningthisresult

with(2.8)weget

(a�b);a�b2 (A�B)

fora2A;b2B:

Therefore,theintervalobtainedby oatingpointintervalarithmeticcon-

tainsboththemathematicallytrueresultandtheresultwegetbyordinary

oatingpointcalculation.

Thisexplainstheresultintheaboveexample.Ifthe oatingpointcalcu-

lationisunstable,i.e.hasalargeroundingerror,thenthe oatingpoint

intervalarithmeticresultsinwideintervals.Theoppositeisunfortu-

natelynotthecase,andthisisprobablythereasonwhysomepeople

saythatintervalarithmeticistoopessimistic.

3.

IntervalAlgebra

InthischapterwelookattherulesthatarevalidforI(R)withthefour

associatedintervaloperations.

Itiseasytoprovethat+

and�arebothcommutativeandassociative,

ForA;B;C2I(R):

A+B

=

B+A;

(A+B)+C

=

A+(B+C);

A�B=

B�A;

(A�B)�C=

A�(B�C):

Moreover[0;0]=0and[1;1]=1areneutralwithrespecttoaddition

andmultiplication,respectively,

A+0=

0+A

=

A;

A�1=

1�A=

A:

Anondegenerateintervalhasnoinversewithrespectto

+

and

�

sinceeitherofthesewithanondegenerateoperandcannotresultinthe

neutralelement,whichisasingleton.

Thedistributiveruleisnotvalidingeneral.

Example3.1.

Asacounterexampleconsider

[1;2]�([1;2]�[1;2])=

[1;2]�[�1;1]=

[�2;2];

[1;2]�[1;2]�[1;2]�[1;2]=

[1;4]�[1;4]=

[�3;3]:

Insteadthesocalledsub-distributivityholds,

A(B+C)�AB+AC:

Theproofofthisisclearfromthede�nition(2.5).

Thelackofvalidityofthedistributiveruleoftencausestroublewhen

formulatingintervalcalculations.Itissounusualnotbeingallowedto

multiplyintoaparenthesis.

Thedistributiveruleisvalidforspecialintervals,e.g.

t(A+B)=

tA+tB

fort2R:

AverydetailedanalysisofotherspecialcasescanbefoundinRis(1972).

4.

IntervalFunctions

Intheintroductionwesawtheuseofintervalcalculationto�ndupper

andlowerboundsfortherangeoftheexponentialfunctionoveragiven

domain.Thisapplicationofintervalsandintervaloperationsisthesub-

jectofthischapterandmoretocome.Itisofinterestbyitselftobe

abletoestimateboundsoftherangeofafunction,buttheconceptspre-

sentedinthischapterwillproveusefullaterwhenweformulatemethods

forintegration,root-�ndingandoptimization.

4.1.

IntervalHulls

Considerthefunctionf:Rn

7!Rm

de�nedonadomainA�I(Rn),cf.

Section2.2.TheimageofAis

f(A)=

(f1(A);:::;fm(A))=

ff(x1;:::;xn)jxi2Aig:

Iffiscontinuous,thenthissetisclosedandcompact.Itisobviousthat

f(X)�f(A)forallX�A.Unfortunatelytheimageisnormallynot

anintervalvector.Thisisillustratedby

Example4.1.

Withf:R

2

7!R

2

de�nedbyy1

=

x1+x2,y2

=

x2 1,the

imagef([0;1];[0;1])istheshadedregionshownbelow.

1 01

2

Figure4.1.

Obviously,wecannotgivesucha�gureastheresultofacalculation.

Thereforeweenclosetheimageinanintervalvector,andusethisas

theresult.Thesmallestintervalvectorthatenclosesf(X)iscalledthe

intervalhulloffanddenotedf.Thus,

f(X)�f(X)

forX

�A:

(4.1)

13

4.IntervalFunctions

Inthespecialcaseofarealfunction,i.e.m=1,theimageisaninterval

inR,andinthiscase(4.1)holdswithequality.

Iffisoneofthefourarithmeticoperations,f(a;b)=

a�bthenthe

intervalhulloffisthecorrespondingintervaloperation,f(A;B)=

A�B.

4.2.

IntervalExtensions

AmappingF

:I(Rn)7!I(Rm)iscalledanintervalfunction.The

intervalhullofafunctionisanexampleofanintervalfunction.

IfwehaveafunctionfasaboveandanintervalfunctionFanditholds

that

f(X1;:::;Xn)�F(X1;:::;Xn)

forXi�Ai;

(4.2)

thenFissaidtobeanintervalextensionoff.Inparticular,theinterval

hullisanintervalextension.

Example4.2.

Wegivethreeexamplesofintervalextensions

f(x)=sinx;F(X)=[�1;1]

f(x)=x(1�x);F(X)=X�(1�X)

f(x1;x2)=(x1+x2;x

2 1);F(X1;X2)=(X1+X2;X1�X1)

The

last

function

was

also

treated

in

Example

4.1.

We

�nd

F([0;1];[0;1])

=

([0;2];[0;1]).

This

is

the

smallest

intervalthat

enclosestheimage,cf.Figure4.1,so

inthiscaseF(X)=f(X).

Figure4.2illustratesthesecond

ex-

ampleon

theintervalX

=

[0;

1 2].

Wegivethegraph

ofthefunction

f(x)=x(1�x)and

thesmallestbox

thatcontainsallpoints(x;y)forwhich

x2X

andy2F(X)=

[0;

1 2].

The

intervalhullisf(X)=[0;1 4

].

0.5 0

0.5

Figure4.2.

4.3.ExtensionsforRationalFunctions

14

4.3.

IntervalExtensionsforRationalFunctions

Forrationalfunctionsthereisasimple,generalmethodforobtainingan

intervalextension:Firstwritearationalexpressionde�ningtherational

function.Next,simplyreplacetherealoperationsbyintervaloperations

andthevariablesbyintervals.

Example4.3.

Wegivetwoexamples,

f(x)=

1�x

2

3+x2

;F(x)=(1�X�X)=(3+X�X)

f(x1;x2;x3)=x1�x2x3

;F(X1;X2;X3)=X1�X2�X3

Thereasonwhywecanusethissimplemethodtogetanintervalexten-

sionliesinthede�nition(2.5)ofintervaloperations.Wecanalsouse

theinclusionmonotonicitytoshowthatFisanintervalextension:Let

x=(x1;:::;xn)2(X1;:::;Xn),then

F([x1;x1];:::;[xn;xn])�F(X1;:::;Xn);

andsincef(x1;:::;xn)=

F([x1;x1];:::;[xn;xn]),itfollowsthat

f(x1;:::;xn)2F(X1;:::;Xn)andtherefore

f(X1;:::;Xn)�F(X1;:::;Xn):

Byuseofthealgebraicrulesforrealoperatorsarationalexpressioncan

beformulatedinmanyways.Note,however,thatwecannotmakethe

samereformulationsofintervalexpressions,e.g.becauseoftheinvalidity

ofthedistributiverule.Thisimpliesthatdi�erentformulationsofa

rationalfunctionwillleadtodi�erentintervalextensions.

Example4.4.

Consider

f(x)=

1�x+x

2�x

3+x

4�x

5

=

(1�x

6)=(1+x)

=

(1�x)(1+x

2+x

4)

15

4.IntervalFunctions

ontheintervalA=[2;3].Itiseasytoshowthatf0(x)<0forallx,and

therefore f

(A)=

[f(3);f(2)]=

[�182;�21]:

Thethreeformulationsoffabovegiverespectively

F1(A)=[�252;49],

F2(A)=

[�242

2 3;�15

3 4]andF3(A)

=

[�182;�21],i.e.,verydi�er-

entintervals.Laterweshallseethatamongthemanypossibleinterval

extensionssomestandoutasparticularlygood.

Withaneyeontheexampleonecouldaskwhetheritisalwayspossibleto

reformulatetherationalfunctionsothattheintervalextensionisequal

totheintervalhull.Alas,thisisnotthecase.Asacounterexample

considerarationalfunctionofonevariablede�nedonanintervalwith

rationalendpoints,andwithirrationalupperand/orlowerlimitofthe

image,e.g.x3�xon[0;1].Thenitisnotpossibleto�ndtheinterval

hullbya�nitenumberofrealoperationsontheendpoints.Thisshows

thatifwewishtocomputetheintervalhulltoarbitraryaccuracy,then

weneedmoretoolsthanjustthefourintervaloperations.Suchtoolsare

describedlater.

Inallimplementationsofintervalarithmeticyouhaveaccesstointerval

extensionsofextendedrationalfunctions.Thisisthesetoffunctions,

wherewecancomputetheexactintervalhull,andconsistoffunctions,

wherethesimplearithmeticoperatorsarecombinedwithstandardfunc-

tionslikesin(x),ex,etc.

5.

TheSetofIntervalsasMetricSpaces

Noteveryintervalextensionisequallyattractive.Insomesituations

wewishthattheextensionis\close"totheintervalhull,buttobe

meaningful,wemustbeabletomeasurethe\distance"betweentwo

intervals.Inotherwords,wehavetoorganizethesetofrealintervals

andthesetofrealintervalvectorsasmetricspaces.

First,wehavetode�neadistancefunction.ForA;B2I(R)wede�ne

thedistancebetweenthetwointervalsas

d(A;B)=

maxfja�bj;ja�bjg:

(5.1)

ItiseasytoshowthatdisametricinI(R),i.e.,itsatis�es

d(A;B)=

0

ifandonlyifA=B

d(A;B)�0

d(A;B)=

d(B;A)

symmetry

d(A;B)�d(A;C)+d(C;B)

insertionrule

(5.2)

Thede�nitionofddoesnotindicatehowfaranelementinAcanbe

from

anelementinB,asmeasuredbytheusualdistanceinR.This

isofinterest,e.g.whenAisacomputedintervalandB

isthedesired

interval,andweknowthatd(A;B)�",where">0.Inthatcase,how

farisanindividualelementinAfromtheelementsinB?

Inorderenableananswertosuchquestionswereformulatethede�nition

(5.1)to

Lemma5.1.LetA;B2I(R).Then

d(A;B)=

max

� max

a2A

fminb2B

ja�bjg;max

b2B

fmina2A

ja�bjg

Proof.

The�rstminimumcanbesimpli�edasfollows

minb2B

ja�bj=

8 < :0

if

a2B

ja�bjif

b<a

forallb2B

ja�bjif

b>a

forallb2B

17

5.MetricSpaces

andthemaximumwithrespecttoais

max

a2A

fminb2B

ja�bjg=

8 > > < > > :

0

if

A�B

j a�bj

if

A�B

ja�bj

if

A�B

maxfja�bj;ja�bjgif

A�B

ThenotationA�B

(respectivelyA�B)meansthatthereexistsno

a2Asothata<b(respectivelya>b)anda2jB.

Similarlyweget

mina2A

ja�bj=

8 < :0

if

b2A

ja�bjif

a<bforalla2A

ja�bjif

a>bforalla2A

and

max

b2B

fmina2A

ja�bjg=

8 > > < > > :

0

if

B�A

ja�bj

if

B�A

ja�bj

if

B�A

maxfja�bj;ja�bjgif

B�A

Finally,wecombineallthepossibilities,takingmaximumandignoring

thecasesthatexcludeeachother.Thisleadstomaxfja�bj;ja�bjg,

whichisthede�nition(5.1)ofthedistancebetweenAandB.

Wecanusethislemmatoansweroneofthequestionsweraisedatthe

beginningofthischapter:

Lemma5.2.LetA;B2I(R)and">0.Thenthestatement

1Æ

d(A;B)�"

isequivalentwith

2Æ

(�)

foralla2Athereexistsb2Bsuchthatja�bj�"

(�)

forallb2Bthereexistsa2Asuchthatja�bj�"

5.MetricSpaces

18

Proof.

WeneedLemma5.1,and�rstlookat1Æ

)2Æ.Sinced(A;B)�

",thenminb2

B

ja�bj�"foralla2Aandmina2

A

ja�bj�"forallb2B.Thus,

1Æ

implies(�)and(�).

Next,toshowthat2Æ

)1Æ,assumption(�)impliesthatfor�xedait

holdsthatminb2

B

ja�bj�",andthisinequalityisalsotruewhenwemax-

imizeovera2A.Similarly,weuseassumption(�)toensurethatthe

secondnumberinthemaximumexpressioninLemma5.1isboundedby

",andtherefored(A;B)�".

ForintervalvectorsA=(A1;:::;An)andB=(B1;:::;Bn)wede�ne

thedistanceas

d(A;B)=

maxi

d(Ai;Bi):

(5.3)

Thus,alsoI(Rn)isorganizedasametricspace.Itissimpletoextend

Lemma5.2tointervalvectors:

Lemma5.3.LetA;B2I(Rn)and">0.Thenthestatement

1Æ

d(A;B)�"

isequivalentwith

2Æ

(�)

foralla2Athereexistsb2Bsuchthatka�bk 1�"

(�)

forallb2Bthereexistsa2Asuchthatka�bk 1�"

Proof.

WeuseLemma5.2andde�nition(5.3).Theassumption

d(A;B)�"impliesthatd(Ai;Bi)�"foralli,andthus,forallai2Ai

thereexistb i2Bisuchthatja i�bij�".Therefore,foralla2Athere

existsb2B

suchthatka�bk 1�".Implication(�)isprovedthesame

way,and2Æ

)1Æ

isequallyeasy.

19

5.MetricSpaces

5.1.

ContinuityofIntervalFunctions

Lemmas5.2and5.3quantifyintermsoftheintervalelements,whatis

meantbysayingthattwointervalsareclose.Now,wewillusethemetric

conceptstode�nelimitsandcontinuityinI(Rn).

A

sequenceofintervalsfX(s)ginI(Rn)issaidtoconvergetoX

if

d(X(s);X)!0fors!1.Fromthede�nition(5.3)itfollowsthatthis

meanscomponentwiseconvergence,i.e.d(X(s)i;Xi)!0,andfromthe

de�nition(5.1)ofthescalardistancethismeansthatx(s)i!xiand

x(s)i!xi,i.e,theendpointsconvergetotheendpointsofthelimit

interval.

AnintervalfunctionF

:I(Rn)7!I(Rm)iscontinuousforX�A

2I(Rn)if li

ms!

1

F(X(s))=

F(X)

for

lims!

1

X(s)

=

X

:

(5.4)

ConstantintervalfunctionsandmappingsofI(Rn)intoI(Rm),where

eachcoordinateinX2I(Rn)iseithereliminatedorduplicatedarecon-

tinuous.Sincecompositemappingsingeneralmetricspacesarecontinu-

ous,weonlyneedtoshowthatthefourintervaloperationsarecontinuous

inordertoverifythatrationalintervalfunctionsarecontinuous.This

isdonewhenwehaveprovedthefollowinggeneraltheorem

aboutthe

intervalhullforacontinuousfunction,

Theorem

5.1.IffisacontinuousmappingofRn

intoRm,de�ned

onanintervalA�Rn,thentheintervalhullfisacontinuousinterval

functionfromI(Rn)intoI(Rm).

Proof.

Wehavetoshowthatthecontinuitycondition

kf(u)�f(v)k1

<"

foru;v2Aandku�vk 1<Æ

impliesthatd(f(U);f(V))<"forU;V�Aandd(U;V)<Æ.

First,weconsidertheimagesf(U)andf(V).Theyarecompact,and

themappingsoftheseundertheprojectionofRnontotheithcoordinate

isalsocompact,whichmeansthatYi=(f(U))iandZi=(f(V))iare

5.2.LipschitzContinuity

20

intervalsinR.Theseintervalsarewhatwegetbytakingtheithcoor-

dinateoftheintervalhull,Yi=(f(U))i,Zi=(f(V))i.Therefore,for

yi2Yithereexistsu2Usuchthatyi=f(u) i.Sinced(U;V)<Æ,Lemma

5.3guaranteestheexistenceofv2V

withku�vk 1<Æandtherefore

kf(u)�f(v)k1

<".Thenumberz iisanelementinZi,andwehave

shownthatcorrespondingtoyithereexistsanelementz iinZisuchthat

jy i�zij<".

Startingwithz iandproceedingthesamewaywegetthesameinequality,

andLemma5.2showsthatd(Yi;Zi)<"whend(U;V)<Æ.Thisholds

foreveryi,andwecandeducethatmaxid(Yi;Zi)=d(f(U);f(V))<".

ThefourordinaryoperationsarecontinuousmappingsofR

2

intoR,

andthereforethetheoremimpliesthatthefourintervaloperationsare

continuous.Further,thisisseentoimplythecontinuityofanyinterval

functionobtainedbythemethodofSection4.3.Inparticular,ifX(s)!

xwithx2Rn,thend(F(X(s));F(x))!0,andsincef(x)=F(x)for

theseintervalextensions,itfollowsthat

d(F(X(s));f(x))!0

for

s!1:

(5.5)

Inwords:ifX(s)isasmallargumentintervalcontainingx,thenF(X(s))

isclosetothefunctionvaluef(x).

The�rstfunctioninExample4.2showsthatthispropertydoesnothold

forallintervalextensions,eveniftheyarecontinuous.Theextension

ofsin(x)toF(X)=[�1;1]isevidentlycontinuousanditdoesnot

satisfy(5.5).

Inthenextchapterweshallseethatwithcertaintypesofintervalex-

tensionsitispossibletoquantifythespeedofconvergencein(5.5).This

isbasedonanestimateofd(F(X);f(X)).

5.2.

LipschitzContinuity

Nowwelookatapresumptiononintervalfunctionswhichisstronger

thancontinuity,viz.Lipschitzcontinuity.

Anordinaryfunctionf

:

Rn

7!Rm

de�nedonanintervalAissaidtobeLipschitzcontinuousif

21

5.MetricSpaces

thereexistsanumberK>0suchthat

d(f(x);f(y))�Kd(x;y)

for

x;y2A:

(5.6)

Thisde�nitionistransferredliterallytointervalfunctions:Aninterval

functionFfromA2I(Rn)intoI(Rm)issaidtobeLipschitzcontinuous

ifthereexistsanumberK>0suchthat

d(F(X);F(Y))�Kd(X;Y)

for

X;Y�A:

(5.7)

Regardingtheintervalhullwehave

Theorem

5.2.

Ifthefunctionf:A�Rn

7!Rm

isLipschitzcon-

tinuous,thenthesameistruefortheintervalhullf:A2I(Rn)7!

I(Rm).

Proof.

Wehavetoconsiderd(f(X);f(Y))forX;Y2A,andaswe

didbefore,welookattheithcoordinate.UsingLemma5.1weget

d(f(X) i;f(Y) i)=

max

� max

x2

Xfminy2

Y

jf(x) i�f(y) ijg;

maxy2

Yfminx2

X

jf(x) i�f(y) ijg:

Hereweusedthatforeveryelement�2f(X)thereisanx2Xsuchthat

�i=f(x) i.Next,fbeingLipschitzcontinuousimpliesthat

jf(x) i�f(y) ij�Kjxi�yij.ThefactorK

goesoutsideallthemaxesand

min

s,andanotherapplicationofLemma5.1leadsto

d(f(X) i;f(Y) i)�Kd(Xi;Yi)�Kd(X;Y):

Thisistrueforalli=1;:::;m

andthetheoremfollows.

Exceptingdivisionbyzero,thefourarithmeticoperationsareLipschitz

continuousfunctions.Therefore,thecorrespondingintervaloperations

areLipschitzcontinuous,andthroughthesamestepsthatwereusedto

showthatrationalintervalfunctionsarecontinuous,wecanshowthat

theyarealsoLipschitzcontinuous.Allweneedis

5.3.Width

22

Theorem

5.3.

IfFandGareLipschitzcontinuousintervalfunc-

tions,thenthecompositefunctionFÆGisLipschitzcontinuous.

Proof.

d(F(G(X));F(G(Y)))�K1d(G(X);G(Y))�

K1K2d(X;Y)withK1;K2>0,andthetheoremisproven.

Inconclusion,rationalintervalfunctionsareLipschitzcontinuous.In

particular,thisistruefortheintervalextensionsfromSection4.3.

5.3.

W

idthofanInterval

The�nalbasicmetricconcepttoconsideristhewidthofaninterval

(vector).Thiswasde�nedinChapter2,

w(A)=

(a�a

ifA2I(R);

maxifw(Ai)gifA2I(Rn):

ForA�B

thereisthefollowingimportantrelationbetweenthewidth

andthedistancede�nedinLemma5.1,

1 2(w(B)�w(A))�d(A;B)�w(B)�w(A):

(5.8)

Further,forA;B;C;D2I(R),a2Rwehavethefollowingrelationsfor

thewidth(thenormkAkisde�nedin(2.3))

w(A+B)=

w(A)+w(B);

w(A�B)=

w(A)+w(B);

w(aA)=

kakw(A);

maxfkBkw(A);kAkw(B)g�w(A�B)

�kBkw(A)+kAkw(B);

w(A=B)�

1 bb

� kBkw(A)+kAkw(B)� ;

(5.9)

andforthedistance

23

5.MetricSpaces

d(A+C;B+C)=

d(A;B);

d(A+B;C+D)�d(A;C)+d(B;D);

d(A�B;C�D)�d(A;C)+d(B;D);

d(A�B;C�D)�kAkd(B;C)+kCkd(A;D);

d(1=A;1=B)�

1

kAkkBk

d(A;B):

(5.10)

Weomittheproofoftheserelations.Itispurelytechnicalandrather

tedious.

Havingintroducedthewidthwecanproveanimportantpropertyofthe

intervalhullofaLipschitzcontinuousfunction,

Theorem

5.4.LetfbetheintervalhullofaLipschitzcontinuous

functionontheintervalA.Then,forX�AthereexistsK>0such

that

w(f(X))�Kw(X):

Proof.

Thereexistu;v2X

sothat

w(f(X))=

maxiw(f(X) i)

=

maxijf(u) i�f(v) ij

�K

maxijui�vij�Kw(X):

Againweexploitedthatallelementsintheithprojectionofthehullare

mappingsofelementsinX,andinparticularthisistruefortheend

pointsoff(X).

NotallLipschitzcontinuousintervalfunctionssatisfytheinequalityin

Theorem5.4.Thisimpliesthatimagesofsingletonsaresingletons,and

F(X)=[�1;1]isasimplecounterexample.

6.

LinearandQuadraticInterval

Extensions

Inthischapterwelookatdi�erenttypesofintervalextensions,and

assesstheirquality.Westartwithanexampletointroducetheconcepts

thatweshallworkwith.

Example6.1.

Therationalfunctionf(x)=x(1�x)hasanintervalexten-

sionF(X)=X(1�X).Weconcentrateontheintervals

X

=

[1 4

�r;1 4

+r]

for

0�r�1 4

;

andfrom

(5.5)weknowthatF(X)!f(1 4

)=

3 16

forr!0.Howfastis

theconvergence?To�ndtheanswer,wemakethefollowingcalculation,

F([1 4

�r;1 4

+r])=

[1 4

�r;1 4

+r]�(1�[1 4

�r;1 4

+r])

=

[3 16

+r

2�r;

3 16

+r

2�r]

=

3 16

+r

2+[�r;r]

ThefunctionfisincreasingonX,soitiseasyto�nditsimage,

f(X)=

[f(1 4

�r);f(1 4

+r)]=

3 16

�r

2+

1 2[�r;r];

andthedistancebetweenF(X)andf(X)is

d(F(X);f(X))=

1 2r+2r

2

=

O(r):

Weshallseethatthisisgenerallyvalid:Intervalextensionsobtainedsim-

plybyreplacingrealargumentsbytheirintervalshave\errors"thatgoto

zerolinearlywiththewidthoftheintervals.

Bymeansofthemeanvaluetheorem

wecangetabetterapproximation

tof.Forx2X

thetheorem

saysthat

f(x)=

f(1 4

)+(x�1 4

)f0(�);

forsome�betweenxand1 4

.Sincef0(�)=1�2�,weseethat

f(x)2FM

(X)�f(1 4

)+(X�1 4

)(1�2X);

i.e.,FM

isanintervalextensionoffontheintervalX.Asregardsthe

di�erencebetweenFM

(X)andf(X),we�nd

FM

([1 4

�r;1 4

+r])=

3 16

+[�r;r]�[

1 2

�2r;

1 2

+2r]

=

3 16

+1 2[�r;r]+2[�r

2;r

2];

25

6.IntervalExtensions

andd(FM

(X);f(X))=3r

2,whichisbetterthand(F(X);f(X))forsmall

valuesofr.WeshallseethatingeneralF(X)hasan\error"O(w(X)),

whilethe\error"withFM

(X)isO(w(X)2).

6.1.

LinearIntervalExtensions

IfF

isanintervalextensionoffonanintervalA�I(Rn)andthere

existsaK>0,independentofX,sothat

d(F(X);f(X))�Kw(X)

forallX

�A;

(6.1)

thenwesaythatF

isalinearintervalextension.Itisobviousthat

alinearintervalextensionsatis�estheconvergencecriterion(5.5)for

w(X)!0.

The�rst,simpleresultaboutlinearintervalextensionsis

Theorem

6.1.

IfF

isalinearintervalextensionofaLipschitz

continuousfunctionf,thenthereexistsaconstantK>0suchthat

w(F(X))�Kw(X):

Proof.

Wemakeuseoff(X)�F(X),Equations(5.8)and(6.1),and

Theorem5.4,

w(F(X))�2d(F(X);f(X))+w(f(X))

�2K1w(X)+K2w(X)=

Kw(X):

Next,welookatcompositefunctions,

Theorem

6.2.IfFandGarelinearintervalextensionsofLipschitz

continuousmappingsf:A�Rn7!Rm

andg:F(A)�Rm

7!Rp,re-

spectively,thenFÆGisalinearintervalextensionoffÆg.

Proof.

LetX�Aandusetheinsertionruletoget

6.1.LinearExtensions

26

d(G(F(X));fÆg(X))�

d(G(F(X));g(F(X)))+d(g(F(X));g(f(X)))

+d(g(f(X));g(f(x)))+d(g(f(x));fÆg(X));

wherexisanarbitrarypointinX.Wenowestimateeachofthefour

terms,

d(G(F(X));g(F(X)))�K1w(F(X))�K1K2w(X):

HereweusedthatGisalinearintervalextensionandthatFsatis�esthe

assumptionsofTheorem6.1.InthesecondtermweusetheLipschitz

continuityofgandthelinearityofF,

d(g(F(X));g(f(X)))�K3d(F(X);f(X))�K3K4w(X):

Inthethirdtermwemakeuseofg(f(x))2g(f(x)),w(g(f(x)))=0,the

relation(5.8)betweenwandd,andTheorem5.4,

d(g(f(X));g(f(x)))�w(g(f(X)))

�K5w(f(X))�K5K6w(X):

Finally,fÆgisLipschitzcontinuousandweapplythesametechniqueto

thelastterm,

d(g(f(x));fÆg(X))�w(fÆg(X))�K7w(X):

CombiningtheseestimatesweseethatFÆGsatis�es(6.1),andtheproof

is�nished.

Constantsintheformofsingletonsandthefourintervaloperationsare

linearintervalextensions.Inbothcasesthisisbecauseweworkwith

intervalhulls.Alsomappingsthatduplicatevariablesoreliminatethem

arelinearintervalextensions.Thisimpliesthattherationalinterval

functionstreatedinSection4.3arelinearintervalextensionsofextended

rationalfunctions.

Insomecasesthede�nition(6.1)ofalinearintervalextensionmaybe

reformulatedto:

27

6.IntervalExtensions

ThereexistsE2I(Rn)with02Eandw(E)�Kw(X)suchthat

F(X)=

f(X)+E:

(6.2)

Inwords,everypointinF(X)isthesumoftwovectors,oneofwhichis

intheintervalhullandtheotherhaslimitedlength.

6.2.

MeanValueForms

Untilnowwehaveseenonegeneralmethodforconstructingintervalex-

tensions,viz.intervalextensionsforrationalfunctions.Now,weshallsee

howthemeanvaluetheoremcanbeusedtoconstructintervalextensions

foramuchwiderclassoffunctions,thedi�erentiablefunctions.

Considerf:Rn

7!R,whichisdi�erentiableinA�Rn.Forsucha

functionthemeanvaluetheoremsays

f(x)=

f(^x)+

n X i=1

@f

@xi

(�)(xi�^xi);

with^x;x2Aand�isapointonthelinesegmentbetween^xandx.Let

F0

beanintervalextensionofthegradient

f0

=

� @f

@x1

;:::;

@f

@xn

� ;

andletX�Abeanintervalthatcontainsboth^xandx.Then

f(x)2f(^x)+F0(^x)�(X�^x);

wherethemultiplicationinthelastterm

istheinnerproductoftwo

intervalvectors.Inthisreformulationwegotridof�,andtheexpression

onlyinvolvesintervalsandintervaloperations.Inorderforthistobe

anintervalextensionoffwemustaccompanytheintervalX

witha

speci�cationofthechoiceof^x.

Ifwechoose^x=m(X),themidpointofX,wesaythattheresulting

intervalfunction

FM

(X)=

f(m(X))+F0(X)�(X�m(X))

(6.3)

isameanvalueform.

6.2.MeanValueForms

28

Example6.2.

ForX

=[�r;r]withr>0ameanvalueformofthefunction

cosis F

M

(X)=

cos(0)+[�1;1]�(X�0)=

1+[�1;1]�X;

sincecos0(�)2[�1;1],andthisintervalconstantisthereforeaninterval

extensionofcos0.FM

isclearlyalinearintervalextension.

Themeanvalueformisnotconstructedsolelybythefourintervalopera-

tions,butinvolvesgettingthemidpointetc.Therefore,itisnotobvious

thatFM

isinclusionmonotonic,butthenexttheoremguaranteesthat

FM

hasthispropertyprovidedthatF0

does.

Theorem

6.3.IfF0

isaninclusionmonotonicintervalextensionof

f0

thenFM

isaninclusionmonotonicintervalextensionoff.

Proof.

LetX;Y�I(Rn)withX�Y.WehavetoshowthatFM

(X)�

FM

(Y).Toeasethenotationweintroducex=m(X)andy=m(Y),and

startbyusingthemeanvaluetheoremonfandtheinclusionmonotonic-

ityofF0, F

M

(X)=

f(x)+F0(X)(X�x)

=

f(y)+f0(�)(x�y)+F0(X)(X�x)

�f(y)+f0(�)(x�y)+F0(Y)(X�x)

with�2Y.Werewritethecurrentresultto

FM

(X)�f(y)+

n X i=1

� f0 i(�)(xi�yi)+F0 i(Y)(Xi�xi)� ;

andlookattheithcontributiontothesum,usingthatf0 i(�)2F0 i(Y),

xi=m(Xi),andyi=m(Yi):

f0 i(�)(xi�yi)+F0 i(Y)(Xi�xi)

=

f0 i(�)(xi�yi)+jF0 i(Y)jw(Xi)[�1 2;1 2]:

Itiseasilyseenthat

jf0 i(�)(xi�yi)j�jF0 i(Y)j(w(Yi)�w(Xi))=2;

29

6.IntervalExtensions

andtherefore

f0 i(�)(xi�yi)+jF0 i(Y)jw(Xi)[�1 2;1 2]

�jF0 i(Y)j(w(Yi)�w(Xi)+w(Xi))[�1 2;1 2]

=

jF0 i(Y)jw(Yi)[�1 2;1 2]=

F0 i(Y)(Yi�yi):

Nowthetheoremfollowsimmediately.

Ifweuseagoodintervalextensionforf0

wecangetbetterresultsthan

illustratedinExample6.2.Thisisthesubjectofthenextsection,but

beforethatwegivetwoexamples.

Example6.3.

Asinthepreviousexampleweconsiderf(x)=

cos(x)on

X

=

[�r;r];r>0.Thederivativeiscos0=

�sin,andwecangetan

intervalextensionforcos0byusingameanvalueform

for�sin,i.e.

�sin(x)=

�sin(0)�(x�0)sin0(�)

=

�xcos(x)2X�[�1;1]:

Therefore,cos(x)21+X�X�[�1;1]:

ComparingthiswiththeTaylorexpansionofcos(x)around^x=0,cos(x)=

1�

1 2x

2+O(x

4);weseethatwehaveobtainedaquadraticapproximation

totheintervalhullofcosontheintervalX.

Example6.4.

AsinExample4.2weconsiderf(x)=x(1�x)ontheinterval

X

=[0;

1 2].Sincef0(x)=1�2x,wecanuseF0(X)=1�2X

=[0;1],

andwithm(X)=

1 4

,f(1 4

)=

3 16

weget

FM

(X)=

3 16

+[0;1][0;

1 2]=

[�

116

;7 16]:

Actually,F0(X)=f0(X),andtheideafrom

thepreviousexamplewould

giveusthesameintervalextensionoff0.InExample4.2wefoundF(X)=

[0;

1 2],andFM

(X)isseentobeabetterapproximationtotheinterval

hullf(X)=[0;1 4

].

Obviously,wecanextendtheresultstofunctionsf:Rn7!Rm

byusing

themeanvaluetheoremseparatelyoneachcoordinatefunction.

6.3.QuadraticExtensions

30

Finally,itshouldbementionedthatwhenweusethemeanvalueform

inpracticeonacomputer,thentherealvectorf(m(X))shouldberep-

resentedbyanintervalvectorwithboundsontheroundingerrorscon-

nectedwiththeevaluationoffatthepointm(X),i.e.f(m(X))should

berepresentedbyF([m(X);m(X)]).

6.3.

QuadraticIntervalExtensions

IfF

isanintervalextensionoffonanintervalA�I(Rn)andthere

existsaK>0,independentofX,sothat

d(F(X);f(X))�Kw(X)2

forallX

�A;

(6.4)

thenwesaythatF

isaquadraticintervalextension.Thisissimilar

tothede�nitioninSection6.1.Theadvantageofaquadraticinterval

extensioniswhenw(X)�1,inwhichcaseFisalmostindistinguishable

fromtheintervalhull.

ForquadraticintervalextensionsofLipschitzcontinuousfunctionswe

haveatheoremsimilartoTheorem6.1,

Theorem

6.4.IfFisaquadraticintervalextensionofaLipschitz

continuousfunctionf,thenthereexistsaconstantK>0suchthat

w(F(X))�Kw(X):

Proof.

SimilartotheproofofTheorem6.1.

Themostimportantexampleofaquadraticintervalextensionisviathe

meanvalueformwithF0

chosenasalinearintervalextension.

Theorem

6.5.Letf:A2I(Rn)7!Rbedi�erentiableandassume

thatthederivativesareLipschitzcontinuous.Further,letF0

bea

linearintervalextensionoff0.Then

FM

(X)=

f(m(X))+F0(X)(X�m(X))

isaquadraticintervalextensionoff.

31

6.IntervalExtensions

Proof.

Weapply(6.2)totheintervalextensionF0,

F0(X)=

f0(X)+E;

whereE�I(Rn)withw(E)�K1w(X)and02E,implyingthatjEj�

w(E)�K1w(X).Nowwecanwrite

FM

(X)=

f(m(X))+( f

0(X)+E)(X�m(X));

showingthateveryelementy2FM

(X)hastheform

y=

f(m(X))+(f0(u)+e)(v�m(X))

withu;v2X

ande2E.Acorrespondingpointintheintervalhullf(X)

isy=f(v),andfromthemeanvaluetheoremweknowthatthereexists

a�2X

suchthat

y=

f(m(X))+f0(�)(v�m(X)):

Weproceedasfollows

d(FM

(X);f(X))�jy�yj

=

j� f0(u)�f0(�)+e�� v�m(X)� j

�n X i=

1� j(f0(u) i�f0(�) ij+je ij� jv i�m(X)j

�n X i=

1� K 2w(X)+K1w(X)� 1 2w(X)

=

n 2(K1+K2)w(X)2:

InthethirdreformulationweusedtheLipschitzcontinuityoff0

andthe

boundonE.Thus,(6.4)issatis�ed,andtheproofis�nished.

7.

AccurateComputationof

IntervalHulls

Inthepreviouschapterswehaveseensomegenerallyapplicableinterval

extensionsthatcanbeusedto�ndapproximationstotheintervalhull.

Undercertainconditionsweareguaranteedthattheerror

d(F(X);f(X))�K�w(X)�;

(7.1)

where�=1fortherationalintervalextensionsofSection4.3andlinear

intervalextensionsdiscussedinSection6.1,while�=2forthequadratic

intervalextensionsofSection6.3.Itispossibletoconstructinterval

extensionswithlarger�-values,butitreallydoesnothelp:wecannot

guaranteeanarbitrarilygoodapproximationifw(X)�1.

Ontheotherhand,ifw(X)�1,then(7.1)showsthatF(X)isvery

closetof(X),andthissuggestsanidea:DividetheintervalX

into

subintervals,eachofwhichhassmallerwidththanX,computeanin-

tervalextensiononeachsubinterval,andgathertheinformationabout

F(X)fromallsubintervals.Letustrytheideaonasimpleproblem.

Example7.1.

AsinExample4.2welookatthefunctionf(x)=x(1�x)

ontheintervalX

=[0;

1 2],withtheintervalextensionF(X)=X(1�X).

InFigure7.1weshowtheresultobtainedwhenwesplittheintervalinto

twoandfourequalparts.ComparingwithFigure4.2weseethatF(X)is

reducedfrom

[0;

1 2]to[0;

3 8]and[0;

5 16],respectively,whicharecloser

totherangeoff(X).

0.250.5 0

0.25

0.5

0.250.5 0

0.25

0.5

Figure7.1.Splittingtheinterval

33

7.AccurateIntervalHulls

Wereturntothisproblem

later.First,however,weformalizetheap-

proach.

Theorem

7.1.Letthefunctionfbede�nedinX�I(Rn),andlet

X(1);:::;X(p)beasubdivisionofX,i.e.,

X(j)�X

and

p [ j=1

X(j)

=

X

:

Further,letFbeanintervalextensionoffonX.Then

f(X)�

p [ j=1

F(X(j)):

Proof.

Ineachsubintervalitclearlyholdsthatf(X(j))�F(X(j)),and

thetheoremfollowsbytakingtheunionofbothsidesoftheinclusion.

Inotherwords,givenanintervalextensionoffandasubdivisionofthe

domainX,wegetanewintervalfunction

Fp(X)=

p [ j=1

F(X(j)):

(7.2)

Thisisanintervalextensionoff,providedthatwede�netheunion

oftwointervalsasthesmallestintervalthatcontainstheunion.This

intervalextensionisthesubjectof

Theorem

7.2.

LetF

beanintervalextensionofA�I(Rn)and

assumethatthereexistsaconstantK>0suchthat

d(F(X);f(X))�Kw(X)�

forallX

�A

with�=1or�=2.Then

d(F(X);f(X))�K

maxj

� w(X(j))�

:

7.AccurateIntervalHulls

34

Proof.

Wecan�ndtwosubintervalswithindiceslandusothat

Fp(X)=

p [ j=1

F(X(j))=

F(X(l))[F(X(u));

andbyusingthede�nitionofthedistanceinLemma5.1andthecon-

structionofFpwe�nd

d(F(X);f(X))�max

� d(F(X(l));f(X(l)));d(F(X(u));f(X(u)))

�K

max

� w(X(l))�;w(X(u))�

�K

max

p j=1

� w(X(j))�

:

Asaspecialcaseconsiderequidistantsubdivisionineachcoordinate

direction,i.e.w(X(j))=

1 qw(X),wherep,q,andthedimensionalityn

arerelatedthroughp=q�n.Thenthelasttheoremtellsusthat

d(F(X);f(X))�Kw(X)�(n=p)�:

(7.3)

Theright-handsideconvergestozero,i.e.Fp(X)!f(X)forp!1.

Thismeansthatnowwehaveamethodforcomputingtheintervalhull

toarbitraryaccuracy.Inpractice,ofcourse,whencarriedoutin oating

pointonacomputer,roundingerrorslimittheobtainableaccuracy.We

refertoSection2.4,andignorethislimitationinwhatfollows.

Example7.2.

Asinthepreviousexamplewelookat

f(x)=

x(1�x)

on

X

=

[0;

1 2]

with

f(X)=

[0;1 4

]:

Weconsiderthetwointervalextensions

F(X)=

X(1�X);

FM

(X)=

f(m(X))+(1�2X)(X�m(X));

anduseequidistantsubdivision,

X(j)

=

[j�1

p

;j p];

j=1;:::;p:

WeillustratedthechoiceF(X)inFigures4.4and7.1forp=1andp=2.

Below,wegiveresultsobtainedbysuccessivedoublingofp.

35

7.AccurateIntervalHulls

p

d(Fp(X);f(X))

d(FM

p(X);f(X))

1

0:250000

0:187500

2

0:125000

0:046875

4

0:062500

0:011719

8

0:031250

0:002930

16

0:015625

0:000732

32

0:007813

0:000183

64

0:003906

0:000046

128

0:001953

0:000011

. . .

. . .

. . .

Asexpected,Fp

showslinearconvergenceandFM

p

showsquadraticcon-

vergence:Eachdoublingofpreducestheerrorwithafactor2and4,

respectively.

TheproofofTheorem

7.2suggestsawayofreducingthenumberof

timesneededtorecomputetheintervalextensiononsubintervals.The

errorsatis�es

d(F(X);f(X))�K

max

� w(X(l))�;w(X(u))�

;

andtoreducethedistancebetweenFp(X)andf(X)itmaynotbenec-

essarytore�neallsubintervals,butonlyX(l)andX(u),thesubintervals

thatcurrentlyde�netheendpointsofFp(X).Thisideahasbeenused

bySkelboe(1974)todevelopanadaptiveversion,wherethesubdivision

ofX

ismonitoredbyF.

WewanttostoptheprocesswhenthecurrentapproximationissuÆ-

cientlyclosetotheintervalhull.Itissurprisinglysimpletogetasimple

estimateoftheerror:Becausef(X(l))�F(X(l))andf(X(u))�F(X(u))

we�ndthat

d(Fp(X);f(X))�max

� d(F(X(l));f(X(l)));d(F(X(u));f(X(u)))

�max

� w(F(X(l)));w(F(Xu))

:

Therefore,ifthesimplestoppingcriterion

max

� w(F(X(l)));w(F(Xu))

�Æ

issatis�edforthedesiredvalueofÆ,thend(Fp(X);f(X))�Æ.

7.AccurateIntervalHulls

36

IfthemeanvalueformFM

isused,thenwenotonlyhavetheadvantage

ofquadraticconvergence,butwealsoknowF0,i.e.wehaveinformation

aboutthepartialderivatives.Supposee.g.,thatF0(X(j)) i>0,thenf

ismonotonicincreasingonX(j)inthedirectionofxi,andifwereplace

theithcoordinateinX(j)withthelowerandupperbound,respectively,

thentheintervalhulloverX(j)iscontainedintheunionofFM

applied

tothetworeduced-dimensionintervals.Thus,monotonyinacoordinate

directiongivesfasterconvergenceinthatdirection,butwestillhave

tousenondegenerateintervalswhenextremaareattainedininterior

points.Moore(1975)andSkelboe(1977)reportconsiderablegainfrom

usingsuchinformationaboutthegradient.

WegivemoredetailsinChapter11,wherewefocuson�ndingthelower

boundoff(X).

8.

IntervalIntegration

Givenacontinuousfunctionf:R7!RontheintervalA=[a;b].We

wanttocomputetheintegral

T

=

Z b a

f(x)dx:

Thisproblemarisesinmanyconnections,andinsomecasesitiseasy:

Ifwecan�ndafunction'suchthat'0(x)=f(x),thenTissimply

evaluatedasT='(b)�'(a).Ifwecannot�ndsuchafunction{or

ifanimplementationofitisnotavailableanditisverycomplicated

todevelop,thenwecanresorttonumericalquadrature,andgetan

approximatione Ttotheintegral.Theapproximationwillbea�ected

bybothroundingerrorsandtruncationerrors,seeChapter1.Itmay

alsohappenthatthegivenfhasinputerror,i.e.itrepresentssome

theoreticalfunctionb finthesensethatf(x)=

b f(x)+e(x),andweare

givenboundsone.Whatisthee�ectofalltheseerrors,i.e.whatcan

wesayaboutjT�e Tj?

8.1.

BasicMethod

Itcancomeasnosurprisethatweo�erintervalanalysisasameansof

answeringallthesequestions.Themathematicalbackgroundisthead-

ditivepropertyofintegrationandthemeanvaluetheoremforintegrals.

Ifa�c�b,then

Z b a

f(x)dx

=

Z c a

f(x)dx+

Z b c

f(x)dx;

andthereexistsa�2X

=[x;x]suchthat

Z x x

f(x)dx

=

w(X)f(�):

Therefore,ifwesplittheintervalAintopsubintervalswithbreakpoints

fajg,

a=a0<a1<:::<ap�

1<ap=b;

(8.1)

8.1.BasicMethod

38

then

T

=

p X i=1

Z a i ai�

1

f(x)dx

=

p X i=1

w(Ai)f(�i);

whereAi=[ai�1;ai]and� i2Ai.

Now,letFbeanintervalextensionoffonA.Then

Ti

�w(Ai)f(�i)�Ri

�w(Ai)F(Ai);

andwehavederivedanintervalinclusionofT,

T

�R

�p X i=

1Ri:

(8.2)

ThewidthoftheintervalR

measureshowwellwehavedetermined

theintegral.IfwetakethemidpointtorepresentT,thenwehavethe

followingboundonthetruncationerror

jT�m(R)j�1 2w(R):

(8.3)

Example8.1.

Consider

T

=

Z 1 0

11+x

dxwiththesolutionT

=

ln2'

0:69314718.Theintegrandismonotonicdecreasingintheintegration

domainA=[0;1],sowecanuseF(X)=[

11+x

;

11+x

].Ifwedivide

Aintopsubintervalsofequalwidth,thenthebreakpointsareaj=j=p,

j=0;1;:::;p,w(Ai)=1=p,andwe�nd

Ri

=

1 p[

pp+i

;

p

p+i�1

];

R

=

[1 p+

1p+1

+���+

1 2p

;

1p�1

+1 p+���+

12p�1

]:

Thus,jT�m(R)j�

1 2w(R)=

1 2� 1 p

�1�

1 2p

� =1 4

p;showingthat

jT�m(R)j�0:005ifp�50.

39

8.IntervalIntegration

Inthissimpleexampleweusedtheintervalhulltorepresentthevariation

offoneachsubinterval.Ingeneral,assumethatFisalinearinterval

extensionoff.ThenTheorem6.1tellsusthat

w(F(Ai))�Kw(Ai)

forsomeconstantK>0,andbymeansof(5.9)and(8.2)we�nd

w(R)=

P iw(Ri)=

P iw(Ai)�w(F(Ai))

�P iw(Ai)�Kw(Ai)=

K�Piw(Ai)2:

(8.4)

Withequidistantbreakpointseachw(Ai)=(b�a)=p,and(8.4)leadsto

w(R)�K�p�� b�

ap

� 2=

K1 p

:

(8.5)

Thus,withagoodintervalextensionoffwecanexpectthatadoubling

ofpreducesthetruncationerrorbyafactor2.

Supposethatwecanignoreinputandroundingerrors,thenwecould

simplyincreasep(e.g.successivelydoubleit)untilw(R)issuÆciently

small.Often,however,aconsiderableamountofcomputationcanbe

savedbyfocusingonthosepartsoftherange,whereF(X)hasthe

greatestwidth.Theframeworkofsuchanadaptivealgorithm

couldbe

thefollowing\divide-and-conquer"algorithm,whichstopswhenwecan

guaranteethatjT�m(R)j�ÆwiththevalueofÆchosenbytheuser,

ok:=false

while

notok

R:=

P iRi

ifw(R)�2Æthen

ok:=true

else j

:=argmaxfw(Ri)g

SplitAjintotwosubintervalsofequalwidth

end

end

(8.6)

8.2.IntervalSimpson

40

8.2.

IntervalSimpson

Themethoddescribedabovebehavessimilartothesimplemethodsfor

\ordinary"numericalintegrationknownastheleftandrightRiemann

formulas,

e T (L)=

b�a

p

� f(a0)+f(a1)+���+f(ap�

1)� ;

e T (R)

=

b�a

p

� f(a1)+���+f(ap�

1)+f(ap)� :

Iftheintegrandisdi�erentiable,wecangetaccurateapproximations

withfewerfunctionevaluations.AsanexampleconsiderSimpson'sfor-

mula.Iffisfourtimescontinuouslydi�erentiable,then

Ti

=

w(Ai)

6

� f(ai)+4f(m(Ai))+f(ai)� �

w(Ai)5

2880

f(4)(�i);

where� i2Ai.IfwehaveintervalextensionsFandGoffandf(4),then

wecanusethesetoget

T

�S

�p X i=

1Si;

(8.7)

with

Si

=

w(Ai)

6

� F(ai)+4F(m(Ai))+F(ai)� �

w(Ai)5

2880

G(Ai):

NotethatthecallsofFwithsingletonargumentsreturnintervalsthat

caterfore�ectsofroundingerrorsanduncertaintyoftheintegrand.

Ignoringthese,andassumingthatG

isalinearintervalextensionof

f(4),weget w

(Si)=

12880

w(Ai)5w(G(Ai))

�

12880

w(Ai)5KG

w(Ai)=

K2w(Ai)6;

andsimilartothederivationof(8.5)weseethatinthecaseofequidistant

breakpointswegetw(S)�K3=p5,showingthatwecanexpectthata

doublingofpreducesthetruncationerrorbyafactor32.

41

8.IntervalIntegration

Example8.2.

FortheproblemofthepreviousexamplewecanuseG(X)=

24=(1+X)5,andbelowwegivetheresultsforequidistantbreakpoints.

Notethatwithpsubintervalsweuse2p+1singletoncallsofFandpcalls

ofG.

p

m(S)

r(S)

1

0.69014756944444

4.04e-003

2

0.69308539735741

1.26e-004

4

0.69314610032219

3.94e-006

8

0.69314716308433

1.23e-007

16

0.69314718028433

3.85e-009

Eachtimepisdoubled,theradiusr(S)isreducedbythefactor32pre-

dictedabove.

Wecannotuse(8.7)onaproblemlikeT=

R 1 0pxdxbecausethefourth

derivativeoftheintegrandissingularatthelefthandendoftherange.

Wecoulduse(8.2),improvedby(8.6),butweshallassumethatsuch

asingularityoccursonlyatoneand/ortheotherend,andsuggestthe

followinghybridmethod,wheretheRiemannapproachisusedinthetwo

\boundaryintervals"andtheSimpsonapproachisusedintheinterior,

T

�H

�R1+Rp+

p�

1 X i=2

Si:

(8.8)

Wecancombinethiswiththedivide-and-conquerapproachof(8.6),with

specialhandlingofthecaseswherethecriticalsubintervalisabound-

aryinterval,j=1orj=p.Inthe�rstcaselet[a0;a1]bethecurrent

�rstintervalwithlength�=a1�a0.Wesplitthisintothenewbound-

aryinterval[a0;a0+�=32]andtheinteriorinterval[a0+�=32;a1].

Similarifthecriticalsubintervalisattheotherend.

IntheimplementationitisexploitedthatF(ai)=F(ai+1)andwhen

aninteriorintervalissplit,thenthemidpointisanendpointofboth

newsubintervals,soweonlyhavetoevaluateF

atthetwonewmid-

pointsandG

onthetwonewsubintervals.Alsothespecialhandling

oftheboundaryintervalscallsfor4evaluationsofanintervalfunc-

tion.Theinitiallayouthasp=3(oneinteriorinterval)andbreakpoints

a0:3=(a;a+�;b��;b),where�=(b�a)=32,andthe�rstH

needs

6evaluationsofanintervalfunction.

8.2.IntervalSimpson

42

Example8.3.

Forf(x)=

px=x

1=2

wecanuseG(X)=�

1516

X�

7=2,and

withthestoppingcriterionr(H)�10�

3

wefoundtheresult

H

=

[0.666333615;0.668563604]

obtainedwith40intervalfunctionevaluations.The�gureshowsthepoints

(x;F(x))thatwereusedinSimpson'sformula.Asexpected,theyconcen-

trateclosetothesingularityatx=0.

00.

10.

20.

30.

40.

50.

60.

70.

80.

91

0

0.2

0.4

0.6

0.81

Figure8.1.

Example8.4.

Forcomparison,wehavealsousedtheM

atlab6function

quadonthesquarerootproblem.Asourintervalalgorithm,thisisan

adaptivemethodbasedonSimpson'sformulaandbisectionof\critical"

subintervals.Thealgorithm

stopswhentwoconsecutiveapproximations

di�erlessthanÆ,thedesiredaccuracy.Inthetablebelowwegiveresults

forthenumberofsubintervalsandtheerrorsfordi�erentchoicesofthe

desiredaccuracy.

Thetwoalgorithmsuseaboutthesamenumberofsubintervals,butwhile

quaduses2p+1evaluationsofa oatingpointimplementationoff,the

intervalalgorithm

uses4p�6evaluationsofintervalfunctions,whichis

morethantwiceascostly.

Ontheotherhand,theintervalalgorithm

returnstheresulttothedesiredaccuracy,exceptforthelastexample.We

usedthe\safetyvalve"p�1000,whilequadmayproceeduntilthenumber

offunctionevaluationsexceeds10000.Inallcases(exceptforthe�rst,

wherep=2issuÆcient)theerroronthequadresultisuptoalmost10

timesthedesiredaccuracy.

Thetruevalueoftheintegral,T=

2 3

isinanintervalofwidth2�

53

'

1.11e-16betweentwo oatingpointnumbers.Withthatinmind,both

methodsachieveimpressiveaccuracy.

43

8.IntervalIntegration

Interval

quad

Æ

p

jm(R)�Tj

p

je S�Tj

1e-02

6

2.71e-04

2

8.91e-03

1e-04

12

4.54e-05

8

3.94e-04

1e-06

23

7.07e-07

18

6.34e-06

1e-08

52

4.48e-09

48

3.58e-08

1e-10

120

2.86e-11

110

5.63e-10

1e-12

296

1.24e-13

280

3.34e-12

1e-14

805

6.66e-16

700

4.90e-14

1e-16

1000

4.44e-16

1740

7.77e-16

TheintervalversionofSimpson'smethodneedsagoodintervalextension

off(4)(x),ortheuseofautomaticdi�erentiation,seee.g.Neumaier

(2001)orCorlissetal(2002).Chapter8inMoore(1966)discusses

furtherintervalintegrationmethodsbasedonTaylorexpansionsofthe

integrand.Alsohere,automaticdi�erentiationisuseful,andhegivesa

shortintroductioninhisChapter11.

9.

RootsofFunctions

Lookingforarootofafunctionf:A�Rn

7!Rm

meansthatwewish

to�ndapointx�

2Asuchthatf(x�)=0.Variantsoftheproblemisto

�ndallrootsinA,etc,etc.

9.1.

BasicMethod

Whenweuseintervalmethodsto�ndarootweneedanalgorithmthat

cantelluswhetheragivenintervalX�I(Rn)containsnorootsandan

algorithmthatcangiveusintervalsthatcontainrootsandareasnarrow

aswewish.

The�rstalgorithmisprovidedbythesimple

Theorem

9.1.IfFisanintervalextensionforf:A�I(Rn)7!Rm

andX�Awith02jF(X),thenfhasnorootsinX.

Proof.

Straightforward,since02jF(X))02jf(X).

IfwecombinethisresultwiththesubdivisionschemeofChapter7,we

havetheframeworkforasimplealgorithm.DivideAintosubintervals,

A=

S p j=1X(j),andoneachsubintervalcheckwhether02F(X(j)).If

not,thenthissubintervalisdiscarded,andtheremainingX(j)arefurther

subdivided.IfFisagoodapproximationtofandthesetofrootsin

Aforfisbenign(e.g.�nite),thenthemethodresultsinacollectionof

arbitrarilynarrowintervalsthatmaycontainroots,andtheremainder

ofAisguaranteedtocontainnoroots.

Example9.1.

LetA=[�1;2],f(x)=x�x

2

andF(X)=X(1�X).We

getF(A)=[�2;4],sotheremayberootsinA.Wesplittheinterval

intoX(1)=[�1;

1 2],X(2)=[

1 2;2],and�ndthat0iscontainedinboth

F(X(1))andF(X(2)).Sowesplitagainandgettheresultstabulated

below.

45

9.RootsofFunctions

j

1

2

3

4

X(j)

[�1;�1 4

]

[�1 4

;1 2]

[1 2;

5 4]

[5 4;1]

F(X(j))

[�2;�

516

]

[�

516

;5 8]

[�

516

;5 8]

[�2;�

516

]

Thus,afterthecomputationof7intervalextensionswecandiscardX(1)

andX(4),whilebothofthetwoneighbouringintervalsX(2)andX(3)may

containroots.Furtheruseofthealgorithm

willisolatethetworootsoff,

0and1.

Example9.2.

Themethodalsoworksformultipleroots.Considere.g.

f(x)=x

2�4x+4=(x�2)2onA=[�4;8]withtheintervalextension

F(X)=X�(X�4)+4.Weget

j

1

2

3

X(j)

[�4;0]

[0;4]

[4;8]

F(X(j))

[4;36]

[�12;4]

[4;36]

WecandiscardX(1)

andX(3)

andsubdividetheintervalX(2),which

containsthedoubleroot.

Inthe�rstexampletheproblemissimpleandtheintervalextensionis

quitegood.Foramorecomplicatedproblem

and/oraworseapproxi-

mationtof(X)manysubdivisionsmaybeneededbeforeweareableto

discardrootfreesubintervals.

MooreandJones(1977)describehowthemethodworksinmultiple

dimensions.Theyhalveconsecutivelyineachcoordinatedirectionand

keepalistofintervalsthatstillhavenotbeendiscarded.

Inthecasen=m=1themethodcanbeextendedtoproveexistenceof

simpleroots:IffiscontinuousandFappliedtothetwoneighbouring

intervalsofX(j)hasoppositesign,thenX(j)containsatleastoneroot,

seeFigure9.1below.

TheexistencetestcanbemademoreeÆcientbyjustexaminingF

at

points{singletons.TheendpointsofF([x;x])re ecttherounding

errorsinthecomputationof (f(x)),butexceptforthatwedonothave

todistinguishbetweenFandf.Nickel(1967)usedthistoconstructan

intervalversionoftheclassicalbisectionalgorithm.

9.2.IntervalNewton

46

X(j−

1)X

(j)X

(j+1)

F(X

(j−1)

) >

0

F(X

(j+1)

) <

0

Figure9.1.Testofexistenceofroot

Example9.3.

TheexistencetestonlyworkswhenX(j)

containsanodd

numberofroots.Theproblem

from

Example9.2hastworoots(viz.the

doubleroot2)inX(2),andthereforeF(X(1))�F(X(3))>0.

Untilnowwehavepresentedvariantsofintervalversionsofbisection.

Thismethodisrobustbutratherslow.Inthenextsectionwepresent

afasterintervalalgorithm,butbeforethatwepresentahybridmethod.

Letexdenotean(approximate)root,computedinordinary oatingpoint

bymeansofanyroot�ndingalgorithm,andletÆdenoteanestimateof

theerror.Thismeansthatweexpectthatthereisarootx�

suchthat

jx�

�exj�Æ.Inotherwords,de�netheinterval[a;b]=ex+[�Æ;Æ],

andifF([a;a])�F([b;b])<0,thenweknowthat[a;b]containsat

leastoneroot.Ifthetestisnotsatis�ed,wecanextendtheintervalto

[a;b]=ex+2[�Æ;Æ],etc.

9.2.

IntervalVersionofNewton'sMethod

Inthissectionwediscussthecasen=m

=1.ConsideranintervalX

thatcontainsarootx�

off,wherefiscontinuouslydi�erentiable.Then

themeanvaluetheoremtellsusthatthereexistsa�2X

suchthat

0=

f(x�)=

f(x)+(x�

�x)f0(�):

Assumingthatf0(�)6=0,thisleadsto

x�

=

x�

f(x)

f0(�)

:

47

9.RootsofFunctions

Now,letF0

beanintervalextensionoff0

andassumethatF0(X)does

notcontain0.Then

x�

2N(x;X)�x�

f(x)

F0(X)

;

(9.1)

wherewehaveintroducedtheNewtonoperatorN(x;X).Therootisalso

containedinX,andthereforeitmustlieintheintersectionbetweenX

andN(x;X),

x�

2X\N(x;X):

Thus,wecanformulatetheintervalversionofNewton'smethod:Start

withX(0)

containingx�

andcomputeanestedsequenceofintervals

X(1);X(2);:::bytheformula

X(s+1)

=

X(s)\N(x(s);X(s))

with

x(s)2X(s);

s=0;1;::::

(9.2)

Theterm\nested"isusedbecauseeachnewintervaliscontainedinthe

previous,X(0)�X(1)����.Thisimpliesthatthewidthsdecrease,

w(X(0))�w(X(1))�:::,andsincealltheX(s)arecontainedinX(0),

theyarebounded.Therefore,thereexistsalimitX�

containingx�.

Example9.4.

Considerthefunctionf(x)=

x2�2withf0(x)=

2xand

X(0)

=[1;2].Wechoosex(s)

=m(X(s))andF0(X)=2X,sothatthe

Newtonoperatoris

N(x(s);X(s))=

m(X(s))�(m(X(s))2�2)=(2X(s)):

The�rststepwith(9.2)is

N(x(0);X(0))=

3 2

�1 4=[2;4]=

[2216;

2316];

X(1)

=[1;2]\[

2216;

2316]=

[2216;

2316]:

Below

weshow

resultsobtainedbytheM

atlab

ToolboxIntlab,and

displayedwithformat

long.

9.2.IntervalNewton

48

s

X(s)

1

[1.37499999999998;1.43750000000001]

2

[1.41406249999998;1.41441761363637]

3

[1.41421355929452;1.41421356594719]

4

[1.41421356237309;1.41421356237310]

TheconditionthatF0(X)doesnotcontain0isnecessaryfortheappli-

cationoftheNewtonoperator,andithasaninterestingimplication,

Lemma9.1.

Iff:R7!Riscontinuouslydi�erentiableonthe

intervalX

and02jF0(X),thenX

eithercontainsasimplerootx�

ornoroots.

Proof.

Theemptypartisobvious,soassumethatthereiseithera

multiplerootormorethanonerootinX.

Arootx�

offofmuliplicityqsatis�es

f(x�)=

f0(x�)=

���=

f(q�

1)(x�)=

0:

However,theconditiononF0(X)impliesthatf0(x)6=0forallx2X.

Therefore,X

cannotcontainarootofmultiplicityq�2.

Next,considertwo(ormore)separate,simplerootsx�;y�

2X,y�

6=x�.

Theymustbeseparatedbyapointz2X,wherefhasalocalextremum,

i.e.f0(z)=0.ButtheassumptiononF0

excludestheexistenceofsuch

apoint.

InExample9.4wehadaveryfastconvergencetowardsthesimpleroot

x�

=

p2.Thisexempli�edthatundertheconditionsofconvergence

thelimitX�

isthesingletonX�

=x�,irrespectiveofhowwechoosethe

pointsx(s)2X(s),

Theorem

9.2.

Ifx�

isarootoffinX(0)andF0(X(0))doesnot

contain0,thenthesequenceX(1),X(2),...de�nedby(9.2)converges

tox�.

49

9.RootsofFunctions

Proof.

From(9.2)andthede�nition(9.1)oftheNewtonoperatorwe

seethat

w(X(s+1))�w(N(x(s);X(s)))=

w(f(x(s))=F0(X(s)))

=

jf(x(s))jw(1=F0(X(s)))

�jf(x(s))jw(1=F0(X(0)))=

Kjf(x(s))j;

(9.3)

whereK

=w(1=F0(X(0)))isindependentofX(s).Sincex�

iscontained

ineachX(s),itispossibletochoosefx(s)gsothatx(s)!x�,inwhich

case(9.3)showsthatw(X(s))!0,X(s)!x�.

Wehavetoprovethattheconvergenceisobtainedforallchoicesof

x(s)2X(s),andaccordingto(9.3)thisisensuredifwecanshowthatitis

onlypossibletochoosea�nitenumberofx(s)forwhichjf(x(s))j>">0.

Suchapointsatis�es

� � � �f(x(s))

F0(X(s))

� � � ��jf(x(s))j

jF0(X(s))j�

jf(x(s))j

jF0(X(0))j

;

and

X(s+1)=X(s)\

� x (s)�

f(x(s))

F0(X(s))

� showsthatthedistancefrom

x(s)toX(s+1)isatleast"=jF0(X(0))j,andsincex(s)2X(s),thisimplies

thatw(X(s+1))�w(X(s))�"=jF0(X(0))j.Now,w(X(0))is�nite,sowe

canonlysubtractthepositivenumbera�nitenumberoftimes,andthis

concludestheproof.

TheNewtonoperatorcanhelpusansweringthequestionraisedinLemma

9.1abouttheexistenceornon-existenceofarootsinagiveninterval,

Theorem

9.3.

IfX

isanintervalwith02jF0(X)andthereexists

anx2X

suchthatX\N(x;X)=;,thenfhasnorootinX.

Proof.

Bycontradiction.IfX

containedaroot,thenitwouldalso

becontainedinN(x;X)andintheintersection.SinceX\N(x;X)is

empty,therecanbenorootinX.

9.2.IntervalNewton

50

Theorem

9.4.

Iffiscontinuouslydi�erentiable,X

isaninterval

with02jF0(X),andthereexistsanx2X

suchthatN(x;X)�X,

thenfhasarootinX.

Proof.

ThisproofismorediÆcult;itusesSchauder'stheoremabout

existenceof�xedpoints.

Considerthepointxandanotherpointy2X.Accordingtothemean

valuetheoremthereexistsa�betweenxandysothat

f(x)=

f(y)+f0(�)(x�y):

(9.4)

Sincef0(�)2F(X),itisnonzero,andwede�nethefunctiong,

g(y)=

y�

f(y)

f0(�)

:

(9.5)

Thisiscontinuous,andfrom(9.4)andtheassumptiononN(x;X)we

get

g(y)=

x�

f(x)

f0(�)2x�

f(x)

F0(X)

=

N(x;X)�X

:

Thus,gisacontinuousmappingofXintoitself.AccordingtoSchauder's

theoremghasa�xedpointinX,y�

=g(y�),and(9.5)showsthatthis

isarootoff.

NowwearereadytosketchanalgorithmthatexaminesanintervalA

forsimpleroots:

1.

SplitA

intosubintervalsA(1);:::;A(p),whereeachsubinterval

satis�eseither02jF0(A(j))orw(A(j))�Æ.Thesecondtypecon-

sistsofsuÆcientlynarrowintervalsaroundlocalextremaandmul-

tipleroots.

2.

LetX(0)=A(j)

beasubintervalwith02jF0(X(0)).

IfX(0)\

N(x(0);X(0))=

;,thenwecandiscardthissubinterval,other-

wiseproceedwith(9.2).

51

9.RootsofFunctions

IfanyoftheiteratesX(s)satis�esN(x(s);X(s))�X(s),thenweareguar-

anteedthatthereisarootinX(s),andthereforeinX(0).Depending

onthechoiceofx(s),someiterationstepsmaybeneededbeforethe

non-existenceortheguaranteedexistenceofarootisrevealed.

From

thede�nition(9.1)oftheNewtonoperatorandtheassumption

02jF0(X)itfollowsthatallpointsintheintervalN(x;X)areeitherto

theleftortotherightofthepointx.Thishastheimportantimplication

w(X\N(m(X);X))<

1 2w(X):

(9.6)

Thus,ifwetakex(s)asthemidpointofX(s),wehaveaguaranteed

reductionoftheintervalwidth.Ifthereisarootintheinterval,then

weachievefaster�nalconvergence:

Theorem

9.5.IfthereisarootinX�X(0),F0

isalinearinterval

extensionoff0,and02jF0(X),thenthereexistsaconstantK>0

suchthat

w((N(x;X))�Kw(X)2

forall

x2X

:

Proof.

From(9.3)weknowthatjw(N(x;X))j�K1jf(x)j,andfrom

themeanvaluetheoremandthelinearityofF0

weget

jf(x)j=

jf(x�)+f0(�)(x�x�)

�jf0(�)jw(X)�jF0(X)jw(X)�K2w(X)�w(X);

andthetheoremfollows.

Thistheoremshowsthatunderthegivenassumptionsthesequence(9.2)

satis�esw(X(s+1))�Kw(X(s))2,i.e.ithasquadraticconvergence,as

illustratedinExample9.4.

ThemethodcanbemademoreeÆcientifwerealizethatx(s)canbe

chosenfreelyinX(s).Theestimate(9.3)showsthatthewidthofX(s+1)

isspeciallysmallifwechoosex(s)closetotheroot,andtoreducethe

amountofintervalcomputationwecanuseanordinary oatingpointal-

9.3.NonlinearSystem

52

gorithmto�ndanapproximationextox�,andusex(s)=ex.Theensuing

computationmustbedoneinintervalarithmetic.

Example9.5.

Itiseasilyseenthat

f(x)=

x3+x

2+3x+1

hasarootbetween�1and0.WithX(0)=[�1;0],F0X)=3+X(2+3X)

and

x(s)=m(X(s))wegetthefollowingresultsfrom

(9.2),

s

X(s)

w(X(s))

1

[-0.43750000000001;-0.12499999999998]

3.13e-01

2

[-0.37936994154677;-0.34535656580493]

3.40e-02

3

[-0.36113616568634;-0.36106913433279]

6.70e-05

4

[-0.36110308055118;-0.36110308050612]

4.51e-11

5

[-0.36110308052865;-0.36110308052864]

5.55e-17

Again,wenotethequadraticconvergence.Thisismostclearlyseeninthe

columnwithw(X(s)).Roundingerrorshavedominatingin uenceinthe

laststep.

Threestepswiththeordinary oatingpointNewtonalgorithm

from

the

startingpoint�0:5givesex=�0:36110308052865,andusingtheinterval

NewtonoperatorN(ex;X(0))weget

X(1)

=

ex�f(ex)

F0(X(0)

=

[-0.36110308052865;-0.36110308052864]

w(X(1))=

3.0e-15

Thus,three oatingpointstepsplusoneintervalstepgivesabetterresult

thanfourintervalsteps.

9.3.

MultidimensionalNewton'sMethod

Inthissectionweconsidercontinuouslydi�erentiablefunctionsf:Rn

7!

Rn,andseekx�

2Rnsuchthatfi(x�)=0;i=1;:::;n.Thisisasystem

ofn(nonlinear)equationsinnunknowns,thecomponentsx� 1;:::;x� n

of

x�.

53

9.RootsofFunctions

Again,westartwiththemeanvaluetheorem,butthistimeinitsmul-

tidimensionalversion.Letx�

2X�I(Rn)bearootforx.Thereexist

vectors� [1];:::;� [n]2X

suchthat

0=

f(x�)=

f(x)+J(x;x�)(x�

�x);

whereJ2Rn�

n

istheJacobianwiththeelements

� J(x;x�)� i;j

=

@fi

@xj

(�[i]):

AssumethattheJacobianisnonsingularforallx2X.Thentheinverse

ofthismatrixexists,and

x�

=

x�

� J(x;x�)� �1f(x)2x�V(X)f(x);

(9.7)

whereVisanintervalmatrixcontainingallthepossibleJ�

1,

f� J(x;x�)� �1;x2Xg�V(X):

Thus,from

X(0)containingtherootx�

wecanconstructanestedse-

quenceofintervals,thatallcontainx�,

X(s+1)

=

X(s)\N(x(s);X(s));

s=0;1;:::

with

N(x;X)=

x�V(X)f(x);

x(s)2X(s):

(9.8)

Example9.6.

Considerthefunctionf:R

2

7!R

2

de�nedby

f(x)=

0 @f 1(x)

f2(x)

1 A =0 @x2 1+x

2 2�1

x1�x2

1 A

ontheintervalX(0)=([

1 2;1];[

1 2;1]).TheJacobianis

J(x;x�

)=

2� 11

2� 12

1

�1! ;

� [1]=(�11;� 12):

Thisisnonsingularif� 11+� 126=0,whichissatis�edforall� [1]2X(0).The

inverseis � J(x

;x�

)� �1=

1

� 11+� 12

1 2

� 12

1 2

��11

! ;

9.3.NonlinearSystem

54

andwecanuse

V(X)

=

1

X1+X2

1 2

X2

1 2

�X1

! .

Ifwechoosethe

midpointsx(s)=m(X(s))in(9.8),wegetthefollowingresults

s

X(s);1=X(s);2

w(X(s))

1

[0.68749999999998;0.71875000000001]

3.12e-02

2

[0.70703124999999;0.70720880681819]

1.78e-04

3

[0.70710677964726;0.70710678297359]

3.33e-09

4

[0.70710678118654;0.70710678118655]

1.11e-16

5

[0.70710678118654;0.70710678118655]

1.11e-16

Weseeafast(quadratic)convergencetothesolution,x� 1

=

x� 2

=

p0:5.

ThewidthofX(4)ishalfofthemachineaccuracy,andwecannotgetcloser

tox�

becauseofroundingerrors.

Example9.7.

Nowletf:R

2

7!R

2

bede�nedby

f(x)=

0 @x2 1+x

2 2�5

x1x2�2

1 A

ontheintervalX(0)

=

([1:6;2];[1;1:4]).Ithastherootx�

=

(2;1).

TheJacobianisJ(x;x�

)=

2�11

�2� 12

��22

� 21

! .ItisnonsingularinX(0)

andwecanuse

V(X)=

1

2(X1X1�X2X2)

X1

�2X2

�X2

2X1

! ;

Ifwetakex(s)

=

m(X(s))in(9.8),wegetthesequence(displayedwith

fewerdigitsthanbefore)

s

X(s);1

X(s);2

w(X(s))

1

[1.938666666;2]

[1;1.06133334]

6.13e-02

2

[1.998451819;2]

[1;1.001548181]

1.55e-03

3

[1.999999001;2]

[1;1.000000998]

9.98e-07

4

[1.999999999;2]

[1;1.000000001]

4.15e-13

5

[1.9999999999;2]

[1;1.000000001]

2.22e-16

6

[2;2]

[1;1]

0

Theonlynew

featureisthatnow

thecomponentsoftherootcanbe

representedin oatingpointwithouterror.

55

9.RootsofFunctions

If,insteadwetakex(0)=[2;1:4],weget

N(x(0);X(0))=

� [0.666666666;3.610666667]

[-1.10666667;2.093333334]� �

X(0):

Therefore,thesequencede�nedby(9.8)getsstuckatX(0),anditshows

thatthechoiceofx(0)2X(0)canhavegreatimportance.

Thelastexampleshowedthatthesequencede�nedby(9.8)doesnotnec-

essarilyconvergetoasingleton.ApossiblecauseisthatV(X)maycon-

tainsingularmatrices,inwhichcasetheintervalV(X(s))f(x(s))cancon-

tainthezerovectoreveniff(x(s))6=0,andx(s)2N(x(s);X(s))�X(s+1)

withx(s)6=x�.Ifwechoosex(s+1)=x(s),bothx(s)andx�

arecontained

inX�

=lims!

1

X(s),andthereforew(X�)>0.However,thefollowing

theoremisvalid,

Theorem

9.6.

Letx�

bearootoffinX(0)andassumethat

V

isinclusionmonotoniconX(0).Ifthefx(s)garechosensothat

lims!

1

x(s)=x�

thenlims!

1

X(s)=x�.

Proof.

X(s+1)�x(s)�V(X(s))f(x(s))�x(s)�V(X(0))f(x(s)),and

thetheoremfollowsfromf(x(s))!0.

Thistheorem

hasanimportantpracticalimplication:Wecanusean

ordinarynumericalmethodtocomputethesequencefx(s)gandcombine

itwith(9.8)toboundtheerror.Themethodcane.g.betheordinary

Newtonmethod,

x(s+1)

=

x(s)�

� J(x(s);x(s))� �1f(x(s));

(9.9)

whereJ(x(s);x(s))istheJacobianevaluatedatx(s).Ifx(s+1)computed

by(9.9)isnotcontainedinX(s+1)ascomputedby(9.8),thenwejust

replaceitbytheclosestpointinX(s+1).IftheNewtonprocessconverges,

sodoesthesequencefx(s)g,andthetheoremcanbeapplied.Asinthe

one-dimensionalcaseaconsiderableamountofintervalcomputationcan

besavedifwejustuse(9.8)toestimatetheerroroftheresultsfromthe

ordinary oatingpointalgorithm.

9.3.NonlinearSystem

56

AseriousproblemwiththemultidimensionalNewtonmethodishowto

�ndV

?Intheaboveexamplesitwasderivedfromanalyticalexpres-

sionsfor

� J(x;x�)� �1,buttoautomatethisapproachwewouldneeda

symboliclanguagelikee.g.Maple.Alternativelywecoulduseanin-

tervalextensionofJ(x;x�)andinvertthisnumerically,asdiscussedin

Chapter10.

Finally,Theorems9.3and9.4abouttheexistenceofarootgeneralize

tothemultidimensionalcase.

Theorem

9.7.IfX

isanintervalinwhichV(X)existsandwecan

�ndanx2X

sothatX\N(x;X)=;,thenthereisnorootinX.

Proof.

Followsfrom(9.7).

Theorem

9.8.Iffiscontinuouslydi�erentiable,Xisanintervalin

whichV(X)exists,andthereexistsanx2XsuchthatN(x;X)�X,

thenfhasarootinX.

Proof.

SimilartotheproofofTheorem9.4:Forthegivenxandany

y2X

thereexistsanonsingularmatrixJ(x;y)suchthat

f(x)=

f(y)+J(x;y)(x�y):

Next,wede�ne

g(y)=y�

� J(x;y)� �1f(y):Thisiscontinuousiny,

� J(x;y)� �12V(X),andweseethat

g(y)=

x�

� J(x;y)� �1f(x)2N(x;X)�X

:

TheremainingpartoftheproofisasintheproofofTheorem9.4.

57

9.RootsofFunctions

9.4.

Krawczyk'sVersionofNewton'sMethod

Krawczyk(1969)presentedaversionofNewton'smethodthatavoidsthe

inversionofintervalmatrices.Thismethodsharesthenicepropertiesof

Newton'smethod,butmayhaveslightlyslowerconvergence.

Asbefore,westartbylookingatanintervalX

containingarootx�

of

f.Letx2X

andletH

beanarbitrarymatrixinRn�

n.Then

x�

=

x�Hf(x)+(x�

�x)�H

� f(x�)�f(x)�

=

x�Hf(x)+(I�HJ(x;x�))(x�

�x)

2K(x;X)�x�Hf(x)+(I�HJ(X))(X�x):

(9.10)

Here,I=diag(1;:::;1)istheunitmatrix,andweintroducedtheso-

calledKrawczykoperatorK(x;X).IfwestartwithanintervalX(0)

containingarootx�,thentheformula

X(s+1)

=

X(s)\K(x(s);X(s))

with

x(s)2X(s)

s=0;1;:::

(9.11)

generatesanestedsequenceofintervals,allofwhichcontaintheroot

x�.ForthissequencewehavethefollowingequivalentstoTheorems9.7

and9.8,

Theorem

9.9.

IfthereexistsanxintheintervalX

andamatrix

H2Rn�

n

suchthatX\K(x;X)=;,thenthereisnorootinX.

Proof.

SimilartotheproofofTheorem9.3.

Theorem

9.10.IfthereexistsanxintheintervalX

andanonsin-

gularmatrixH2Rn�

n

suchthatK(x;X)�X,thenthereisaroot

inX.

9.4.Krawczyk'sMethod

58

Proof.

AsintheproofofTheorem9.4welookat�xedpoints.For

anyy2X

weseethat

y�Hf(y)=

x�Hf(x)+y�x�H(f(y)�f(x))

=

x�Hf(x)+(I�HJ(y;x))(y�x)

2x�Hf(x)+(I�HJ(X))(X�x)

=

K(x;X)�X

:

Thus,thecontinuousfunctiony�Hf(y)mapsX

intoitself,andthere

existsa�xedpointy�:y�

=y�

�Hf(y�).BecauseH

isnonsingular,

thisimpliesthatf(y�)=0,i.e.thereisarootforfinX.

Itfollowsfrom(9.11)thatifx(s)andH(s)canbechosensothatK(x(s);X(s))

isnarrow,thenwegetfastconvergence.Moreprecisely,from(9.11)we

�nd

w(X(s))�w(K(x(s);X(s)))

=

w((I�H(s)J(X(s)))(X(s)�x(s))

�kI�H(s)J(X(s))kw(X(s))�%sw(X(s)):

Thenorm

k�kofanintervalmatrixwasde�nedin(2.4).Wewant

tomake%s

assmallaspossible.ThisisachievedbytakingH(s)

=

� m(J(X(s)))

� �1,becausethenH(s)J(X(s))iscenteredaroundI.The

choice

x(s)

=

m(X(s))

H(s)

=

( anapproximationto

� m(J(X(s)))

� �1if%s�%s�

1

H(s�

1)

if%s>%s�

1

(9.12)

isthesubjectof

Theorem

9.11.LettheKrawczykoperatorusethechoice(9.12).If

K(x(0);X(0))�X(0)and%0<1,thenx�

2X(s),lims!

1

X(s)=x�

and

w(X(s))�%s 0w(X(0)).

59

9.RootsofFunctions

Proof.

Followsimmediatelyfrom

w(X(s))�%s�

1w(X(s�

1)),andthe

sequencef%sgismonotonicdecreasingbecauseoftheinclusionmono-

tonicityofJ(X)andthechoiceofH(s)in(9.12).

Thus,ifG(0)=I�H(0)J(X(0))satis�eskG(0)k<1,thenweareensured

atleastlinearconvergence.Underthisconditionwemayalsosimplify

theprocesswithoutloosingthelinearconvergence:Replace(9.11)by

X(s+1)

=

X(s)\

� x (s)�H(0)f(x(s))+G(0)(X(s)�x(s))� :(9.13)

ThenweavoidthecomputationofJ(X(s))andtheinversionofitsmid-

pointineachstep,buttypicallythissimpli�cationincreasesthenumber

ofstepsneededtoobtainasatisfactorynarrowinterval.

Example9.8.

Weseekarootforf(x)=

� x2 1+x

2 2�1

x2 1�x2

� ,andassumethat

weknowtheapproximaterootx(0)=(0:8;0:62),e.g.foundbya oating

pointmethod.WecanuseKrawczyk'smethodbothtotestwhetherthere

isarootclosetox(0),sayinX(0)=([0:7;0:9];[0:5;0:7]),andtogeta

betterapproximationifthisissatis�ed.

ItistrivialtoseethatJ(X)=

� 2X1

2X2

2X1

�1� ,

andwith

H

=

� 0:279

0:346

0:446

�0:446

� '� J(x

(0))� �1

weget

G0

=� [�0:1250;0:1250][�0:0446;0:0670]

[�0:1784;0:1784][�0:0704;0:1080]� ;

%0=0:2864<1;and

K(x(0);X(0))=

� [0:7657;0:8064];[0:5872;0:6481]� �X(0):

ThisshowsthatthereisarootinX(0)andwecanuse(9.13)to�ndit.In

thetablebelowwegivetheresultsfromthisalgorithmwithx(s)=m(X(s))

andthestoppingcriterionw(X(s))�10�

6.

9.4.Krawczyk'sMethod

60

s

X(s)1

X(s)2

w(X(s))

1

[0.7657323;0.8063645]

[0.5872375;0.6480857]

6.08e-02

2

[0.7815712;0.7907271]

[0.6111227;0.6249431]

1.38e-02

3

[0.7851161;0.7871866]

[0.6164709;0.6195970]

3.13e-03

4

[0.7859172;0.7863856]

[0.6176805;0.6183875]

7.07e-04

5

[0.7860984;0.7862044]

[0.6179540;0.6181140]

1.60e-04

6

[0.7861394;0.7861634]

[0.6180159;0.6180521]

3.62e-05

7

[0.7861486;0.7861541]

[0.6180298;0.6180381]

8.18e-06

8

[0.7861507;0.7861520]

[0.6180330;0.6180350]

1.85e-06

9

[0.7861512;0.7861516]

[0.6180337;0.6180342]

4.18e-07

ThetrueKrawczykmethod(9.11)withthechoices(9.12)convergescon-

siderablyfaster;quadratically.

s

X(s)1

X(s)2

w(X(s))

1

[0.7657323;0.8063645]

[0.5872375;0.6480857]

6.08e-02

2

[0.7850995;0.7872034]

[0.6164672;0.6196009]

3.13e-03

3

[0.7861485;0.7861542]

[0.6180298;0.6180382]

8.35e-06

4

[0.7861513;0.7861514]

[0.6180339;0.6180340]

5.93e-11

Example9.9.

Theone-dimensionalversionoftheKrawczykoperatoris

K(x;X)=

x�Hf(x)+(1�HF0(X))(X�x)

withH

'1=f0(x)asagoodchoice.Newton'smethod(9.2)canbeused

onlyif02jF0(X(s)),butthisversionofKrawczyk'smethodonlydemands

thatf0(x(s))6=0.

The1D

versionof(9.13)iscloselyrelatedtotheordinaryroot�nding

algorithm

knownasthe\saw-toothmethod",x(s+1)

=

x(s)�Gf(x(s)),

wheretheconstantGis(anapproximationto)1=f0(x(0)).

Finally,theremarkinExample9.9generalizes:withKrawczyk'smethod

wedonotdemandthatallmatricesinJ(X(s))beinvertible.Thetest

ofexistenceinTheorem

9.10onlydemandsthatH(s)isnonsingular,

andthetestinTheorem

9.9doesnotevenrequirethis.

Therefore,

Krawczyk'smethodcanbeusedtosearchregionsinRnforroots.Moore

andJones(1977)usedthiscombinedwithintervalsubdivisiontogetan

eÆcient,globalalgorithmforroot�nding.

10.

LinearSystemsofEquations

Thisisprobablythemostcommonnumericalproblem,bothinitsown

rightandaspartofaniterativeprocessaswesawinthepreviouschapter.

Inmatrix-vectornotationtheproblemis:GiventhecoeÆcientmatrix

A2Rn�

n

andright-handsidevectorb2Rn,�ndx2Rn

suchthat

Ax

=

b:

(10.1)

WeassumethatA

isnonsingular,inwhichcasethesolutionxexists

andisunique.Now,supposethattheelementsinAandbareuncertain,

butweknowboundsforeachofthem.Wecanusetheseboundsasend

pointsofintervals,andreplace(10.1)with

AX

=

b;

(10.2)

whereAandbaretheintervalmatrixandvectorasdescribedabove,

andweseekX2I(Rn)thatcontainsthesolutiontoeveryproblemen-

compassedin(10.2).WeassumethatAisregular,meaningthatevery

realmatrixcontainedinAisnonsingular.

Example10.1.

Considerthesystem

[2;3]X1+[�1;2]X2

=

[3;4]

[1;3]X1+[4;6]X2

=

[2;4]

Avectorx=(x1;x2)2X

mustsatisfybothequations.The�rstequation

canbereformulatedto

[2;3]x1

=

[3;4]�[�1;2]x2

;

whichhasthesolution

x1

=

8 > > > > < > > > > :[3 2

�x2;2+

1 2x2]

for

x2

�3 2

[1�

2 3x2;2+

1 2x2]for

0�x2<

3 2

[1+

1 3x2;2�x2]

for�3�x2

<0

[3 2

+1 2x2;2�x2]

for

x2

<�3

Thesetofpoints(x1;x2)givenbythisexpressionaremarkedbyhorizontal

hatchinginFigure10.1.Similarly,we�ndthatthepointssatisfyingthe

secondequationaregivenby

10.1.GaussianElimination

62

x2

=

8 > > > > < > > > > :[1 2

�3 4x1;

2 3

�1 6x1]for

x1�4

[1 2

�3 4x1;1�

1 4x1]for

2 3

�x1<4

[1 3

�1 2x1;1�

1 4x1]for0�x1<

2 3

[1 3

�1 6x1;1�

3 4x1]for

x1<0

Thesepointslieintheverticallyhatcheddomain.Thesetofsolutionsto

thesystemistheintersectionofthetwodomains,thestar-shaped,doubly

hatchedregion.

(6, 0

.9)

(0.4

, −4)

Figure10.1.

AsdiscussedinExample4.1,wecannotgivesucha�gureastheresult,

buthavetomakedowiththeintervalhull,markedbythedottedrectangle

inthe�gure.Thus,thesystem

hasthesolution

X

=

� [0:4;6]

[�4;0:9]� :

10.1.

GaussianElimination

Themethodsusedforordinary oatingpointsolutionoflinearsystems

ofequations(atleastforsmallandmediumsizedproblems,n< �200)

arevariantsofGaussianelimination,possiblywithpivoting.Togivethe

backgroundforintervalversionswestartwithaquitedetaileddescrip-

tion.

63

10.LinearSystems

Step1.Transform

Ax=btoUx=y,whereUisupper

triangular.ThenonzeroelementsofUoverwrite

theuppertriangleofAandyoverwritesb

fork=1;:::;n�1

pk=argmaxi=k;:::;njAi;kj

ifpk>kthenswaprowskandpk

inAandb

fori=k+1;:::;n

` i;k=Ai;k=Ak;k

Ai;j:=Ai;j�` i;kAk;j;

j=k+1;:::;n

b i:=b i�` i;kb k

end

end

Step2.SolveUx=ybybacksubstitution.

fori=n;n�1;:::;1

xi=

� b i�P n j

=i+1Ai;jxj

� =Ai;i

end

(10.3)

ThematrixA2Rn�

n

isnonsingulari�allthepivotsUk;k

(storedin

Ak;k)arenonzero.

Fortheintervalsystem

(10.2)thesimplestideaistoreplaceallvari-

ablesandoperationsintheGaussalgorithmbytheirintervalequivalent.

Unfortunately,thisapproachoftengivesverypessimisticresults{ex-

cessivelywideintervalsXj

{andtoooftenitbreaksdownbecauseof

thecurrentpivotAk;k

containing0.P.Wongwises(1975)madeavery

detailedstudyandafurtherdiscussioncanbefoundinNickel(1977).

Theessentialproblemisthatsomeofthematrixelementstakepartin

upton�1ofthetransformationsAi;j:=Ai;j�` i;kAk;j,andeachtime

therelativewidthoftheintervalcangrow.

Example10.2.

Fortheproblem

ofExample10.1,

� [2;3][�1;2]

[1;3]

[4;6]

� X

=

� [3;4]

[2;4]� ;

Step1oftheintervalversionof(10.3)withoutpivotinggives

` 2;1=[

1 3;

3 2];

U=

� [2;3][�1;2]

0

[1;7:5]� ;

y=

� [3;4]

[�4;3]� ;

10.1.GaussianElimination

64

andbybacksubstitutioninUX

=yweget

X

�� [�1:5;6]

[�4;3]

� :

BycomparisonwithExample10.1weseethatforthisproblem

wegetthe

correctvalueforx1

andx2,butinbothintervalstheotherendpointis

toopessimistic.

Example10.3.

Todemonstratetheover-pessimismaboutsingularityofthe

matrix,weneedaproblem

ofordern�3.Considertheproblem

(10.2)

with

A

=

0 @[2;4][0;2]

[�1;1]

1

[3;4][�:5;1:5]

[4;6][3;5]

[8;10]

1 A ;b=

0 @[8;10]

[10;12]

[8;10]

1 A :

Bymeansoftheintervalversionof(10.3)withoutpivotingwe�nd

` 2;1=[:25;:5];

` 3;1=[1;3];

` 3;2=[�1:5;2:5];

U=

0 @[2;4][0;2][�1;1]

0

[2;4][�1;2]

0

0

[0;16]

1 A ;e b=

0 @[5;10]

[�47;17]

[�64:5;27:5]1 A :

Since02U3;3

wecannotproceed.Wewillnowshow,thatthezerocan

notbeobtainedwithanyofthematricesinA.Itfollowsfrom

(10.3)and

theabovematrices,that

U3;3

=

A3;3�` 3;1A1;3�` 3;2U2;3

=

[8;10]�[1;3]�[�1;1]�[�1:5;2:5]�[�1;2]

=

[8;10]�[�3;3]�[�3;5]:

TogetthezeroinU3;3

wehavetocombinetheworstcaseinthetwo

subtrahends.TheyoccurforA1;3=1andU2;3=2,butthisendpointof

U2;3

isachievedonlyifA1;3=

�1,i.e.theotherendoftheintervalA1;3.

BymeansofthemethoddescribedinSection10.2belowwefoundthat

X

=

0 @[�2;10:0500]

[0:1200;5:7000]

[�10:1000;�0:0258]

1 A :

65

10.LinearSystems

Example10.4.

Welookatthesameproblem

asinthepreviousexample,

exceptthatwechangeA3;3to[8:5;10].Theonlychangeinthereduction

(Step1in(10.3))isthatnowU3;3=[0:5;16],andproceedingwithback

substitutionwe�nd

X

�0 @[�142;123:75]

[�44:5;99]

[�94;34]

1 A :

Thesetofallsolutionsisnowcontainedin

X

=

0 @[�1:8947;9:3914]

[0:2037;5:2632]

[�8:7827;�0:0258]1 A :

Theboundsobtainedbythesimplereplacementof oatingpointvariables

andoperationsbytheirintervalequivalentaremorethan10timeswider

thanthetrueinterval.

InordertoimprovetheaccuracywemaytryKrawczyk'smethodfrom

Section9.4.Asolutionx2X

to(10.2)isarootofthefunctionf(x)=

Ax�b,andtheKrawczykoperator(9.10)takestheform

K(x;X)=

x�H(Ax�b)+(I�HA)(X�x);

whereH

=� m(A)� �1istheoptimalchoiceforthematrixH.IfkI�

HAk<1andweknowaninitialX(0)thatcontainsasolution,thenthe

methodshouldwork.

Example10.5.

Fortheproblem

ofExamples10.1and10.2weget

H

=

� 2:50:5

2

5� �1

=

1 23

� 10�11

�45

� ;

I�HA

=

1 23

� [�6;6][�16;16]

[�7;7][�11;11]� ;

kI�HAk=

2223

<

1;

sothenecessaryconditionforconvergenceofKrawczyk'smethodissatis-

�ed.However,withtheintervalfoundinExample10.2,X(0)=

([�1:5;6];[�4;3])wehavenotbeenableto�ndanx2X(0)thatworks.

InallthecasestriedwegetK(x;X(0))�X(0),soKrawczyk'smethodis

stuckatX(0).

10.1.GaussianElimination

66

Example10.6.

Fortheproblem

ofExample10.4we�ndkI�HAk=

1:261>1,sowecannotuseKrawczyk'smethod.IfwereplaceAandbby

b A=m(A)+0:1r(A)=

0 @[2:9;3:1]

[0:9;1:1]

[�0:1;0:1]

1

[3:45;3:55]

[0:4;0:6]

[4:9;5:1]

[3:9;4:1]

[9:175;9:325]1 A ;

b b=m(b)+0:1r(b)=� [8:9;9:1];[10:9;11:1];[8:9;9:1]� ;

thenthemethodworks.WegetkI�HAk=0:1261<1,andwith

X(0)

=

0 @[1:7415;2:4394]

[2:4978;3:0292]

[�1:6906;�1:0396]1 A ;

obtainedbytheintervalversionof(10.3),andx(0)=m(X(0))weget

K(x(0);X(0))=

0 @[1:7429;2:4319]

[2:5204;2:9552]

[�1:5919;�1:0867]1 A ;

X(1)

=

X(0)\K(x(0);X(0))=

0 @[1:7429;2:4319]

[2:5204;2:9552]

[�1:5919;�1:0867]1 A ;

andproceedinglikethiswiththestoppingcriterion

d(X(s);X(s�

1))�10�

12

;

(10.4)

westopwith

X(12)

=

0 @[1:7519;2:4229]

[2:5271;2:9484]

[�1:5854;�1:0931]1 A :

ThisisbetterthanX(0),butstillquitefarfrom

thetruesolution,b X=

([1:7878;2:3937];[2:5479;2:9311];[�1:5783;�1:1314]).

IntheTablebelowwegiveresultsfortheproblemsgivenby

b A=m(A)+ r(A);

b b=m(b)+ r(b);

fordecreasingvaluesof .Foreachproblem

thevectorX(0)

istheresult

from

theintervalversionof(10.3),weusex(s)=m(X(s)),anditsisthe

numberofiterationstepsbefore(10.4)issatis�ed.

67

10.LinearSystems

kI�H

b Akits

d(Xits;b X)

10�

1

1.26e-01

12

3.83e-02

10�

2

1.26e-02

6

3.71e-04

10�

3

1.26e-03

4

3.70e-06

10�

4

1.26e-04

3

3.70e-08

10�

5

1.26e-05

3

3.70e-10

10�

6

1.26e-06

2

3.71e-12

10�

7

1.26e-07

2

3.86e-14

10�

8

1.26e-08

2

2.66e-15

10�

9

1.26e-09

2

2.66e-15

Notehoweachreductionofthewidthofthematrixandright-handside

byafactor10reducesthedistancebetweentheKrawczyksolutionand

thetruesolutionbyafactor100.Thelastthreeresultsforthedistance

areincreasinglya�ectedbyroundingerrors.

ComparedwiththediscussioninSection9.4theresultsinthelast

twoexamplesaresomewhatdisappointing.

However,weintroduced

Krawczyk'smethodforproblems,wherethetruncationwasthedom-

inatingerrorsource.

TheinitialerrorwassuÆcientlysmallforthe

methodtoconverge,anditwasreducedwiththewidthoftheinter-

val.Now,weapplythemethodonproblemswithinputerrors,thatstay

�xedthroughouttheiteration.Example10.6illustratesthatthemethod

performswellforproblemswithsmallinputerrors.Thiswasalsothe

conclusionofWongwises(1975),whousedthemethodtocomputeerror

boundsforcomputedsolutionstoproblemsoftheform(10.1),i.e.she

usedthemethodtoestimatethee�ectofroundingerrors.

10.2.

Sign-AccordAlgorithm

ThisalgorithmwaspresentedbyRohn(1989)andiscurrentlyconsidered

asthebestmethodforsolvingsystemsoflinearintervalequations.Our

presentationisbasedonMadsenandToft(1994).First,letX=X(A;b)

denotethesetofsolutionsto(10.2),i.e.

X=

� x2Rn� �� Ax=� bforreal� A2A;� b2b :

(10.5)

Figure10.1illustratesthatthissetisnotconvex.Thispropertyalso

followsfrom

thefollowing,surprisinglysimplecharacterizationofthe

solutionset,

10.2.Sign-AccordAlgorithm

68

Theorem

10.1.

LetA2I(Rn�

n),b2I(Rn).Thesolutionsetof

AX

=bis X

=

� x2Rn� � jm(A)x�m(b)j�r(A)jxj+r(b)

:(10.6)

Proof.

SeeOettliandPrager(1964).

Thenonconvexityfollowsfrom

thefactorjxj.Supposethatu;v2X,

i.e.eachofthem

satisfy(10.6),butitdoesnotfollowthatanylinear

combinationofthemase.g.x=u+vsatis�esit.

AmongalltheelementsinXthecornersolutionshavespecialinterest.

ThesubsetXCornconsistsofthevectors� A�

1� b,whereallelementsin� A

and� barechosenasoneoftheendpointsofthecorrespondinginterval

elementinA

andb,respectively.Thenumberofcornersolutionsis

2n2+n.

Example10.7.

Fortheproblem

ofExample10.1wehavecomputed250

elementsofX.64ofthesearethecornersolutions,markedby`+"in

Figure10.2below,andtheremaining186correspondtorandom

choices

of� Ai;j

intheintervalAandsimilarfor� b.

0.4

6

−4

0.9

Figure10.2.

69

10.LinearSystems

ThedottedlinebordersConv(X),theconvexhullofX.Notethateach

cornerofthispolygonisacornersolutionofthesystem.

Theobservationinthisexampleistrueingeneral:

Theorem

10.2.

LetA2I(Rn�

n)beregularandletb2I(Rn).

Then

Conv(XCorn)=

Conv(X):

(10.7)

Proof.

SeeToft(1992).

ThesolutionX

tothesystem(10.2)isanintervalvector,andasalready

mentioned,X

istheintervalhullofX,X=X.

Theorem

10.3.

LetA2I(Rn�

n)beregularandletb2I(Rn).

ThesolutiontoAX

=bis

X

=

XCorn:

(10.8)

Proof.

Theinclusions

X�Conv(X)�X

areobvious,and(10.8)followsfromX=Xand(10.7).

ThistheoremwasoriginallyprovedinBeeck(1972).Itshowsthatthe

problem

is�nite:wecan�ndX

viathecomputationofthecorner

solutions.Eachoftheseisfoundbysolvingarealsystem

ofordern,

andthecomplexityofthisapproachisO(n32n2+n),whichisprohibitive

evenforsmallvaluesofn,

10.2.Sign-AccordAlgorithm

70

n

2

5

10

n32n2+n

5:12�102

1:34�1011

1:30�1036

Wecandomuchbetter.Thepointswheretheinequalityin(10.6)turns

intoequalityarethesocalledextremesolutions,

XExtr

=

� x2Rn� � jm(A)x�m(b)j=r(A)jxj+r(b)

:(10.9)

Inconnectionwith(10.14)belowweshowthatXExtr�XCorn.Wein-

troducethesignvector

y=

sgn

� m(A)x�m(b)� :

ThisisanelementinSn,

Sn=

� y2Rn� � y i2f1;�1g;i=1;:::;ng:

Now,(10.9)canbereformulatedto

XExtr

=

� x2Rn� � m(A)x�m(b)=Dy(r(A)jxj+r(b))

forsomey2Sn

;

(10.10)

wherewehaveintroducedthediagonalmatrix

Dy

=

diag(y1;:::;yn):

(10.11)

ThenexttwotheoremsofRohn(1989)showthatwecanreducethe

problem

from

�ndingthe2n2+n

elementsinXCorn

to�ndingthe2n

elementsinXExtr.

Theorem

10.4.

LetA2I(Rn�

n)beregularandletb2I(Rn).

Foreachy2Snthenonlinearequationin(10.10)hasexactlyone

extremesolutionx=xy,and

Conv(XExtr)=

Conv

� x y� � y2Sn

=Conv(X):(10.12)

71

10.LinearSystems

Theorem

10.5.

LetA2I(Rn�

n)beregularandletb2I(Rn).

ThenthesetXExtrofextremesolutionsforAx=bsatis�es

XExtr

=

X:

(10.13)

Example10.8.

Fortheproblem

ofExamples10.1,10.2,10.5and10.7the

extremesolutionsare

y

(1;1)

(1;�1)

(�1;1)

(�1;�1)

xy

(20 9

;4 9)

(6;�4)

(0:4;0:8)

(1415;�0:2)

ThefxygarethecornerpointsofthepolygoninFigure10.2.

Thereductioninthenumberofpointsisnotforfree.Eachpointin

XExtristhesolutiontoanonlinearsystemofequations.From(10.10)

weseethatforagiveny2Snwehavetosolve

m(A)x�m(b)=

Dy

� r(A)jxj+r(b)� :

Introducingz2Snde�nedbyz=sgn(x)(i.e.jxj=Dzx)weseethat

thissystemisequivalentwith

Axyx

=

b y

whereAxy=m(A)�Dyr(A)Dx;b y=m(b)+Dyr(b):

(10.14)

ItisseenthatallelementsinAxy

andb yareatanendpointofthe

correspondingelementinAandb,respectively,sothesolutionisacorner

solution,andwehaveveri�ed(10.9).Further,ifthesolutionxsatis�es

sgn(x)=z,thenx=xy.

Now,wearereadytoformulatethealgorithmforcomputingthesolution

toAX

=b. X

:=;

foreachy2Sndo

�ndtheextremesolutionxy

X

:=X[fxyg

(10.15)

10.2.Sign-AccordAlgorithm

72

Theextremesolutionisfoundbythesign-accordalgorithm

duetoRohn

(1989),

giveny2Sn

chooseinitialz2Sn;

fd:=false

repeat

solveAxyx=b y

if

allz ixi�0then

xy:=x;

fd:=true

else

k:=argminjfzjxj<0g

z k:=�zk

end

untilfd

(10.16)

Thestoppingcriterionisthatthesignofxisinaccordancewithz.Rohn

(1989)showedthatthealgorithmstopsafteratmost2n

iterationsteps,

andrecommendstheinitialchoicez=sgn

� m(A)�1b y

� .

Inthebestcasethe�rstxisinsign-accordancewiththeinitialz,so

thecomplexityofalgorithm(10.15)isbetweenO(n32n)andO(n322n).

WerefertoMadsenandToft(1994)aboutfurtherenhancementsofthe

algorithm,aspectsof oatingpointarithmetic,andhowthealgorithm

canbesuccessfullyimplementedonaparallelcomputer.

11.

GlobalOptimization

Manytechnicalandeconomicproblemscanbeformulatedasmathe-

maticalprogrammingproblems,i.e.astheminimizationofacontinuous

nonlinearfunctionf:D7!R,whereD�Rn.Often,thefunctionfhas

severallocalminima,butweareonlyinterestedinthesmallestone

f�

=

inf� f(x)

� � x2D:

(11.1)

f�

istheglobalminimum

andtheproblemof�ndingf�

andthepoints

X�

=fx2Djf(x)=f�gwhereitisattainediscalledglobaloptimiza-

tion.Methodsforglobaloptimizationcanbedividedintotwoclasses,

deterministicmethodsandstochasticmethods,seee.g.Pint�er(1996).

Weshallpresentadeterministicmethodbasedonintervalanalysis.

11.1.

BasicMethod

Theproblemof�ndingtheglobalminimumiscloselyrelatedtothesub-

jectofChapter7:f�

isequaltothelowerboundoftheintervalhull

f(D),andweshallexpandtheideasoutlinedattheendofChapter7,

combinedwithrelevantalgorithmsfromChapters8and9.Thepresen-

tationisbasedonMadsen(1991)andproofofconvergencemaybefound

inMooreandRatschek(1988).

Thealgorithmisabranchandboundtypemethod.Atanystageofthe

algorithmwehavethecandidatesetC.Thisisa�nitesetofsubregions

C(j)�DwiththesetX�

ofminimizersoffcontainedintheunionof

fC(j)g.Theaimistoreducethecandidateset,andtodothat,letFbe

anintervalextensionoffandnotethat

min jfL(C(j))g�f�

�min jfU(C(j))g��;

(11.2)

wherethefunctionsLandUaretheboundsoftheintervalreturnedby

F,

L(C(j))=

lowerboundofF(C(j));

U(C(j))=

upperboundofF(C(j)):

(11.3)

Therefore,if

L(C(j))>�;

(11.4)

11.1.BasicMethod

74

thenC(j)canbediscardedfromthecandidateset.Otherwise,wecan

getsharperboundsbysplittingC(j)intotwo,smallersubregions,cf.the

algorithmsinChapters7{9.Ifn=1,thenthesplittingisdoneby

simplebisection,andinmultipledimensionswebisectinthedirection

ofthelargestcomponentoftheradiusofC(j).

Nowwearereadytosketchthebasicalgorithm.Wesplitthecandidate

setCintoaworksetSandaresultsetR,sothatX�

isalwayscontained

intheunionofSandR.If

w(F(S(j)))�Æ;

(11.5)

thentheelementS(j)ismovedfromStoR.

Algorithm

GlobalOptimization1

S(1):=D;S:=fS(1)g;�:=U(S(1));

R:=;

while

S6=;

i:=argminjfL(S(j))g

X

:=S(i);

removeS(i)fromS

splitX

intoX(1)andX(2)

^�:=minfU(X(1));U(X(2))g

if^�<�

then

�:=^�;

use(11.4)toreduceS

end

fork=1;2

ifL(X(k))��

then

ifw(F(X(k)))�Æthen

R:=R[X(k)

else

S:=S[X(k)

end

end

end

(11.6)

Example11.1.

Considerthefunctionf(x)=(1�x

2)cos(5x)onD=[0;2]

withtheintervalextensionF(X)=(1�X

2)cos(5X).

InFigure11.1wegiveselectedresultsfrom

theiterationwithAlgorithm

(11.6).kdenotesthenumberofrepeats.

75

11.GlobalOptimization

01

2

−202

k =

1

01

2

−202

k =

2

01

2

−202

k =

3

01

2

−202

k =

4

01

2

−202

k =

5

01

2

−202

k =

6

01

2

−202

k =

7

01

2

−202

k =

8

01

2

−202

k =

26

01

2

−202

k =

27

01

2

−202

k =

28

01

2

−202

k =

29

01

2

−202

k =

30

Figure11.1.

ThestartingpointisS=DwithF(D)=[�3;3].Fork=5theinterval

[1:75;2]satis�es(11.4)andisdiscarded.Fork=9;:::;25computationis

focusedontheregionaroundthetrueminimum,butwestillkeepthelocal

11.2.UseofGradient

76

minimum

around0.6inthecandidateset.Thenthelowerboundaround

theglobalminimum

hasbecomesolargethatthecandidatesaroundthe

localminimum

aresplitanddiscarded.

WithÆ=10�

2

ittakes20furtheriterationstepsbeforethecandidateset

isempty.Theresultlistholds24neighbouringintervalsspanningthe

intervalX

=

[1:3457;1:3926],whichcontainsX�

.Therelation(11.2)

takestheform

�0:7444�f�

��0:7352.

11.2.

UseofGradientInformation

SupposethatwenotonlyhaveaccesstotheintervalextensionF,but

alsotoandintervalextensionF0

ofthegradientf0

=� @f

@x1

;:::;

@f

@xn

� .

Thisinformationcanbeusedintwoways,

1.Monotonicity.If02j

� F0(S(s))� j,thenfismonotoneinthecom-

ponentxj

onthesubdomainS(s).Therefore,wecanreducethejth

componentoftheintervalvectorS(s)toitslower(respectivelyupper)

boundif

� F0(S(s))� j>0(respectively

� F0(S(s))� j<0).Furthermore,if

thisreducedcandidateisinteriortotheoriginaldomainD,thenwecan

discarditfromSbecauseaninteriorminimumisastationarypoint,i.e.

thegradientiszero.

2.Stationarypoints.Asjustmentioned,thegradientataninterior

minimizeriszero.Thismeansthataninteriorminimizerisarootofthe

equationf0(x)=0,andwecanuseoneofthemethodsfromChapter9

tolocatethisminimizer.Especially,ifwealsohaveanimplementationof

theJacobianoff0,i.e.theHessianf00

(oruseautomaticdi�erentiation

tocomputeit),thenwecanuseKrawczyk'sversionofNewton'smethod

(9.11)totryand�ndtheminimizer.Therearethreepossibleresultsfor

theintervalX

=S(i):

i)

Krawczyk'smethodshowsthatX

containsnoroot.IfX

is

interiortoD,thenwecandiscard(S(i))fromthecandidateset,

otherwisewecanreduceittotheunionofitsnon-interioredges.

ii)

Xcontainsaroot,andKrawczyk'smethodisusedto�nditwith

highaccuracy.S(i)canbereplacedbytheresultinginterval.

77

11.GlobalOptimization

iii)

Krawczyk'smethodstallsatX,maybebecausetheintervalis

toowide.Usesimplesplittingtoreducethewidth.

Now,wecanoutlinethealgorithm.Theupdatingofthethreshold�and

thesetsSandRisperformedasinAlgorithm11.6.

Algorithm

GlobalOptimization2

S(1):=D;S:=fS(1)g;�:=U(S(1));

R:=;

while

S6=;

i:=argminjfL(S(j))g

X

:=S(i);

removeS(i)fromS

ifMonotone(X)then

X

isreducedtoRX,see1.above

elseifNewton(X)worksthen

X

isreducedtoRX,seei)andii)above

else sp

lit:RX

:=fX(1);X(2)g

with

X

=X(1)[X(2)

end

useRX

toupdate�andS

withallelementsinRX

discardorstoreinSorR

end

end

(11.7)

Example11.2.

Wehaveappliedthisalgorithm

tothesameproblem

asin

Example11.1andgottheperformanceillustratedinFigure11.2.Astar

signi�esanelementintheResultsetR.

The�rstthreeiterationstepsareidenticalwiththeresultsfrom

Algo-

rithm

11.6,andfork=4the\best"subregionisX

=

[1:25;1:5],and

Krawczyk'smethodgivesustheglobalminimizerwithassociatedthresh-

old�=

�0:73979,whichalsodiscardsthecandidate[1;1:25].There-

mainingiterationstepssuccessivelydiscardtheremainingelementsinthe

candidateset.Theresultlistholdsoneelement,

x�

2X

=[1:36864016023728;1:36864016023729];

andtheminimumis

11.2.UseofGradient

78

01

2

−202

k =

1

01

2

−202

k =

2

01

2

−202

k =

3

01

2

−202

k =

4

01

2

−202

k =

5

01

2

−202

k =

6

01

2

−202

k =

7

01

2

−202

k =

8

01

2

−202

k =

9

01

2

−202

k =

10

Figure11.2.

f�

2F(X)

with

m(F(X))=�0:7397955,r(F(X))=1.55e-15:

Thus,Algorithm

11.7needsfeweriterationsandgivesamuchmoreaccu-

rateresultthanAlgorithm

11.6.

Asmanyotherproblemsglobaloptimizationsu�ersfromthe\curseof

dimensionality",i.e.thecomputationalworkgrowsfastwiththedimen-

sionn.Thereforeitisappropriatetouseparallelcalculationinorderto

decreasereal-time.Madsen(1991)describesaparallelversionofAlgo-

rithm(11.7).

79

References

References

[1]

H.

Beeck

(1972),

� Uber

die

Struktur

und

Absch�atzungen

der

L�osungsmengevonlinearenGleichungssystem

mitIntervalkoeÆzienten,

Computing10,pp231{244.

[2]

G.Corliss,C.Faure,A.Griewank,L.Hasco�etandU.Naumann(2002)

(editors),Automaticdi�erentiationofalgorithms,Springer,NewYork.

[3]

R.Krawczyk(1969),Newton-AlgorithmenzurBestimmungvonNul-

stellenmitFehlerschranken,Computing4,pp187{201.

[4]

K.Madsen(1991),Parallelalgorithmsforglobaloptimization,Report

91-07,IMM.DTU.

[5]

K.MadsenandO.Toft(1994),A

parallelmethodforlinearinterval

equations,IntervalComputations3,pp81{105.

[6]

R.E.Moore(1966),IntervalAnalysis.Prentice-Hall,EnglewoodCli�s,

N.J.

[7]

R.E.Moore(1976),Oncomputingtherangeofvaluesofarational

functionofnvariablesoveraboundedregion,Computing16,pp1{

15.

[8]

R.E.MooreandS.T.Jones(1977),Safestartingregionsforiterative

methods,SIAM

J.Numer.Anal.14,pp1051{1065.

[9]

R.E.MooreandH.Ratschek(1988),Inclusionfunctionsandglobalop-

timizationII,Math.Progr.41(3),Aseriespp341{356.

[10]

A.Neumaier(1990),Intervalmethodsforsystemsofequations,Cam-

bridgeUniversityPress.

[11]

A.Neumaier(2001),IntroductiontoNumericalAnalysis,Cambridge

UniversityPress,Cambridge.

[12]

K.Nickel(1967),DieVollautomatischeBerechnungeinerEinfachen

NullstellevonF(T)=

0einschliesslicheinerFehlerabsch�atzung,Com-

puting2,pp232{245.

[13]

K.Nickel(1977),Interval-Analysis.pp193{226inD.Jacobs(ed),

ProceedingsoftheConferenceontheStateoftheArtinNumerical

AnalysisheldattheUniversityofYork,April12th{15th,1976.

[14]

W.OettliandW.Prager(1964),Compatibilityofapproximatesolution

oflinearequationswithgivenerrorboundsforcoeÆcientsandright-

handsides,Numer.Math.6,pp405{409.

References

80

[15]

J.D.Pint�er(1996),Continuousglobaloptimization:abriefreview,Op-

tima52,pp1{8.Availableathttp://plato.la.asu.edu/gom.html.

[16]

F.N.Ris(1972),Intervalanalysisandapplicationstolinearalgebra,

D.Phil.Thesis,Oxford.

[17]

J.Rohn(1989),Systemsoflinearintervalequations,LinearAlgebra

Appl.126,pp39{78.

[18]

S.Skelboe(1974),Computationofrationalintervalfunctions,BIT14,

pp87{95.

[19]

S.Skelboe(1977),Trueworst-caseanalysisoflinearelectricalcircuits

byintervalarithmetic,ReportIT11,InstituteofCircuitTheoryand

Telecommunications,TechnicalUniversityofDenmark,Lyngby.

[20]

O.Toft(1992),Sequentialandparallelsolutionoflinearintervalequa-

tions,Master'sthesis,InstituteforNumericalAnalysis,TechnicalUni-

versityofDenmark,pp1{98.

[21]

P.Wonqwises(1975),ExperimentelleUntersuchungenzurnumerischen

Aufl�osungvonlinearenGleichungssystemenmitFehlererfassung.

pp

316{325inK.Nickel(ed),"IntervalMathematics",LectureNotesin

ComputerScience29,SpringerVerlag.

81

Notation

Notation

a;a

LowerandupperlimitofintervalA

p.6

jAj

UpperboundofintervalA

(2.1),(2.2)

kAk

NormofintervalA

(2.3),(2.4)

d(A;B)

DistancebetweenintervalsAandB

(5.1),(5.10)

f(X)

IntervalhulloffonX

(4.1)

fM

(X)

Meanvalueform

(6.3)

J(x;y)

Jacobian

p.52

K(x;X)Krawczykoperator

(9.10)

m(A)

MidpointofintervalA

(2.1),(2.2)

Mij

(i;j)thelementintheintervalmatrixM

p.7

N(x;X)Newtonoperator

(9.1)

r(A)

RadiusofintervalA

(2.1),(2.2)

w(A)

WidthofintervalA

(2.1),(2.3),(2.4),p.22

Xi

ithelementintheintervalvectorX

p.6

X(s)

sthelementinasequenceofintervals

pp.33,47,73

X

SetofsolutionstoAx=b

(10.5)

Index

82

Index

adaptivealgorithm,35,39

associativerule,11

automaticdi�erentiation,43,76

bisection,46

branchandbound,73

cancellation,10

candidateset,73

commutativerule,11

compositemapping,19

continuity,19

convergence,19

linear,35

quadratic,35,51,60

convexhull,69

cornersolution,68

curseofdimensionality,79

degenerateinterval,6

distance,16

distributiverule,11

divide-and-conquer,39

existenceofroot,46

extendedrationalfunctions,15

extremesolution,70

oatingpoint,34,54,72

Gaussianelimination,63

gradient,27,36,76

hybridmethod,41,46

inclusionmonotonicity,9,14,28,55

inputerrors,4,37,67

intervalextension,13,24,25,30,33

{function,13

{hull,12,15,21,62,69

intervalmatrix,7

{vector,6

inverseelement,11

Jacobian,52,53,76

Krawczykmethod,57,60,65,76

Lipschitzcontinuity,20

machineaccuracy,54

meanvalueform,27

{theorem,24,27,37,50

metric,16

midpoint,6,27

multipleroots,45,48

nestedsequence,47,57

neutralelement,11

Newton'smethod,47,55,76

norm,7

parallelcomputation,72,79

radius,6

rationalintervalfunction,22

regularmatrix,61

Riemannformulas,40

roundingerrors,3,30,34,37,54,67

saw-toothmethod,60

Simpson'sformula.,40

singleton,6,23,26

stoppingcriterion,35,42,59,66,74

sub-distributivity,11

Taylorexpansion,29

truncationerrors,4,37,38,67

width,6,22