Optimizing QoS Routing in Hierarchical

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART C: APPLICATIONS AND REVIEWS, VOL. 33, NO. 3, AUGUST 2003 297

Optimizing QoS Routing in HierarchicalATM Networks Using Computational

Intelligence TechniquesAthanasios Vasilakos, M. P. Saltouros, A. F. Atlassis, and Witold Pedrycz, Fellow, IEEE

Abstract—In this paper, the use of a computational intelligenceapproach —a Reinforcement Learning Algorithm (RLA)—foroptimizing the routing in asynchronous transfer mode(ATM)networks based on the private network-to-network interface(PNNI) standard is proposed. This algorithm which is speciallydesigned for the quality of service (QoS) routing problem, aimsat maximizing the network revenue (allocating efficiently thenetwork resources) while ensuring the QoS requirements for eachconnection. In this study, large-scale networks are consideredwhere it becomes necessary to be organized hierarchically so thata scale in terms of computation, communication and storage re-quirements will be achieved. A comparative performance study ofthe proposed and other commonly used routing schemes is demon-strated by means of simulation on existing commercial networks.Simulation results over a wide range of uniform, time-varying andskewed loading conditions show the effectiveness of the proposedrouting algorithm, and disclose the strength and weakness of thevarious schemes.

Index Terms—ATM QoS routing, computational intelligence, re-inforcement learning algorithm.

I. INTRODUCTION

T HE implementation of the emerging high-speed commu-nication networks, such as broadband ISDN, has been

proposed by the ITU-T [1] to be based on the ATM technology.In ATM networks, various kind of sources (such as voice,video, or data) with different traffic features andquality ofservice (QoS) requirements—such ascell loss ratio (CLR),maximum cell transfer delay(maxctd) andcell delay varia-tion (CDV)—are statistically multiplexed at very high rates.However, the integrated character of ATM networks causes anumber of serious congestion problems characterized by celllosses and excessive delays making the statistical multiplexingineffective without control.

In ATM networks, the QoS of a connection is closely linkedto the allocated network resources. A connection request for a

Manuscript received August 31, 2002; revised April 13, 2003 and June 17,2003. This paper was recommended by Guest Editor S. Karnouskos.

A. Vasilakos is with the Institute of Computer Science, Foundation forResearch and Technology—HELLAS (FORTH), Crete 15410, Greece (e-mail:[email protected]).

M. P. Saltouros and A. F. Atlassis are with the National Technical Univer-sity of Athens, Electrical and Computer Engineering Department, Heroon Poly-techniou 9, GR-157 73, Athens, Greece (e-mail: [email protected]. gr;[email protected]. gr; [email protected]. gr).

W. Pedrycz is with the Department of Electrical and Computer Engi-neering, University of Alberta, Edmonton, AB, T6G 2G7, Canada (e-mail:[email protected]. ca).

Digital Object Identifier 10.1109/TSMCC.2003.817354

given call is accepted only when sufficient resources are avail-able to carry the new connection through the entire network at itsrequested QoS while maintaining the agreed QoS of the alreadyestablished connections. It is theconnection admission control(CAC) which is responsible for deciding whether a new con-nection request should be accepted or not. To operate efficiently,CAC takes into account the connection traffic descriptor and therequested QoS for both the incoming call and the already estab-lished connections, as well as the available network resources.Once the connection has been established, the requested QoSshould be provided as long as the connection is compliant withthe negotiated traffic contract.

Part of the CAC actions is the routing algorithm [1], whichidentifies paths that meet QoS constraints between the source-destination pair of an incoming call, and selects one that leadsto high overall network efficiency. Without an efficient QoSbased routing algorithm, network may fail to find a route andreject a request for the call connection, though there are enoughresources to successfully establish that call. A routing algo-rithm aims at improving network efficiency which is usuallyexpressed in terms of increase inrevenue, where revenue is typi-cally a function of the number flows or the amount of bandwidthcarried by the network, while guaranteeing the QoS constraints.Thus, for a QoS-based routing algorithm, not only is it impor-tant to select a route that satisfies all the QoS requirements, butalso it is significant to manage efficiently the network resources,e.g., buffer space and bandwidth. So, it is clear that an efficientscheme should aim at reducing the consumption of resourcesand at balancing the traffic load across the network when nec-essary. The later property is particularly helpful for maximizingthe ability to accommodate future calls. Furthermore, a high per-formance ATM routing protocol should be dynamic, being ca-pable of selecting the routes in response to variation of the trafficload and network failure, contributing in this way to the betterexploitation of the network resources.

In this paper, a computational intelligence approach tothe ATM flat and hierarchical (PNNI) QoS routing issue ispresented based on theStochastic Estimator Learning Algo-rithm (SELA), an efficient Reinforcement Learning Algorithm(RLA). RLA as simple propabilistic models performs robustlyin the face of uncertainty conducting safety -based routing(inaccurate bandwidth feedback from aggregated topology)and thus, moving from deterministic to propabilistic modelswhen dealing with networking complexities, is encouraged.The learning approach presented in this paper, treats routing

1094-6977/03$17.00 © 2003 IEEE

298 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART C: APPLICATIONS AND REVIEWS, VOL. 33, NO. 3, AUGUST 2003

Fig. 1. Two-level PNNI network hierarchy model.

as a stochastic allocation problem amenable to a dynamic de-centralized approach. Based on real-time feedback of networkstatus, Learning Algorithms (LAs) operating at various networkswitches determine how the traffic is to be routed.

The proposed routing methodology is evaluated by means ofsimulation on existing commercial topologies and simulationresults demonstrate its effectiveness for the cases of connec-tions with specific CLR and maxCTD requirements. This ef-fectiveness is highlighted when compared in terms of averagecell transfer delay, bandwidth rejection ratios, bandwidth ac-ceptance ratios, network throughput, and network revenue withother commonly used routing algorithms over a wide range ofuniform, time-varying and skewed loading conditions. More-over, the proposed algorithm is simple and can be implementedin hardware easily according to the procedures described in [2].

The rest of the paper is structured as follows. Since the routeoptimization methodology is proposed for ATM networks basedon the PNNI specification, at first background information onthe PNNI standards is provided, and prior work on hierarchicalrouting is summarized in Section II. In Section III, we describebriefly how the RLAs operate and justify their use in routingproblems in networks. In Section IV, a concise description of theSELA algorithm is given. The proposed routing methodologybased on SELA is developed in Section V, while in Section VIthe simulation model is described and the extracted numericalresults from the comparative performance study between thevarious routing schemes are exhibited. Finally, Section VII con-cludes the paper and intimates some interesting aspects of futurework.

II. BACKGROUND

In this section, at first a brief overview of the PNNI standardsfor ATM networks is presented, and then prior work on hierar-chical routing is summarized.

A. General Survey of PNNI Standards

PNNIprotocols [3] construct a logical hierarchy atop a phys-ical ATM network. In a global network it is unpractical foreach node to contain a complete description of the entire net-work because the costs of storing and updating this informa-tion are prohibitive. Thus, a hierarchical scheme allows nodesto have reduced “views” of the network (aggregated topologyinformation), since it organizes the network nodes into levels.According to this concept, the network nodes and links are or-ganized hierarchically. At the lowest level of the hierarchy, eachnode represents an ATM switch and each link represents a phys-ical link or an ATM virtual path (VP). The nodes and links ofeach level can be recursively aggregated into higher levels, suchthat a high-level node represents a collection of one or morelower level nodes, and a high-level link represents a collectionof one or more lower level links. The ATM PNNI is a hierar-chical, dynamic link-state routing protocol, designed to scale tothe largest possible ATM networks, encompassing thousands ofswitches. It may support therefore a maximum of 105 hierarchylevels.

To support this hierarchy, PNNI defines a uniform networkmodel at each level, with a set of logical nodes connected bylogical links. In PNNI a collection of nodes at the same hier-archical level, geographically collocated and/or belonging to agroup, is organized into a PNNIpeer group (PG)which is as-signed a uniquepeer group identifier (Pgid). At the lowest levelof hierarchy a PG consists of physical nodes connected via phys-ical links. At the next level of hierarchy the PG consists of a setof logical group nodes (LGNs), which are abstractions of thelower-level peer groups. Two LGNs are interconnected by alog-ical link which corresponds to a logical connection that consistsof one or more links of the lower level in tandem or/and in par-allel. LGNs summarize and advertise reachability informationfor their respective lower-level PGs. The functions associated

VASILAKOS et al.: OPTIMIZING QOS ROUTING IN HIERARCHICAL ATM NETWORKS 299

with an LGN are actually performed by thepeer group leader(PGL). The PGL is determined by means of a distributed elec-tion process, executed by all the LGN’s in the PG.

When neighboring nodes sense an active link they exchange“hello messages” which contain their PGids. If the PGids arealike, they belong to the same PG. After that each node ties itsstate information, including status of its links to its immediateneighbors, to aPNNI topology state element(PTSE), which isflooded through the entire PG. The collection of PTSEs receivedby a node constitute the node’s current view of its PNNI routingdomain. PTSEs are sent both horizontally across a peer group,and downward from a parent LGN to a child PG. Consequently,each node has complete routing information (connectivity andstate) about the nodes and links in its own lowest-level PG, butpartial and approximate routing information about the nodes andlinks that belong to different PGs. Moreover, when neighboringnodes conclude on the basis of exchanged “hello messages” thatthey belong to different PGs, they become border nodes. In otherwords, aborder nodein a peer group is a logical node that hasat least one link that crosses the peer group boundary. Fig. 1shows a schematic of the PNNI network hierarchy model withtwo levels.

In PNNI, the first (originating) switching system that receivesthe connection setup request, routes the incoming call to thedestination (source routing) according to thetopology informa-tion that each one of the candidate switching systems advertises.This information can be the state of the links attached to theseswitching systems (either physical or logical), the QoS param-eters that can be guaranteed, as well as the ability to transmitvarious types of traffic. In other words, the originating switchingsystem determines the path to the destination comprising all thedetail of the hierarchy known to it. This is called ahierarchicallycomplete source route. Such a path is not a fully detailed sourceroute because some portions of it are abstracted as a sequenceof logical group nodes to be traversed. Thus, this source routedoes not hold the details of the path outside the originator’s peergroup and is considered as alogical path. The actual selection ofa physical path outside the source switching system peer grouphas to be made by each border node of peer groups along thelogical path. When the call setup arrives at the border node of apeer group, this node is responsible for selecting a lower levelsource route describing the crossing of that peer group.

There are significant reasons that induce the compressionand the aggregation of the information that is exchanged in apeer-group and is advertised to the higher level (parent PG).First of all, a decrease in the amount of the exchanged infor-mation decreases the resources that are consumed for their dis-tribution, which is very important for WAN. Besides, it is notpractical for every entity of the network (e.g., every node) tomaintain detailed information about the state of all the othersentities. Moreover, there are cases that an internal topology of anetwork should be concealed from nodes outside it for securityreasons. However, the compression and the aggregation of theinformation (which is calledtopology aggregation) should beimplemented in such a manner that the real network topology berepresented satisfactorily so that efficient routing and resourceallocation will be achieved.

The problem regards what information each switching systemshould advertise and also how often this information shouldbe distributed to the other switching systems. The problem ofhaving fast updating so that each switching system will haveup-to-date information about all the other switching systemsis particularly difficult due to the large propagation delay thatlarge-scale ATM networks introduce. An increase in the fre-quency of updating and in the amount of the exchanged informa-tion increases the probability of choosing a more suitable route,but it also increases both the processing complexity and the re-sources that are consumed, and vice versa.

Several schemes for topology aggregation have been pro-posed; the best known are thestar, the full-mesh, and thesimple-nodeapproaches. The star approach, which has beenadopted by PNNI [3], is an efficient compromise betweenthe other two. Full-mesh approach uses a full graph for therepresentation of the connections between the border nodes thatbelong to a particular peer-group, while simple-node approachrepresents the whole peer-group with a single node. A fullydetailed and a brief description of these topology aggregationschemes can be found in [3] and [5], respectively.

B. Prior Work on Hierarchical Routing

There have been several studies in the literature dealing withhierarchical routing. Some interesting approaches can be foundin [4]–[13]. The authors in [3] study some general QoS pathselection problems when the routing information is inaccuratedue to the aggregation process that occurs in hierarchicallyinterconnected networks. Lee examines some issues of topologyaggregation for hierarchical PNNI routing in ATM networks in[5], while in [6] he proposes a spanning tree method for link stateaggregation in large communication networks. In [7], Awerbuchand Slavitt present a distortion-bounded solution for state aggre-gation of a sub-network. The authors also show how this solutioncan be applied to PNNI. In [8], they give a comparison of theinfluence of some different aggregation schemes on networkperformance, by means of simulation. However, in the modelused only one source-destination pair is considered. In [9] and[10],Barasetal.presentamethodbasedonahierarchical reducedload approximation to evaluate PNNI routing. Yet, no simulationresults are presented. Orda suggests approximation methodsfor the fundamental problem of constrained path optimization(typical of QoS routing) in [11], which exploit the structure ofhierarchical aggregated topologies to give å-optimal solutions.In [12], the same author proposes routing algorithms that basetheir decision on a probability vector (probabilistic routing) fornetworks characterized by uncertain parameters. Kimet al.sug-gest a scalable inter-domain routing scheme which addresses theissueof selectinga path with multiple metrics for building QoS ina homogeneous high speed WAN in [13]. The route computationload imposed by the PNNI routing scheme is investigated andproved to be unevenly distributed among the network nodes in[14]. Finally, the authors in [15] evaluate the performance oftopology aggregation techniques applied in hierarchical routingand investigate the interaction of this topology aggregation withfactors such as routing update frequency, routing algorithms, andnetwork configuration.


III. REINFORCEMENTLEARNING ALGORITHMS AND THEIR

APPLICATION TO THEROUTING PROBLEM IN NETWORKS

An RLA is defined by the sextuple .number of possible actions;action ;

: set of the actionsoffered by the environment;action selected at time;action that has the highest mean reward;input set of possible environmental responses,

;environmental response at time, ;set of possible internal states of the LA;Pn : choice proba-bility;

: pmf over the set of ac-tions;updating scheme that modifiesat each iteration byusing the response of the environment:

;: output function;

;;

pdf ;;

;(prime) implies an estimate;mean of the last environmental responses to;learning window: a predefined small integer;

;: internal state of the automa-

tion;resolution parameter (a constant, internal to the LA);

(a constant);The RLA is a sequential machine characterized by the

following:

1) set of actions;2) an action probability distribution;3) a reinforcement learning scheme.It is connected in a feedback loop to the random environment

(the function to be optimized, the process to be controlled,etc.,).At every sampling instant (see Fig. 2) LA chooses an actionfrom a finite set of actions on the basis of a probability distri-bution. The selected action causes a response from the environ-ment, which in turn is the input signal for the LA. With a rein-forcement scheme, the RLA recursively updates its probabilitydistribution, and modifies its state by means of a transition func-tion, trying to learn the optimal action the environment offers.The objective in the design of the LA is to determine how thechoice of the action at any stage should be guided by past actionsand environment responses. The important point to note hereis that the decisions must be made with very little knowledgeconcerning the nature of the environment. The latter may havetime-varying characteristics. Alternately, the uncertainty may bedue to the fact that the output of the environment is influencedby the actions of other agents unknown to the decision maker.

Fig. 2. RLAs that interacts with a stochastic environment.

In all cases the LA must be designed to improve some overallperformance function. The decision maker, in such a feedbackconfiguration of decision maker (or LA) and environment, is re-ferred to as alearning automaton.

The power of the LA’s approach is best realized in complexsystems when several automata operate in a distributed fashionwith each LA having a small number of actions. Such situa-tions arise frequently in large-scale systems, where practicalconstraints make distributed decentralized control mandatory.Traffic routing in communication networks is an example ofsuch systems where the RLAs can operate at various networkswitches determining how the traffic is to be routed. Besides, theneed for learning in identification or control problems dependson the prior information that is available regarding the system,the characteristics of the noise present, and the constraints thatexist on the inputs. For low levels of uncertainty, learning maynot be the most effective approach. But for highly uncertain sys-tems it may be essential for adequate system performance.

Consequently, the effectiveness of learning routing tech-niques on telecommunication networks is justified by the factthat the states of these networks, which vary with time, areregarded as nonstationary environments and the informationconcerning these states at a particular instant may no longerbe valid at the subsequent instant. Moreover, in large-scalenetworks it is impractical for any single entity (i.e., nodes andlinks) to have access to detailed state information about allthe other network entities. This leads to the need of topologyaggregation for reasons of scaling in terms of computation,communication and storage requirements, but also for securityreasons since the internal topology of a network may haveto be hidden. As a result, the selection of a route acrosssuch large networks will typically be performed based onpartial or inaccurate information derived from the process oftopology aggregation performed. In general, the accuracy ofthe provided state information decreases as the amount of theaggregation that is carried out increases. This inaccuracy canlead to wrong routing decisions [4]. Learning algorithms tech-niques, choosing a path probabilistically, attempt to minimizethe effects of inaccuracies on decision making. The greatestpotential of a RLA’s methodology is that it permits the analysisof very complex dynamic systems and enables an optimumglobal operation. Even in the case of an incomplete access tothe state information of the environment stochastic policies canyield higher performance than deterministic policies and tendto stabilize a nonstationary system to predict its behavior [16].


IV. STOCHASTICESTIMATOR LEARNING ALGORITHM

The SELA used in the proposed methodology is a powerfuland flexible ergodic RLA, especially when operating in a non-stationary stochastic environment [17]. Ergodic means that itconverges at the optimal action with a distribution independentof the initial state. SELA has already been successfully used incontrol-related problems in networks, e.g., Routing [18], con-nection admission control (CAC) [19] and UPC [20], [35].

The SELA is defined as where ,, , , and are as defined in section C, and E is the

environment’s estimator defined below.is the set of the n actions

offered by the environment. The action selected at the timeis symbolized as . is the set of the pos-sible environment responses (feedback). The feedback of action

at time instant is symbolized as .We denote by the probability vectorof choosing each action, i.e., is the probability of choosingaction . The estimator contains at any time instantthe estimated environment characteristics. We define:

whereis the true estimate vector. The true estimateof the selectedaction is the mean reward which this action received the last

times that it was selected. It is computed as

(1)

where is the total reward received by the LearningAlgorithm during the last times that action was selected.The parameter (where is an in-teger internal LA parameter called “learning window” and isused for ignoring old -and probably invalid- environmental re-sponses.

is the Oldness Vector;of action at any time instant is a nonnegative integer

number which expresses the time passed (counted in number ofiterations) from the last time that action was selected.

is the stochastic estimatorvector. the stochastic estimate of action is defined as

(2)

where symbolizes a random number selected witha normal probability distribution, with mean equal to 0 andstandard deviation proportional tothe time passed from the last time each action was selected.Specifically “a” is an internal LA parameter that determines howrapidly the Stochastic Estimates become independent from theTrue Estimates, while is the maximum permitted standarddeviation of the stochastic estimates, that bounds the standarddeviation.

A. SELA Algorithm

Initialization : Set Pi = 1=n and di(t) = mi(t) =

ui(t) = 0, 8i 2 f1; 2; . . . ; ng.

Step 1: Select an action a(t) = ak according to

the probability vector.

Step 2: Receive the feedback bk(t) of action akfrom the environment.

Step 3: Compute the new True Estimate dk(t) of

the selected action ak according to (1) .

Step 4: Update the Oldness Vector by setting

mk(t) = 0 and mi(t) = mi(t� 1) + 1 8i 6= k.

Step 5: Compute the new Stochastic Estimate

ui(t)8i according to (2) .

Step 6: Sort the set of the n actions in in-

creasing order of their stochastic estimate

of mean reward so that the first element of

that classification (“1st-optimal” action am)

will be the one with the highest value and the

last one (“ nth-optimal” action ar) that with

the lowest value. Thus, um(t) = maxfui(t)g and

ur(t) = minfui(t)g.

Step 7: Update the probability vector in

the following way: For every action ai(i =

1; 2; . . .m � 1;m + 1; . . . ; n) with Pi(t) > 0 set:

Pi(t + 1) = Pi(t) � 1=N where N is a parameter

called “resolution parameter” and determines

the step size �(� = 1=N) of the probability

updating. For the “optimal” action am set:

Pm(t + 1) = 1 �i 6=m

Pi(t + 1), with 0 � Pi � 1 andn

i=1Pi = 1.

Step 8: Go to step 1. A complete formal de-

scription of SELA can be found in [17] . It

should be noted that the choice of LA’s pa-

rameters (a; �max; N) is a critical issue, rel-

ative to the LA’s performance under various

switching environments. An RLA is called

" � optimal [21] , if there is an internal pa-

rameter N such that: lim(limN!1EfPm (t)g) = 1.

(The symbolism E f� � �g stands for the expected

value). We have:

Theorem: The SELA RLA is -optimal [21] in every sto-chastic environment that offers symmetrically distributed noise.Let be the mean rewards offered by the envi-ronment to the actions , , respectively. If action

is the optimal one ( for )and , then for every value

of the resolution parameter there is a time instantsuch that for every it holds that .

The proof of the theorem is given in [34] and also for com-pleteness and the reader convenience in Appendix. The assump-tion of symmetrically distributed noise is not unreasonable. Thenoise of all known stochastic environments is symmetricallydistributed about the mean rewards of the actions. The-op-timality of the algorithm implies that in case that there is anoptimal action, SELA converge to it. Thus, corresponding eachpossible route of a single source-destination pair to a SELA ac-tion, SELA converge to the optimal action and thus, to the op-timal route. SELA, is able to be adapted dynamically in changesthat take place to the environment; thus, it succeeds in con-verging to the optimal action irrespective of the traffic or/andnetwork changes. This results in choosing each time the optimalroute the environment offers.


V. ROUTING SCHEME BASED ON SELA

In this section a QoS-based, dynamic source routing algo-rithm is presented, in compliance with the PNNI routing pro-tocol, aiming at an efficient resource allocation and at a max-imization of the network revenue. In particular, each sourcenode maintains a topology database consisting of a collectionof all PTSE’s received from its neighbors that represents thatnode’s present view of the PNNI routing domain. Moreover,as it has been shown in several studies dealing with nonhierar-chical (flat) QoS routing (e.g., [22]), efficient resource exploita-tion can be realized when routing is restricted to short paths,even though there are longer candidate paths that may satisfythe QoS requirements. Hence, in this study such a restrictionon candidate paths is applied deliberately. Specifically, at eachsource node a routing table (database) of pre-computed hierar-chically complete k-shortest path routes connecting this sourcenode to every possible destination node is constructed and main-tained. This routing table needs to be updated whenever the net-work topology is modified. One of the algorithms described in[23] can be used off-line to compute the k-shortest path (min-imum-hop) routes for each source-destination pair. Upon re-ceiving a request to establish a new connection with specifictraffic parameters and QoS requirements, the source node se-lects one of the pre-computed k-shortest path routes that can beaccepted by the CAC using its path selection routing algorithm.If no such a route exists, then the incoming call is rejected.

Below the definition of the SELA actions, the computation ofthe environmental feedback, and finally the description of theSELA routing algorithm are given.

A. Definition of the Actions of the SELA

In this study a dynamic decentralized learning approach isused where one SELA LA operates at each source node, de-termining how each incoming call is to be routed from thisnode to every possible destination node. The SELA algorithmuses k actions that represent the k-shortest path (minimum-hop)routes for each source-destination pair. k depends on the net-work topology size and the relative traffic flows.

B. Environmental Feedback Received by SELA

In the proposed methodology, the utilization of the links (ei-ther logical or physical) are used in order to compute the feed-back of SELA. Therefore, the algorithm requires that the nodesretain a current link utilization table (database) for the entirenetwork in their level of hierarchy. It is also reminded that pathselection is performed at the source node as well as at each entryborder node. Hence, the feedback of the algorithm at the sourcenode is computed using the actual metrics inside the source peergroup and the aggregated metrics outside the source peer group.On the other hand, the feedback of the algorithm at each entryborder node is derived from the actual metrics inside its peergroup.

Furthermore, in order to enhance the performance of theSELA routing algorithm in heavy loading conditions -whereuncontrolled alternate routing can lead to increased callblocking rate [18] we developed it to include the concept oftrunk reservationwhich is a technique suitable for achieving

this objective. Its effectiveness is based on the fact that at heavytraffic load, where the saving of network resources is very crit-ical, the highly utilized links are used only for minimum-hoproutes resulting in even better usage of the network resourcesand so, in increased throughput. When the expected routeutilization (which is defined as the maximum expectedutilization of the links that comprise this route) is greater than apredefined utilization threshold , the call is established atthis route only if it is a minimum-hop one. Otherwise, anotherroute is tried. It is noted that the expected utilization of thelinks that comprise a route is defined as the utilization the linkswill have assuming that the route will be accepted by the CACand accommodate the connection request.

Thus, if n links comprise the route that corresponds to theaction ak chosen by SELA, and is the expected utilizationof the th link, the feedback of the LA for this action iscomputed as

(3)

where

ifotherwise

and MAX_NO_OF_HOPS is the maximum number of hops thata route between any source-destination pair can have.

It should be noted that the first term on the right hand side ofthe (3) represents a constant, predefined, maximum theoreticalbound for the sum of the utilization of the links that a route be-tween any source-destination pair can have. This term indicatesthe maximum feedback that a route can take, which occurs whenall the links that comprise the specific route are not utilized atall .

Equation (3) indicates that as the sum increases, which meansthat the previously selected route becomes congested, the feed-back of SELA deteriorates; such operation seems to be quitenatural. Moreover, this sum also increases implicitly when thenumber of the hops of the route increases, which implies an al-ternate (not a minimum-hop) route. Thus, the algorithm gen-erally favors the shortest path routes. It favors alternate routesonly when the congestion level of the shortest routes is muchgreater than the one of the alternate routes. In addition to this,when an alternate route is selected, its corresponding environ-mental feedback strongly deteriorates if that route consists oflinks that have expected utilization greater than the trunk reser-vation threshold . In that way the over-utilization of the al-ternate routes is avoided when the network becomes congested(inherent characteristic of the trunk reservation mechanism).

C. SELA Routing Algorithm

The SELA routing algorithm is summarized as follows:Offline operation:

At each source node:

1) Find the k-shortest (minimum-hop) routes

for each source-destination pair using one

of the algorithms described in [23] and


inform the routing tables (databases) at

these nodes.

2) Correspond each candidate route to one of

the k actions of SELA.

Online operation:

1) Whenever there is a new call-request at

the source node, estimate the expected

utilization of the links of the candidate

routes.

2) Run the SELA algorithm computing the envi-

ronmental feedback.

3) Select the 1st-optimal action.

4) IF that action corresponds to a shortest

(minimum-hop) route AND this route is ac-

cepted by the CAC, THEN choose that action

as the selected one and establish the call

over the corresponding route.

5) ELSE IF that action corresponds to an al-

ternate route which is accepted by the CAC

AND whose utilization �routeexp � �TRT , THEN

choose that action as the selected one and

establish the call over the corresponding

route.

6) ELSE IF nothing of the above stands, THEN

repeat Step 3 concerning the next optimal

action.

7) IF all actions have been tried without any

of them being chosen as the selected one,

THEN the call request is rejected.

D. Properties of the SELA Routing Algorithm

Excellent Fault—Tolerance:The estimator considers the his-tory of the link. Misleading network feedback doew not seri-ously affect the SELA Routing performance. SELA routing is asimple probabilistic model that performs robustly in the face ofuncertainty conducting safety-based routing (inaccurate band-width feedback from aggregated topology).

a) Delayed feedback.This is a fundamental problem inadapting routing. Cases with delayed feedback were sim-ulated using dynamic environment . Withthe same conditions of the experiments of Tables XI andXII and with delayed feedback SELA routingachieves 7.6% less while all other algorithms more than15% performance degradation.

b) Node/link failure.If a node/link fails then, the networkresets; i.e., the shortesst path (minimum—hop) routing[23] is called and finds the new minimum—hop paths.The SELA restarts and, due to its high speed of conver-gence, adapt, to the new topology quickly. The same pro-cedure can be followed when the topology is changed(new nodes/links are added). Because topology changescan be interpreted (i.e., changing the offered load), wesimulated using time—varying (burst) traffic. The min-imum—hop algorithm must react quickly. However, min-imum—hop algorithms need a minimum time and theminimum—hop algorithm to be chosed is not very cru-cial because the faster the minimum—hop algorithm, the

higher the complexity. So, the time needed for the min-imum—hop to recompute the shortest paths is not verycrucial in SELA routing compared with the time SELAuses to find the best routes again.

Small Computational Overhead:A very important feature ofSELA routing is the very small computational overhead it has.

This is due to the

1) minimal amount of feedback information the SELAneeds;

2) simple updating procedures;3) control messages are much fewer than classic routing

schemes.LA are amenable to simple hardware implementaiton because

basic stochastic computing elements have long proved their suc-cees in synthezing LA [2].

VI. SIMULATOR DESIGN

The simulator was written in C++ programming language andexecuted on an Ultra Spark Sun workstation. The simulationruns were carried out at cell level. The confidence intervals ofour measurements were 95% constructed with the method ofindependent replications. The Welch’s procedure was used todetermine the warm-up (initial transient) period so as to dis-card possible misleading data. The length of the runs was es-timated using the replication/deletion approach [24] in order togather reliable statistics. All the routing algorithms were per-formed using the same stream of random numbers (used for thegeneration of call requests), so that a fair comparison among thealgorithms can be achieved, because, in this way any differencesregarding the calls arrival pattern are eliminated.

The candidate paths that a source node or an entry bordernode can choose belong to the minimum-hoprestricted can-didate paths set. It is clarified that the minimum-hop setcontains the minimum-hop paths, and additionally these pathswhich are only one and two hops longer. This is desirable be-cause using a longer path to route a connection ties up resourcesat more intermediate nodes, and subsequently decreases the net-work revenue, especially at heavy traffic load [25].

A. Network Model

Simulation runs for flat and hierarchical routing were per-formed on the vBNS (very high performance backbone networkservice) and NSFnet backbone topologies (existing commercialnetworks) respectively shown in Figs. 3 and 4. For hierarchicalrouting the NSFnet network was arbitrarily divided into fourpeer-groups, each with three or four switches, corresponding inthis way to a two level hierarchical network. The star topologyaggregation scheme was adopted and performed. Distances be-tween two neighboring switches correspond to 1500 Km. As-suming high-speed networks, where the propagation delay de-pends on the distance between both endpoints, we choose thepropagation speed to be two thirds the speed of light. The net-works parameters are summarized in Table I.

B. Topology Aggregation

At first the aggregated (logical) links between two adjacentpeer groups are defined asinter-group links, while the logical


Fig. 3. NSFnet simulated network which consists of 14 nodes, 22bi-directional links, average degree 3, and is divided into four PGs for atwo-level hierarchical routing.

Fig. 4. Commercial VBNS backbone network that runs on a synchronousoptical network (Sonet) at the OC-12 rate of 622.08 Mb/s and interconnects12 points of presence (POPs).

TABLE INETWORK PARAMETERS

links between every two border nodes in the same peer groupare defined asintra-group links. In general, the topology aggre-gation consists of the aggregation of nodal and link information.

In this simulation study, the aggregation of links hop-count,maxCTD (topology metrics), and allocated bandwidth(topology attribute) are considered. In particular, to accommo-date traversing a logical node as well as routing to the “inside”of the node, the symmetricstar topology aggregation schemewith a uniform “radius” is adopted and applied (default noderepresentation in the PNNI complex node representation). Thecenter of the star is the interior reference point of the logicalnode, and is referred to as the nucleus. The logical connectivitybetween the nucleus and a port of the logical node is referred toas a spoke. For each topology state parameter associated witha logical node, a “radius” is derived from the “diameter” ofthe logical node. For a topology metric, the “radius” is simplyhalf the “diameter.” For a topology attribute, the “radius” is thesame as the “diameter.” For each topology state parameter a“radius” has to be advertised to be used as the default value for

TABLE IITRAFFIC CHARACTERISTICS

the spokes. Specifically, in this simulation study the “diameter”of a logical node for the hop-count, and the maxCTD metricsis defined as the maximum number of hops and the maximumend-to-end delay of the corresponding candidate low levelpaths constituting all the intra-group links within the same peergroup. It is noted that a logical intra-group link may correspondto a number of low level (physical) paths connecting the samesource and destination border nodes. As far as the “diameter” ofa logical node for the allocated bandwidth topology attribute isconcerned, we regarded it as the maximum allocated bandwidthof the links that constitute the corresponding candidate lowlevel paths of all the intra-group links within the group.

Similarly, an inter-group link may correspond to several phys-ical links between two adjoining peer groups and consequentlyits aggregated allocated bandwidth and maxCTD can be definedin the same manner.

C. Triggered Updates of Topology State Information

PNNI topology state information which has not changedand has not been updated must be re-originated periodicallyto prevent it from aging out of the network. Re-origination ofthe PTSE is done using a timer-based update triggering policysince it is easier to control update overhead. We implemented asimple version of flooding in our simulator: each node updateits topology state information at regular time intervals (defaultvalue of 5 s) and sends it to all its neighbors. When receivinga state information update, a node updates its database andforward the information to all its neighbors, except the onefrom which the update was received. Duplicate routing updatesare discarded.

D. Traffic Model

As far as the traffic model is concerned, real-time VariableBit Rate (rt-VBR) traffic was used. Specifically, bursty (ON-OFF)sources were assumed with exponentially distributed burst andsilent periods. So, when the source is active it generates cellsaccording to a Poisson process, while the burst size is assumedto follow an exponential distribution. The duration of the estab-lished calls and the inter-arrival time of new call requests werealso assumed to be exponentially distributed (Poisson arrivalprocess). The traffic parameters of the offered calls are summa-rized in Table II. As far as their QoS requirements are concerned,the CLR was set to 10–6, while the maxCTD constraint valuesranged from 15 ms to 65 ms.

The output buffer management scheme has been imple-mented. With the output buffer scheme, each switch has buffersof finite size for each outgoing link. We assume output queuingbecause it achieves optimal throughput/delay performanceamong the various buffer schemes [26]. The queue memory size


requirements (buffer size) of a system based on fixed lengthpackets depend on the performance requirements of the system(the acceptable cell loss rate, load, delay) and the selectedqueuing principle [27]. Since it has been proven in [27] that aswitching element based on the output queuing principle has avery well controlled behavior up to loads of 80 to 85%, we setthe buffer size for a certain cell loss ratio (CLR) requirement of

. This probability of losing a cell is a consequence of thefact where more cells than a queue in the switch can store willsimultaneously compete for this queue. In ATM switches, thisprobability must be kept within limits to ensure a high semantictransparency.

In our study, we consider uniform, time-varying, and skewedworkloads. Inuniform workload, the same call arrival rate isused for each source-destination pair. In this case, the routing al-gorithms were examined for light, moderate, and heavy loadingconditions being determined by the values of the call arrivalrate. Moreover, the performance of the routing algorithms wastested under changing traffic conditions. In this scenario, afterevery passage of a predefined simulated time period (in seconds)the network load varied by changing the call arrival rates foreach source-destination pair. For the vBNS network the call ar-rival rate was uniformly distributed in the range 0 to 200 calls/swhile for the NSFnet in the range 0 to 530 calls/s. Finally, inskewed workload-applied only to the NSFnet- some switchesare selected as the destination of the majority of the calls andconsequently are regarded as “hot-spots.” Specifically, for eachsource-destination pair the calls/s call arrival rate wasused, except for the pairs where the “hot-spots” switches werethe destination nodes. In this later case, the calls/s callarrival rate was used. In this simulation study, the switches 1 and13 were selected as the “hot-spots” switches.

E. Connection Admission Control Scheme

The CAC scheme we adopted is the one proposed by Guerinet al. in [28], which is based on the fluid-flow approximationmodel of [29]. In summary, the equivalent capacity of N multi-plexed sources is given by

where and are the mean

idle and burst periods of source, respectively, and isits peak cell rate (PCR). Furthermore, for a buffer sizeand abuffer overflow probability requirement (CLR) smaller than,the factor is given by

Using the above equivalent bandwidth approximation, the ex-pected utilization of the links that constitute the candidate routecan be easily estimated by the relation

where is the link capacity.In order to satisfy the CLR requirements in a single node

in case of a new call set-up request, the CAC accepts the new

candidate connection only if . Otherwise, theCAC rejects the candidate connection.

As far as the maxCTD requirements are concerned, when aconnection is routed over a path which consists of linkswith link delay , the end-to-end delay is computed bythe following:

(4)

whereIn (4) and are thepropagationand

transmissiondelay respectively at link, while andare theprocessingand thequeuingdelay, respectively

that a cell experiences at the previous switch where linkis at-tached. It is noted that, while the first three of them are con-stant, the queuing delay depends on the queue at the buffer ofthe switch. In this study, for the estimation of the queuing delay,we consider the worst-case where a cell enters a buffer with size

as the th cell and thus, it experiences the maximum possiblequeuing delay. In this case, of (4) is an upper bound ofthe maximum end-to-end delay. Consequently, a candidate newconnection with end-to-end delay constraint will be ac-cepted by the CAC if .

VII. SIMULATION PERFORMANCESTUDY

A. Compared Routing Algorithms

In this section, the routing algorithms that were compared inthe simulation study are briefly described.

SELA-routing Algorithm Described in Section V.Competitive Call Routing (CompCR):Define the cost metric

for link as , where and are the occupied and totalbandwidth for link and is a parameter. Define the cost of apath as the sum of the link costs. Select the minimum cost pathprovided it is feasible (i.e., it satisfies the QoS requirements ofthe specific call request) and its cost does not exceed a threshold. Selecting appropriate values for parametersand turns

out to be nontrivial [30]. In this paper, we set andafter pilot simulation runs to achieve the best possible

performance.Shortest Distance Routing (SDR):Define the cost metric for

link as , where is the bandwidth available on link.Define the cost of a path as the sum of the link costs. Select theminimum cost path provided it is feasible [22].

Least Loaded-Shortest Path Routing (LSR):A feasible path(i.e., a path that satisfies the QoS requirements of the specificcall request) with the minimum hop count (minimum-hop path)is tried. If there are several such paths, the one with the max-imum available bandwidth (least loaded path) is chosen. If sev-eral such paths exist, one of them is randomly chosen. The callrequest is rejected if there is not any minimum-hop path that sat-isfies the QoS requirements.

Shortest-Least Loaded Path Routing (SLR):A feasible pathwith the maximum available bandwidth is tried. If there are sev-eral such paths, the one with the minimum hop count is chosen.If several such paths exist, one of them is randomly chosen. Acall request is rejected only if there is not any feasible path.


TABLE IIISELA’S PARAMETER VALUES

Shortest-Least Loaded Path Routing With Trunk Reservation(TSLR): A shortest-least loaded path routing algorithm thatuses the trunk reservation concept.

Least Loaded-Alternate Path Routing (LALT):A feasibleleast loaded minimum-hop path is tried. If no feasible min-imum-hop path exists, try the least loadedalternate(other thanthe minimum-hop) paths in order of preference (i.e., first thepaths which are one hop longer, then the two hop longer and soon). If several such paths exist, one is randomly chosen. A callrequest is rejected only if there is not any feasible path.

Least Loaded-Alternate Path Routing With Trunk Reservation(TLALT): A least loaded alternate path routing that uses thetrunk reservation concept.

It should be mentioned that the values of the SELA param-eters that were used during the simulation are tabulated inTable III. These parameters were set after pilot simulation runsin order to achieve the best possible performance. Finally, thetrunk reservation threshold was set to 0.81 for all the routingalgorithms which make use of the trunk reservation concept.

B. Performance Metrics

Consider the following notation. Let the network be rep-resented as a capacitated (directed or undirected) graph

with n nodes and m links, where representsthe capacity of the link . Let be the total number ofcall requests and and the number of therejected calls where each call request is described by the tuple

. Let nodes and be the source anddestination of the request, the bandwidth (equivalent ca-pacity) required by the and denote the holding time (i.e.,duration) of the call. We define the performance metrics used inthe evaluation of the routing algorithms as follows: BandwidthRejection Ratio (BRR). It is defined as the percentage of thetotal requested bandwidth that was rejected

Bandwidth Acceptance Ratio (BAR). This metric introducedin [22] and expresses the percentage of the total requested band-width that was routed successfully. Note that this measure ig-nores the duration of the calls and does not take into accountthe length of paths or other measures of efficiency.

Earned network revenue ratio (ENRR). We define the rev-enue for each call to be proportional to its capacity and holding

TABLE IVPERFORMANCE OF THEROUTING ALGORITHMS IN TERMS OF THEEARNED

NETWORK REVENUE RATIOS UNDER VARIOUS UNIFORM LOADING

CONDITIONS FOR THEVBNS-BACKBONE TOPOLOGY

TABLE VPERFORMANCE OF THEROUTING ALGORITHMS IN TERMS OF THEBANDWIDTH

REJECTIONRATIOS UNDER VARIOUS UNIFORM LOADING CONDITIONS

FOR THEVBNS-BACKBONE TOPOLOGY

TABLE VIPERFORMANCE OF THEROUTING ALGORITHMS IN TERMS OF THEEARNED

NETWORK REVENUE RATIOS UNDER TIME-VARYING LOADING CONDITIONS

FOR THEVBNS-BACKBONE TOPOLOGY

TABLE VIIPERFORMANCE OF THEROUTING ALGORITHMS IN TERMS OF THEBANDWIDTH

REJECTIONRATIOS UNDER TIME-VARYING LOADING CONDITIONS FOR THE

VBNS-BACKBONE TOPOLOGY

time (bandwidth-duration product). We introduce the measureENRR as the percentage of the mean total earned network rev-enue described by the following:

Average cell transfer delay (ACTD). It is defined as the av-erage delay to deliver a cell from origin to destination.


TABLE VIIIPERFORMANCE OF THEROUTING ALGORITHMS IN TERMS OF THEAVERAGE CELL TRANSFERDELAY IN SECS (I.E., THE AVERAGE

DELAY REQUIRED TO DELIVER A CELL FROM ORIGIN TO DESTINATION) UNDER VARIOUS UNIFORM LOADING CONDITIONS

FOR THENSFNET-BACKBONE TOPOLOGYCONFIGURED AS A TWOLEVEL HIERARCHICAL NETWORK

TABLE IXPERFORMANCE OF THEROUTING ALGORITHMS IN TERMS OF THEBANDWIDTH ACCEPTANCERATIO (EXPRESSING THENETWORK REVENUE) UNDER

VARIOUS UNIFORM LOADING CONDITIONS FOR THENSFNET-BACKBONE TOPOLOGYCONFIGURED AS A TWOLEVEL HIERARCHICAL NETWORK

TABLE XPERFORMANCE OF THEROUTING ALGORITHMS IN TERMS OF THEACHIEVED THROUGHPUT(EXPRESSED AS THENUMBER OF THESUCCESSFULLYESTABLISHED

CALLS) UNDER VARIOUS UNIFORM LOADING CONDITIONS FOR THENSFNET-BACKBONE TOPOLOGYCONFIGURED AS A TWOLEVEL HIERARCHICAL NETWORK

Network throughput (NT). We consider as a measure of thenetwork throughput the total number of the successfully estab-lished calls

C. Numerical Results

In this section the numerical results obtained from the com-parative simulation performance study among the proposedmethodology and the routing algorithms mentioned above on

the vBNS and NSFnet backbone topologies are presented. Theperformance results are tabulated in Tables IV–XIII. Due tolimited space, the numerical results are not commented withdetails, but, instead, some general observations are given

Giving priority to shortest paths, a routing scheme alwaysperforms better. This is due to the fact that the use of long pathsis undesirable since they tie up network resources at more inter-mediate nodes than minimum hop paths. Thus, since the amountof the consumed resources increases, the number of the connec-tions that can be accepted decreases.


TABLE XIPERFORMANCE OF THEROUTING ALGORITHMS IN TERMS OF THEAVERAGE

CELL TRANSFERDELAY IN SECS(I.E., THE AVERAGE DELAY REQUIRED TO

DELIVER A CELL FROM ORIGIN TO DESTINATION) UNDER TIME-VARYING AND

SKEWED LOADING CONDITIONS FOR THENSFNET-BACKBONE TOPOLOGY

CONFIGURED AS A TWOLEVEL HIERARCHICAL NETWORK

TABLE XIIPERFORMANCE OF THEROUTING ALGORITHMS IN TERMS OF THEBANDWIDTH

ACCEPTANCERATIO (EXPRESSING THENETWORK REVENUE) UNDER

TIME-VARYING AND SKEWED LOADING CONDITIONS FOR THE

NSFNET-BACKBONE TOPOLOGY CONFIGURED AS A TWOLEVEL

HIERARCHICAL NETWORK

TABLE XIIIPERFORMANCE OF THEROUTING ALGORITHMS IN TERMS OF THEACHIEVED

THROUGHPUT(EXPRESSED AS THENUMBER OF THE SUCCESSFULLY

ESTABLISHED CALLS) UNDER TIME-VARYING AND SKEWED LOADING

CONDITIONS FOR THENSFNET-BACKBONE TOPOLOGY CONFIGURED

AS A TWO LEVEL HIERARCHICAL NETWORK

In light loading conditions, even though a routing schemeshould favorably select shortest paths for use, it should also bal-ance the load of the network since there are plenty of resources.As the network load increases, it should use alternate routesmore selectively decreasing the load balancing, so that waste ofnetwork resources and consequently, premature network con-gestion will be avoided.

Generally, the algorithms that use trunk reservation (SELA,TSLR, TLALT) exhibit better performance than the others(LSR, SLR, LALT) in every loading scenario except for thecase where the network is loaded very lightly. In this lattercase, LALT shows the best performance establishing moreconnections than the other algorithms.

The SELA routing algorithm achieves the best performanceover all the other routing algorithms, in almost all the loadingconditions. Specifically, it shows the lower blocking ratio inseven out of the eight examined loading scenarios, allowingmore connections to be established. The only case that one of theother algorithms -specifically the LALT- blocks fewer calls thanSELA is under very light loading conditions (when the arrival

rate is ). In this case SELA exhibits the second lower blockingratio. As far as the average cell delay performance metric is con-cerned, SELA manages to deliver the cells from origin to desti-nation exhibiting the second lower value in all of the examinedloading conditions, since the LSR always achieves the lower av-erage cell delay as it is expected.

Finally, it should be noted that the fact that as the load in-creases the average cell transfer delay decreases is not peculiar,although it seems supprises at first. This happens because whenthe load increases, fewer alternate routes are being selected,thus lowering the average propagation delay and consequentlythe average cell transfer delay (transmission delay and queuingdelay are much smaller than the propagation one and those, theydo not affect the overall cell transfer delay as significantly as thepropagation delay do.

Generally, it could be concluded that among the comparedrouting schemes, the SELA routing algorithm always hasthe best (or the second best) performance irrespective of theloading conditions. Thus, it manages to admit more callsinto the network than the other algorithms, and deliver thecells from origin to destination achieving a quite satisfactoryperformance regarding the average cell delay metric, whileguaranteeing the QoS requirements (since the CAC algorithmrejects the calls when the QoS constraints cannot be satisfied).The SELA routing algorithm achieves to route the offeredtraffic effectively because it avoids unnecessary use of alternateroutes while balancing the load when required. By predictingan oncoming congestion, it routes the offered loads throughthe less loaded paths. The simulation results demonstrate theeffective compromise that the SELA routing algorithm attainsbetween the use of minimum-hop routes and the load balancing,which is the key of its efficiency.

VIII. C ONCLUSIONS ANDFUTURE WORK

In this paper, a dynamic call routing methodology was pro-posed that uses an efficient RLA suitable for real-time appli-cation in ATM networks. RLAs as simple propabilistic modelsperforms robustly in the face of uncertainty conducting safety-based routing (inaccurate bandwidth feedback from aggregatedtopology). This algorithm uses hierarchical route information ofthe connection, aggregated topology and loading information ofthe network to select feasible paths that satisfy the QoS con-straints while achieving high resource efficiency.

In particular, it is demonstrated that for the implementationof the algorithm two sets of information are required. The firstone regards the topology information that is necessary for thecomputation of the k-shortest path hierarchically completeroutes, while the other includes the attribute information of theallocated bandwidth on the links that comprise these routes.This latter information is used by the proposed methodologyfor calculating and retaining a link utilization database, whichis used by the routing algorithm for the selection of the mostsuitable among these k-shortest path routes. Since the PNNIprotocol adopted by the ATM forum [3] includes mecha-nisms for distributing the information as well as hierarchymechanisms, which provide an aggregated view of clustersof switching elements, it is clear that the proposed routing


methodology can be implemented in the framework of thePNNI.

A comparative performance study among the proposedand other well-known path selection routing algorithms wasdemonstrated by means of simulation on existing commercialnetworks. The used performance metrics were the average celltransfer delay, bandwidth rejection ratios, bandwidth accep-tance ratios, network throughput, and network revenue thatthe various algorithms exhibited under uniform, time-varyingand skewed traffic workloads. Simulation results showed theeffectiveness of the proposed routing algorithm and disclosedthe strength and weakness of the various schemes.

Our future research includes studying other aggregationschemes and route selection policies. Our objective is tocompare the performance of the various aggregation schemesand study the influence that other important factors exer-cise on that performance, such as routing update frequency,routing algorithms, and network configuration. Besides, wealso plan to extend the scheme to handle point-to-multipointand multipoint-to-multipoint routing. Finally, a promisingapproach seems to be the use of AntNets mobile distributedagents(or completely fault-tolerant ant agents) that collect on-line information to improve/enrich the operation of the SEL’sagents, statically connected to the network nodes and directlyresponsible for routing decisions. This is a ReinforcementLearning multiagent system showing many interesting featuresi.e the routing traffic does not increase with the rate of changein the network making these algorithms suitable for highlydynamic networks (adaptability to user requirements, demand,cost structure, and topology changes) [31].

APPENDIX

Proof of Epsilon Optimality

(For Detailed Analysis of the SELA and its Epsilon Opti-mality the Reader Should Consult [34]). We shall first provesome supplementary lemmas.

Lemma 1: Assume two statistically independent randomvariables , with mean equal to , respectively. Definethe random variable . If the density functions

, of , are symmetric (with respect to theirmeans , ) then the random variable is also symmetric(with respect to its mean ).

Obviously this lemma can be easily generalized forwhere are symmetric

random variables.Proof: Define the random variables , as :

, and with density functions ,and characteristic functions , , respectively. Ob-viously the random variables

Proof: Define the random variables , as

(5)

with density functions , and characteristic functions, , respectively.

Obviously, the random variables and have means equalto 0 and are symmetric with respect to 0. It is known ([32],

Theorem 6.2.6.) that under these conditions , arereal valued.

Now, let’s define the random variable . It is known([33], Theorem 13) that the characteristic function ofis:

(6)

Since and are real valued it is derived thatis also real valued.

Thus, is symmetric with respect to 0.Since

it is derived that is symmetric with respect to. Lemma 1 has been proved.

Lemma 2: Let to be (respectively) the meansand to be the variances of the symmetricallydistributed (with respect to their means) rewards of theactions , offered by the environment. Let

be corresponding current stochasticestimates of the mean rewards at the time instant t. Then,

are symmetrically distributed (withrespect to their means) random variables with means equal to

, respectively.Proof: The symbolism denotes that the

random variable is normally distributed with a mean and avariance

Let be the window size. Consider action .Let be the content of theth place of action’s

window at time instant.If the action was the last selected at time and

we have

(7)

By definition, the random variables forare symmetrically distributed with a meanand a variance

is also a symmetric random variable. Thus, is a sum ofsymmetric random variables. So, from Lemma 1 is derived that

is also a symmetric random variable. Its meanand its variance are as follows :

In the same way if thenThus, is a symmetrically distributed (with respect

to its mean) random variable with a mean equal to(for every). Lemma 2 has been proved.

Lemma 3: For any two actions ai and aj we define:. If then there is a positive real number

such that and consequentlyfor any time instant. We assume that has a

nonzero limit.Proof: Define the random variable: .

From Lemmas 1 and 2 is derived that is a symmetric (withrespect to its mean) random variable with a mean equal toand a variance equal to .


Let be the density function of . As discussed ear-lier, is symmetric about the line .

If we define we have:.

Thus, we have

(8)

It is known that and is symmetric aboutthe line . So, (4) can be written as

(9)

whereSince the variance of is bounded

it is derived that . Itis obvious that

Thus

Thus we have thatfor any time instant. Lemma 3 has been proved. Note that thisis not proven conditional on the state of the learning automata.

Lemma 4: Assume any subset of the action set ,such that . Thus,

. For any action define:for .

Then according to the SELA LA, for any two actions,such that and we have

for any time instant. (Where is defined as in Lemma 3).Proof: We shall prove Lemma 4 by utilizing mathematical

induction on the size of the subset.Basis step. We have to prove that the theorem holds

for . Proof of basis step.For any subsetwe have

Thus, for any timeinstant .

Induction hypothesis. Assume that the theorem holds for. Thus, for any subset An of the action set A such

that it holds that according to the LA of the SELAautomaton, for any two actions, such that and

we have: forany time instant .

Induction step. We have to prove that the theorem holdsfor: .

Thus, we have to prove that for any subset of the actionset such that it holds that according to the

SELA LA, for any two actions , such that , and, we have

(10)

In the same way

(11)

by utilizing (6) and (7) we have

(12)

So (8) becomes

Thus, Lemma 4 has been proved.Lemma 5: Let for . Thus

action is an optimal one. Ifthen according to the SELA

LA for any time instant t we have for.

Proof: Let action be an optimal one. Thus,for .

From Lemma 3 is derived that for everyit holds that: . Thus, for every

it holds that:

(13)

Now, by combining (9) with Theorem 3 (for subset )is derived that for every it holdsthat

Thus, for . Lemma 5 hasbeen proved.

Theorem 1: The SELA learning automaton is-optimal inevery environment that offers symmetrically distributed noise.Thus, if action is an optimal one ( for

) and , then for every valueof the resolution parameter there is a time

instant such that for every it holds .Proof: For every two actions , and every time instant

t define .As in Lemma 5 let’s define the quality:


Assume that at a time instantwe have and. If neither action or has the highest estimated

mean reward at this instant ( for) then both and are decreased by the same

quantity (where is the resolution parameter of the SELAautomaton). Thus

If action or action has the highest estimated mean re-ward at the time instant

T( , or for ) thequantity is an average increase by

where the quantities , are as defined above.Thus, if at a time instantwe have and

then the average modification of the quantity is

(14)

For any pair of actions , define the quantity as:

If and then:

If then from Lemma 3 is derived thatfor any time instant. In this case we have

(15)

It is known (see proof Lemma 3) that for .Thus, from (11) is derived that if , are two actions so that

then

Lets define

We have:

(16)

From (12) is derived that for any pair of actions, with, there is a time instant such that:

(17)

If then .Thus, before will become equal to zero to average differ-ence between and is

There are two possible cases.

i) and ,ii) and .If condition (13) is satisfied then Case ii) becomes

Thus, if condition (13) is satisfied then in both case (i) and(ii) we have .

Thus, .We have proved that for every pair of actions, withthere is a time such that:

(18)

Let action be an optimal one ( for).

Thus, for any section we have and conse-quently, from (14) is derived that for any action thereis a time instant that

If we select then for everyand, subsequently,

Theorem 1 has been proved. The SELA learning automaton is-optimal in every stationary stochastic environment.

REFERENCES

[1] “Traffic Control and Congestion Control in B-ISDN ,”, Geneva, ITU-T,Recommendation 1.371, 1996.

[2] P. Mars and W. J. Poppelbaum,Stochastic and Deterministic AveragingProcessors. Stevenage, U.K.: Peregrinus, 1981.


[3] “Private Network-Network Interface Specification Version 1.0(PNNI 1.0),” The ATM Forum Technical Committee, ATM Forumaf-pnni-0055.000, 1996.

[4] R. Guerin and A. Orda, “QoS-Based Routing in Networks With Inaccu-rate Information: Theory and Algorithms,”, IBM Res. Rep., RC 20515,1996.

[5] W. Lee, “Topology aggregation for hierarchical routing in ATM net-works,” Comput. Commun. Rev., pp. 82–92, 1995.

[6] , “Spanning tree method for link state aggregation in large commu-nication networks,” inProc. IEEE INFOCOM, Apr. 1995, pp. 297–302.

[7] B. Awerbuch and Y. Shavitt, Topology Aggregation for Directed Graph,to be published.

[8] B. Awerbuch, Y. Du, B. Khan, and Y. Shavitt, Routing Through TeranodeNetworks with Topology Aggregation, to be published.

[9] H. Xie and J. Baras, “Performance Analysis of PNNI Routing in ATMNetworks: Hierarchical Reduced Load Approximation,” Univ. Mary-land, Institute System Research Tech. Rep. TR 97-65, 1997.

[10] J. Baras and M. Corson, “Tactical and strategic communication networksimulation and performance analysis,” in Proc. 1st Annual Conf. onAdvanced Telecommunications and Information Distribution ResearchProgram (ATIRP), to be published.

[11] A. Orda, “Routing with end-to-end QoS guarantees in broadband net-works,”IEEE/ACM T-Networking, vol. 7, no. 3, pp. 365–374, June 1999.

[12] D. Lorenz and A. Orda, “QoS routing in networks with uncertain param-eters,”IEEE/ACM T-Net, pp. 768–778, Dec. 1998.

[13] S. H. Kim, K. Lim, and C. Kim, “A scalable QoS-based inter-domainrouting scheme in a high speed wide area network,”Comput. Commun.,vol. 21, pp. 390–399, Apr. 1998.

[14] E. Felstaine and R. Cohen, “On the distribution of routing computationin hierarchical ATM networks,”IEEE/ACM Trans. Net., vol. 7, no. 6,pp. 906–916, Dec. 1999.

[15] F. Hao and E. Zegura, “On scalable QoS routing: performance evalua-tion of topology aggregation,” inProc. IEEE INFOCOM, Mar. 2000, pp.147–156.

[16] Marie, S. Singh, T. Jaakkola, and M. Jordan, “Learning without stateestimation in partially observable Markovian decision processes,” inProc. 11th Machine Learning Conf., New Brunswick, NJ, July 1994,pp. 284–292.

[17] A. V. Vasilakos and G. Papadimitriou, “Ergodic discretized estimatorlearning automata with high accuracy and high adaptation rate fornonstationary environments,”Neurocomput., vol. 4, pp. 181–196, May1992.

[18] A. F. Atlasis, M. P. Saltouros, and A. V. Vasilakos, “On the use of astochastic estimator learning algorithm to the ATM routing problem: Amethodology,” inProc. IEEE GLOBECOM, Dec. 1998.

[19] A. F. Atlasis, A. V. Vasilakos, and N. H. Loukas, “The use of learningalgorithms in ATM networks call admission control problem: Amethodology,” inComputer Networks Amsterdam, North Holland,The Netherlands.

[20] A. F. Atlasis and A. V. Vasilakos, “LB-SELA: Rated-based access con-trol for ATM networks,” in Proc.IEEE INFOCOM 94, Mar. 1994, pp.1552–1569.

[21] K. Narendra and M. Thathachar, “Learning automata: A survey,”IEEETrans. Syst. Man, Cybern., vol. TSMC–4, pp. 323–334, July 1974.

[22] Q. Ma and P. Steenkiste, “On path selection for traffic with bandwidthguarantees,” in Proc. IEEE Int. Conf. on Network Protocols, Atlanta,GA, to be published.

[23] T. H. Cormen, L. Ch, R. L. Rivest, and C. Stein,Introduction to Algo-rithms, 2nd Edition ed. Cambridge, MA: MIT Press, 2001.

[24] A. M. Law and W. D. Kelton,Simulation Modeling and Analysis, seconded, E. M. Munson and L. Margery, Eds. New York: McGraw-Hill,1991.

[25] I. Matta and A. U. Shankar, “Dynamic routing of real-time circuits,” inProc. IEEE Int. Conf. on Network Protocols, to be published.

[26] M. Hluchyi and M. Carol, “Queuing in high-performance packetswitching,” IEEE J. Select. Areas Commun., vol. 6, Dec. 1988.

[27] D. P. Martin, Asynchronous Transfer Mode: Solution for BroadbandISDN, 2nd ed. West Sussex, U.K.: Ellis Horwood, 1993.

[28] R. Guerin, H. Ahmadi, and M. Naghshineh, “Equivalent capacity andits application to bandwidth allocation in high-speed networks,”IEEEJ. Select. Areas Commun., vol. 9, pp. 968–981, Sept. 1991.

[29] D. Anick, D. Mitra, and M. M. Sondhi, “Stochastic theory of a data-handling system with multiple sources,”Bell Syst. Tech. J., vol. 61, pp.1871–1894, Oct. 1982.

[30] L. Zhang, M. Andrews, S. Bhatt, and K. Krishnan, “A performance com-parison of competitive on-line routing and state-dependent routing,” inProc. IEEE GLOBECOM, 1997, pp. 1813–1819.

[31] G. D. Caro and A. V. Vasilakos, “AntNet combined to SELA for QoSrouting in ATM networks,” inProc. Ants’2000, From ant Colonies Arti-ficial Ants: 2nd International Workshop ant Algorithms, Brussels, Bel-gium, Sept. 8–9, 2000.

[32] K. l. Chung,A Course in Probability Theory. Orlando, FL: HarcourtBrace Jovanovich, 1968.

[33] H. Cramer, Random Variables and Probability Distribu-tions. Cambridge, MA: Cambridge Univ. Press, 1970.

[34] A. Vasilakos and G. Papadimitriou, “A new approach learning automata:Stochastic Estimator Learning Algorithms,” inNeurocomputing, 1995,vol. 7, pp. 275–297.

[35] W. Pedrycz and A. Vasilakos,Computational Intelligence in Telecom-munications Networks. Boca Raton, FL: CRC, 2000.

Athanasios Vasilakoshas born in Pella, Greece, in1959. He received the B.S. degree in electrical en-gineering from the University of Thrace, Greece, in1983, the M.S. degree in computer engineering fromthe University of Massachusetts, Amherst, 1986, andthe Ph.D. degree in computer engineering from theUniversity of Patras, Greece, in 1988.

From 1988 to 1991, he was with ComputerTechnology Institute, University of Patras. From1991 to 1995, he was Professor at the Hellenic AirForce Academy, Greece. From 1995 to 2002, he was

Researcher at the Computer Science Institute, Foundation for Research andTechnology-Hellas(FORTH), Greece. Since 2002, he is a Professor at the Uni-versity of Thessaly, Greece, and Manager in the Hellenic Aerospace Industry.His research interests involve telecommunication networks (IP/ATM, mobile,active nets) and computational intelligence. He is a co-editor (with W. Pedrycz)of the volume entitledComputational Intelligence in TelecommunicationsNetworks, (Boca Raton, FL: CRC Press, 2001). He has published more than 90peer-reviewed journal and conference papers.

Dr. Vasilakos is a member of the technical progeram committees of a numberof conferences and serves on editorial boards of several journals, includingComputer Communications(1997–2002),ACM Applied Computing Review,Soft Computing, and IEEE Communications Magazine. He has been a GuestEditor of several journals, such as IEEE TRANSACTIONS ON SYSTEMS, MAN,AND CYBERNETICS, IEEE JOURNAL ON SELECTEDAREAS INCOMMUNICATIONS,Soft Computing, and theJournal of Interactive Learning Research. He is amember of the ACM.

M. P. Saltouros was born in Xouthi, Greece in 1973. He graduated from theHellenic Air Force Academy in 1995 and received the Ph.D. degree from Na-tional Technical University, Athens, Greece, in 2001.

He has been an Officer in the Hellenic Air Force Academy since 1995.

A. F. Atlassis was born in Thessaloniki, Greece in 1974. He graduated fromthe Hellenic Air Force Academy in 1993 and recieved the Ph.D. degree fromNational Technical University, Athens, Greece in 1999.

He has been an Officer in the Hellenic Air Force Academy since 1993.

Witold Pedrycz (F’99) is a Professor and Chairin the Department of Electrical and ComputerEngineering, University of Alberta, Edmonton AB,Canada. He is also a Canada Research Chair inComputational Intelligence. He is actively pursuingresearch in computational intelligence, fuzzymodeling, knowledge discovery and data mining,fuzzy control including fuzzy controllers, patternrecognition, knowledge-based neural networks,telecommunication networks, relational computa-tion, and software engineering. He has published

numerous papers in this area. He is also an author of seven research mono-graphs covering various aspects of computational intelligence and softwareengineering.

Dr. Pedrycz has been a member of numerous program committees of IEEEconferences in the area of fuzzy sets and neurocomputing. He currently servesas an Associate Editor of IEEE TRANSACTIONS ON SYSTEMS, MAN, AND

CYBERNETICSand IEEE IEEE TRANSACTIONS ONFUZZY SYSTEMS.

Optimizing QoS Routing in Hierarchical

Documents

Transcript of Optimizing QoS Routing in Hierarchical