INTERNATIONAL JOURNAL OF DIGITAL AND ANALOG CABLED SYSTE~1S. VOl 77-85 (191!8)

P ARIS: AN APPROACH TO INTEGRA TED

NETWORKS

-IIGH-SPEED PRIVATE

ISRAEL CIDON AND INDER S. GOPAL

IBM T. I. Walson Research Cenler, Yorklolvn Heighls, N. Y. 10598, U.S.A

SUMMARY

This paper describes a design of a high-speed packet switching system for integrated voice, video anddata communications. The system makes use of a simplified network architecture in order to achievethe low packet delay and high nodal throughput necessary for the transport of voice and video. Aprototype of this system has been implemented and is now being tested under a variety of packettraffic loads. We have demonstrated that this system provides a cost-effcctive solution for privateintegrated networks.

KEY WORDS Integrated networks Packet switching Broadband ISDN Private networks

INTRODUCTION network comprised of several switching nodes inter-connected by links of speeds of the order of 100Mb/s. Although lower speed links may exist, webelieve that the availability of leased T3 and fibreoptic links will make it cost effective to have a fewlinks of high speed rather than many links of lowerspeed. In addition to the backbone there will exista peripheral network which will essentially provideaccess into the switching nodes. The peripheralnetwork will be comprised of relatively low-speedlinks and may not use the same protocols orswitching techniques used in the backbone. Inaddition, the peripher~1 network will perform thetask of multiplexing the relatively slow end users tothe high-speed backbone. Thus, the backboneswitching nodes will primarily handle high-speedlines. The number of high-speed links entering eachswitching node will be relatively small (probablyless than 20) and the aggregate throughput will bein the 1-4 Gb/s range. In addition to the requirementof low delay (less than a millisecond per node) andhigh throughput, cost and design complexity are keyfactors in our environment. The main thrust of thispaper is to show that for such an environment,packet switching is a cost effective and viable

technology.Another thrust of the paper is to demonstrate the

value of designing the network as a single completesystem. In particular, rather than designing network-ing architecture and nodal hardware structure separ-ately, we tailor the architecture and hardware towork efficiently with each other. There are severalinstances where the design of the networkingarchitecture with the hardware in mind considerablysimplifies the hardware. The consequence of thissystem design approach is a high-speed packetswitching node that can satisfy the throughput anddelay requirements and is simple enough to be

Today's communication systems carry data trafficthrough packet switching techniques but use circuitswitching techniques for voice. Conventional wisdomhas argued that the statistical multiplexing offeredby packet switching makes it ideally suited to trafficof a bursty nature such as interactive data. Forsteady streams of traffic, such as voice, the nodalprocessing overheads necessary for each packetovercome any bandwidth savings that statisticalmultiplexing may achieve, and circl,lit switchingtechniques are more appropriate. The conventionalwisdom was extensively studied and tested by severalstudies in the late seventies.1-3 Essentially, all ofthese works were. unable to show conclusively thatpacket switching is suitable for voice because theywere attempting to use general packet switchingtechniques (SNA, ARP A) that were developed forthe transport of data.4. s These techniques use a

general purpose processor to do the packet switchingin software, and consequently are not able to providethe substantial throughputs necessary for voice.

The first work to realize the true potential ofpacket switched voice was Reference 6, wherein thebasic idea of off-loading the packet switchingfunction onto specialized hardware is proposed. Thetarget there is a replacement of the telephone tollswitch with a packet switch. Thus, high throughputis the major consideration. The design is fairlycomplex and requires extensive use of custom VLSIchips. The design requirement calls for a node thatsupports around a thousand 1.5 Mb/s links with anaggregate throughput of around 6 Gb/s.

Our design Tequirements are somewhat different.We are interested in designing a private integratedvoice/video/data network. We envisage that such anetwork will consist of a high-speed backbone

Received 4 May 19880894-3222/88/020077-09$05.00

@ 1988 by John Wiley & Sons, Ltd

78 I. CIDON AND 1.5. GOPAl

3.

components in gathering information (e.g.traffic statistics) that is needed in performingthese functions. It also interacts with the NCUsin other nodes both in gathering informationand in performing the functions themselves.The SS performs the intermediate node func-tions that are involved in the transport ofpackets. These functions include packet fram-ing, packet buffering, determination of theoutgoing link, the actual switching function oftransferring the packet from the incoming tothe outgoing link, congestion control to preventexcessive packet queueing, priority functions.In addition to intermediate node functions, theSS performs statistics gathering to report tothe NCU.The EPP is responsible for constructing thepacket in the appropriate format and deliveringit to the network. This involves inserting theappropriate headers and delimiting flags andperforming bit-stuffing and destuffing oper-ations (if needed). In addition, the EPPperforms the end-to-end functions of flowcontrol, error recovery and reassmbly/playoutetc. These end-to-end EPP functions are sensi-tive to the nature of the traffic that is beingtransmitted.

NETWORK TRANSPOR'

The packet structure

Figure 2 describes the structure of a packet. Theimportant points are as follows:

I. Leading and trailing delimiters (flags) fordefining packet boundaries.

2. A two-byte control which is largely unused atpresent except for packet priority level (twobits), a copy bit which indicates whether thepacket should be copied to the NCU in additionto being transferring the outgoing link and abroadcast bit which causes the packet to besent on all outgoing links.

3. The automatic network routing (ANR) field,which is composed of h words, each wordbeing two or more bits in length (h is the

readily implemented with off-the-shelf components.We refer to our system as PARIS, which is anacronym for Packetized Automatic Routing Inte-

grated System.An initial prototype of p ARIS has been

implemented and is currently working in our lab.Each switching node can support up to 10 fibreoptic lines each operating at 100 Mb/s. The designcan be easily enhanced to support aggregate through-puts of up to 4 Gb/s per node.

The PARIS networking architecture can bedecomposed into two major parts:

1. Network transport. These are the functionsthat are executed in the actual transmission ofpackets. These functions include flow control,error recovery, intermediate node routing, etc.

2. Network control. This essentially comprisesthe functions that are related to the set-up ofa call. The functions include the call acceptancefunction that makes decisions on whether ornot to permit a new call access to the network,the route computation function that determinesthe path that a call is to be routed over, aswell as directory and other application levelfunctions which perform important servicessuch as the location of remote networkresources.

The P ARIS approach to network transport isquite different from conventional packet switchednetworks. The key driving requirements are toreduce the end-to-end delay in 9rder to satisfy realtime delivery requirements and to achieve highnodal throughput. In order to accomplish this, theprocessing in the intermediate nodes is reduced toa minimum. The intermediate node routing functionis performed through a technique called automaticnetwork routing (ANR) which requires no tablelook-up or storage. Congestion control and prioritiesare implemented through very simple procedures.Most of the transport level functions (such asflow control and error recovery for data, andpacketization and reassembly for voice and video )are performed on an end-to-end basis.

The P ARIS approach to network control is adecentralized one. For fault tolerance and perform-ance reasons, it is well accepted that for moderate-sized networks decentralized control is preferableto reliance upon one or more central controller(s).Thus, in the PARIS system, every backbone nodeparticipates in a set of distributed algorithms whichcollectively comprise the network control of the

system.Figure 1 is a network picture showing the various

components in a P ARIS node. A P ARIS nodeconsists of three basic components, the networkcontrol unit (NCU), thc switching subsystem (SS)and the end point processor (EPP). The functionsof these components are briefly described bclow.

1. The NCU implcments all the network controlfunctions. In addition it interacts with the other

79'ARIS

Figure 2. Packet structure

number of hops that the packet has to travel).Each word represents an outgoing link on thepacket's path. We shall give more details onthis field later on in the section.

4. The information field, which is of variablelength. There is a maximum and a minimumsize for the information field which dependson the actual implementation. Typically, themaximum is about 4K bytes and the minimumabout 8 bytes. The information field alsocontains headers and trailers that relate to theend-to-end protocols.

Automatic network routing

As the ANR is a key point of the design, we shallelaborate on the ANR field a little further. Asmentioned above, the ANR field is composed of hwords. The ith word in the ANR field defines theoutgoing link label of the ith hop along the packetpath. The outgoing link label is essentially theinternal switch ID or address (SID) of the outgoinglink adaptor. Thus, the packet header contains allthe routing information necessary for the routing ofthe packet within each intermediate node along thepath. As the packet progresses through the networkthe 'used' SIDs are stripped off, so that the firstbits in the ANR field always contain the routinginformation for the current node. This process isdepicted in Figure 1. Thus, every node will examinea fixed location in the header without having toknow of its pnsition in the path. No externaltable look-ups or processing are necessary, therebyensuring minimal nodal delay. The ANR techniqueis analogous to the self-routing header techniqueused to route information through a multi-stageswitch.7 The difference is that we are using theheader across an entire general topology networkrather than within a single node.

The ANR technique has several advantages ina high-speed packet switching system and somedisadvantages. The major advantage over conven-tional packet switching techniques such as the LPIDswapping procedure used by TYMNETK is the factthat no table look-up or processing is necessary atan intermediate node. This eliminates any potentialprocessing bottle-necks and minimizes intermediatenode delays. Most importantly, it removes the needfor the intermediate node routing tables to beupdated at the time of call set-up while preservingthe ability for dynamic route selection. This reducesconsiderably the load on the NCU of the intermedi-ate nodes during call set-up, thereby eliminatingpotential throughput bottle-necks and permitting

calls to be set up and taken down much faster thanin conventional circuit switched systems. Thus, routeswitching (i.e. changing the path of an ongoing call)without disruption to the end user becomes feasible.In addition, the hardware structure of a switchingnode can be streamlined as no routing table updateshave to be sent from the NCU to the switchingsubsystem. This permits the adaptor design to bemuch simpler, potentially permitting a much morecompact implementation of the switching node. Inaddition to reducing the delays in the call set-upphase, the ANR technique also permits packets tobe sent from the source to destination without a set-up phase. Thus, data applications which benefitfrom a 'datagram' service, typically those thatgenerate occasional packets but require rapidresponse times, can be effectively supported by this

technique.However, there are some disadvantages with the

ANR technique. First, the inclusion of the entirepath in the header wastes communication capacity.Thus, in communication capacity limited systems,such as conventional data packet switched systems,the ANR technique is not suitable. In networkswith high capacity links, communication capacity isno longer a bottle-neck and some wastage isacceptable. Moreover, studies on typical networksand on randomly generated large networks9 haveindicated that typical communication paths arealmost always less than 7 hops long. Since usedSIDs are being stripped off, the average overheadis only half of the path length resulting in relativelyshort ANR fields. Another disadvantage of ANR isthe added burden placed upon the originating node,as it has to know the entire path. Again, thisdisadvantage is not as significant as it may seem asmany conventional packet switched systemss. 10

already maintain full network topology informationat every node even though they perform hop-by-

hop routing.There are two SIDs that are reserved for special

use. The reserved SIDs are the all-zeros SID, whichis always the SID of the NCU adaptor, and the all-ones SID, which is termed the dummy SID and isalways unused. Observe that since the SIDs mightbe of different length, it is important to enforce aprefix condition within a single node. In otherwords, it is important to ensure that no SID is aprefix of another SID within the same node. 1hisprefix condition must hold for the NCU SID andthe dummy SID as well.

The ANR field is terminated by two successivedummy SIDs (which are stripped off at the endpoint.) The reason for this is to ensure that biterrors in the ANR field which may cause the packetto be misdirected do not cause packets to travelaround the network for a very long time. After arelatively small number of hops, the dummy SIDswill be the used to route the packet to a non-existent link adaptor, causing the packet to bediscarded.

80 CIDON AND I.S. GOPA

For simplicity reasons in our prototype we areusing a fixed-Iength SID of a single byte. Four ofthese bits are used for addressing the actual outgoinglink. The other four bits are used for selective copyand broadcast mechanisms.

The intermediate node functiolrs

We describe here the network transport functionsthat are implemented in the switching subsystem ofeach PARIS node.

Figure 3 describes the components of the SS. Thekey components of the SS are:

1. The switching kernel (SK), which performs thebasic switching function of transferring packetsfrom source to destination.

2. The link adaptors, that are each comprised ofa receive part and a transmit part. The receivelink adaptors receive incoming packets fromthe link and send them through the switchingkernel. The transmit link adaptors receivepackets from the SK, buffer them as necessaryand transmit them over the outgoing link.

3. The monitor that collects traffic statistics andreports them to the NCU .

There are two basic motivations behind the design:

1. To minimize the transit time for a packetthrough an intermediate node.

2. To achieve the above goal with minimalhardware complexity.

Optimizing channel bandwidth use, a primary objec-tive of many current packet switching systems, isviewed here as a secondary objective as linkcapacities are assumed to be very large. In otherwords, in order to minimize nodal transit time andto reduce nodal complexity, we are prepared to usemore channel bandwidth than necessary. There are

NETWORKCONTROL

UNIT

fi

some key aspects, unique to our design, that enableus to achieve our goals. We shall elaborate on theseaspects further as we follow through a packettransfer from incoming to outgoing link.

A packet arrives at the receive adaptor as a high-speed serial bit stream. The receive adaptor firsthas the job of recognizing the packet, performinga serial-to-parallel transfer and storing it into itsincoming buffers. In addition the Sill of thisparticular adaptor is stamped to the beginning ofthe packet. This procedure allows the NCU and themonitor (if need~d) to identify the source link ofthis packet. If the link protocol requires a bit-stuffing operation we have adopted an end-to-endbit-stuffing protocol. Thus, rather than performingthe bit stuffing and destuffing on every hop, as inconventional link protocols such as HDLC, weperform it once on an end-to-end basis. The stuffingis done only in the information field. The headerfields are to be used in the intermediate nodes andtherefore cannot be bit-stuffed end-to-end. Thus weenforce a structure on the header, which ensuresthat this portion of the packet does not contain anyflags, and therefore need not be bit-stuffed and canbe used by the intermediate nodes directly. Thelogic at the receiver adaptor is therefore relativelysimple. Once the receiver has stored a completepacket, it signals the SK that it wishes to transmita packet.

The method by which the SK transmits the packetis, of course, dependent on its internal structure. Ifthe SK has a single-path' architecture such as a bus,it is important that the contention for the SK beresolved quickly. This is to ensure that queueingdelays at the receiver are strictly limited and thesystem has effectively a single queueing point at thetransmit adaptor. This can be achieved if the SKadopts a round-robin approach to serving theadaptors and if the speed of the SK is at least aslarge as the sum of the speeds of all incoming links.Round-robin operation of the SK is also inherentlyfair as it guarantees every source equal access. Forthe single-path SK, contention delay can be reducedstill further or eliminated altogether by pipeliningthe contention resolution with the transmission ofthe previous packet.

Another important aspect of an implementationon a single-path SK is that once th6 receiver adaptor(source) has obtained control of the SK, it shouldretain control for the duration that it takes to sendone or more complete packets. This transmission ofa packet as a complete entity has the basic advantageof eliminating contention at the transmit adaptor(destination). In other words, a destination willreceive only one packet at a time, and does nothave to consider the possibility of simultaneouslyreceiving packets from two separate sources. Thissimplifies considerably the design of the transmitadaptor .

The use of a single-path SK along with a round-robin arbitration policy ensures that the number ofbuffers at the receive adaptor is bounded as longFigure 3. The !;witching subsystcm

81PARIS

they include flow control and error recovery for

data and the packetization/playout protocol for

voice.

Data protocols. The data protocols in PARISare optimized to work with real time data. We adopta layered structure, as depicted in Figure 4. Thelowest level is the input 'throttle', which is a flowcontrol mechanism designed for real time data. Ithas the property that is guarantees a certainminimum average rate for every session with theability to send bursts of information at much higherthan the minimum average rate. The next level isthat of error recovery. This layer retransmits packetsthat are lost from buffer overflows or from randombit errors. The highest layer is that of end-to-endpacing which ensures speed matching between thereceiver and transmitter. This is to avoid thepossibility of receiver buffer overflow.

The input 'throttle'

The flow control protocol is based on the simpletechnique of regulating the input. We first describethe basic scheme, which guarantees a fair use ofbandwidth by all users but is relatively conservativein nature. In the set-up of a data call there is,associated with that call, an average bit rate (whichwe denote by A packets's) and a burst size parameter(which we denote by N).

The scheme, depicted in Figure 5, operates asfollows. A packet leaving the throttle mechanismrequires a token. Tokens are produced periodicallyat a rate of one token every l'A seconds unless thepool has reached a maximum value of N tokens.Thus, if a session has several tokens in the pool itcan transmit a burst of packets at the rate at whichthe link can permit, probably much faster than itsaverage rate. Once it goes beyond this initial burst,

as the SK throughput is at least large as the sum ofall input rates. It has been provedll that no morethan 3.35 buffers (each of a maximal length packet)are needed at each input of the SK in the worst

case.The destination identified by the first SID in the

ANR field copies the packet into its buffers. Inorder to guarantee low delay for certain processes weprovide various priority levels. This is accomplishedthrough a dedicated buffer at each destination, foreach priority level. Depending on the priority bitsin the header of the packet, the packet is placedinto the corresponding buffer. Once a completepacket has been collected, the destination beginsreading out the contents of the buffer and transmit-ting it over the outgoing link. A non-preemptivepriority is accomplished, whereby low priority pack-ets are read only if the higher priority buffers donot contain a full packet.

A detailed performance analysislJ shows that ina typical network at 85 per cent utilization the nodaldelay is bounded by 1ms with packet loss betterthan 10-6. These results also suggest that the usualassumption made that voice must have priority overdata is not valid.

Before actually transmitting the packet over thelink, the transmission adaptor has to perform acouple of extra tasks. First, in order to ensure thatthe SID for the next hop is at the right bit positionin the packet, the current SID along with the sourcelink SID stamp have to be removed. Secondly,packet start and termination delimiters have to bereconstructed. Finally, a parallel-to-serial conversionhas to be performed.

We have completed tracing the packet paththrough the switching kernel. Control packets thatare destined to the NCU travel in a similar fashion,except that the destination address is the networkcontrol adaptor address, i.e, the all-zeros SID.

The monitor is the last, but by no means the leastimportant, part of our SS. The dynamic routecomputation algorithm uses, as a basic input,measurement of link loading. The monitor providesa simple, efficient way to gather such measurements.In each switching subsystem, there exists a singlemonitor that 'observes' the traffic through the SK.

For each packet, the moni~or collects the prioritylevel, the SID bit positions to indicate the outgoinglink, and the packet length. In case the monitor isnot able to keep up with the speed of the links, itwill randomly sample packets, and collect theinformation from that random sample. A processorthen reads the raw information, and periodicallyperforms some computations in order to deriveinformation about the status of all its outboundlinks, including average link loading, packet queuelengths etc. The monitor informs the NCU of thecurrent status.

DESTINATION NODE

I I

SOURCE NODE

! Input'Throttle

f::1-

"l- NETWOR' ~,

~~orDetecti~ I

Error ~ecovery I>\

End-to-end pacing-1>1::)-

Figure 4. The data protocol

}-t>

~I~

Every 1/8 secondsTokens-Min( 1, Tokens+ l'

E\/ery I/A secondsTokens-Min(N, Tokens+ I)The end-point functions

In this selection we discuss the protocols that are

implemented in the EPP. As mentioned before, Till: h:l!iic flow control !icill:ml:Figure

82 I. CIDON AND I.S. GOPAL

From the end node perspective the schemeoperates as follows. A session always uses its basicassigned rate and will not attempt to change itunless packets begin to build up. If this occurs, andif no congestion has occurred, the node will increaseits average rate by steps until packet queues at theinput are small. If congestion occurs, the rate isreduced until it reaches the basic rate below whichit never goes.

Occasionally, during a data session, a need for ahigher long-term average speed arises. This mayhappen, for example, when the mode of work ischanged from an interactive one to a large filetransfer. Reliance upon the adaptive mechanism isundesirable as it does not provide any guarantee ofhigher bandwidth. In such cases, using the fast callset up service of the network, a new call set upprocedure is initiated and more capacity is reservedby the networks nodes (re-routing of the call maybe needed). When the end user does not need thisextra capacity any longer, it initiates a call take-down procedure for releasing that capacity.

Error recovery

Error recovery is also moved to the end nodesand no link-by-Iink error recovery is used. Themotivation behind this is again to reduce intermedi-ate node loading, and the relatively low error ratesexpected of high-speed links make this approachfeasible from a performance viewpoint. The end-to-end recovery protocol is a standard ARQ schemebased on CRC checking at the receiver, an acknowl-edgement mechanism, together with time-outs andpacket retransmission at the transmitter. In orderto compute the correct time-outs, the path lengthand links capacities along the path of the specificcall are used. The end-to-end data protocol alsoprevents buffer overflow at the end nodes due to arate mismatch between the end users.

it is constrained to transmitting one packet every 11A seconds.

This scheme, while maintaining the average inputrate, allows for a temporary increase of the peaktransmission rate, often needed in such environ-ments. The justification for allowing for temporarilyincrements in the traffic rate is the law of largenumbers. As many calls share a high capacitylink and the different calls are uncorrelated, theprobability that many of them are simultaneouslyin their peak rate is very low. However, in orderto place an additional protection upon the network,we place a second throttle in series with the first.This ensures that the speed of packet transmissionnever exceeds the 'bottle-neck' rate of B packets Is.It is particularly necessary if the input into thenetwork is comparable or higher in bandwidth thanthat of the backbone links.

Essentially the choice of N determines the burst-iness of the transmission. A value of 1 ensures asmooth flow of traffic. Typically, the choice of Nshould match the application being supported.

The basic scheme is fair but relatively conserva-tive. It does not permit data users to temporarilytake advantage of capacity that is unused and isbeing wasted. Thus, we have built into the mechan-ism some adaptive capability whereby it can dynam-ically raise the average rate, A, if the network islightly loaded. In the adaptive scheme the parameterA can vary dynamically based on network conditions.The first decision that needs to be made is thechoice of congestion measure. As our scheme is anend-to-end scheme the most desirable measures arethose that can be estimated directly by the endnode without intermediate node involvement. Themeasures that fall into this category are round trippacket delay and loss probability. Unfortunately(from that perspective), loss probability is very small(of the order of 10-6) and is therefore very difficultto estimate precisely. Similarly, the variability ofthe delay which is the indicator of congestion isonly a few milliseconds. Thus, we are forced toabandon these measures and instead adopt a simplemeasure that requires intermediate node involve-ment.

The measure is intermediate node congestion andwe estimate it as follows, Whenever a transmit linkadaptor experiences buffer overflow. it stamps thecongestion bit of all acknowledgement packets thatuse that link in the reverse direction (receivelink adaptor). As all the routes in PARIS arebidirectional, the destinations of packets in thereverse direction are sources for packets in theforward direction. This ensures that the source ofthe traffic realizes that there is congestion and takessteps to alleviate the congestion in the forwarddirection. We use acknowledgements because thenumber of acknowledgements in the receive direc-tion is proportional to the traffic in the forwarddirection, therefore eliminating potential problemscaused by asymmetric traffic loads.

Voice protocols. The basic purpose of the voiceprotocol is to convert a continuous voice bit streaminto packets at the transmitter and to reconstructthe continuous bit stream at the receiver from thearriving stream of packets. The difficulty arisesbecause the packets encounter different delays dueto the random queuing delays at the intermediatenodes. In addition, packets may be lost in theintermediate nodes requiring interpolation on thepart of the receiver in order to preserve theappearance of a continuous voice bit stream. Some-times, however, missing packets may indicate silenceperiods in the speech and the receiver is requiredto recognize and reproduce these silence periodsaccurately. All this is further complicated by thefact that the transmitter and receiver clocks maynot be completely synchronized, resulting in packetsbeing generated and played out at slightly differentrates.

The p ARIS voice protocol described in Reference

83PARIS

12 performs these functions without requiring theuse of sequence numbers or time stamps in theheader of each packet.

NETWORK CONTROL

As mentioned before the design philosophy of thePARIS network control is based on decentralization.Thus, each node participates in a collection ofdistributed algorithms that, together, comprise thenetwork control. The NCU in each node implementsthe distributed algorithm.

Route selection

The fundamental difference between route selec-tion for packetized data networks and route selectionfor packetized voice/data integrated networks is thatin the latter kind of network it is not possible toslow down the real-time traffic (voice) through flowcontrol mechanisms. Consequently, before admittinga call into the network, some guarantee must beprovided that the communication capacity requiredby the call can be satisfied. If not, the call must bedenied access into the network or 'blocked'.

The basic design choice for the P ARIS system isa distributed route selection mechanism using arouting topology database similar to the one inReference 10. Basically, each NCU maintains arouting topology database with link weights reflect-ing the traffic over each link. When link weightschange it substantially updates flow to every nodeusing a broadcast algorithm described in Reference13. At the call set-up time, the source NCUcomputes a path from its local routing topologydatabase and computes the ANR field from sourceto destination and back. This information is sent tothe source user. The end-to-end call set-up packetflows over this path. Special care is given to thecase where the required call bandwidth is largecompared to the capacity of one or more links alongthe path. In such a case an extra reservation actionmight be needed during the end-to-end call set-upprocess. This will be described in the next section.

If no suitable path can be found between sourceand destination, the call will be blocked. The schemeprovides control of the path at the source andobtains relatively efficient paths. However, as theupdate of link weight takes finite time, there is thepossibility of some unnecessary blocking because oftemporarily inaccurate information in the routingdatabase. An efficient way of performing the linkweight update that uses the fast hardware copyfunction of PARIS is described in Reference 13.This new algorithm has lower time complexity thanconventional algorithms.5.15 Together with the speedof the network this algorithm reduces the problemof transient inconsistencies of topology databases.

As the full topology is known, the actual processof route computation for a given call is basicallycentralized. An extensive study of different shortest-

path schemes with different cost functions wasdone.J6 The study results show that schemes thatuse accurate link utilization give very small improve-ment in blocking probability compared to schemesthat uses only few threshold parameters. Conse-quently, the load update algorithm should not be

triggered very frequently.In addition to the primary route there is a

possibility for certain important sessions that asecondary route is calculated in advance. Once afailure is suspected over the primary route the callis switched to the secondary route with' minimalinterruption. In order to avoid common failures,the primary and the secondary routes should sharea minimum number of nodes.

In the following we describe a way for calculatinga secondary route which shares the minimum numberof nodes with the primary route. The secondaryroute is basically a route that is 'most likely' to beactive when the primary route fails. It is defined tobe the route that has the fewest nodes in commonwith the primary route. Let the set of nodes thatthe route r; traverses be labelled N(r;). Let us alsoindicate the primary route by r p and the secondaryroute by r;. The secondary route has the propertythat IN(rs)nN(rp)1 is as small as possible.

A/gorithm for computing secondary routes1. Select a large number, M, which is larger than

the number of links in the network.2. Assign weights to the links in the network

according to the following rules.

(a) For links which are on the primaryroute but not adjacent to the end nodes(source or destination) a weight of2M + 1 is assigned.

(b) For links adjacent to a node in theprimary route ( except the end nodes) aweight of M + 1 is assigned.

(c) For all other links a weight of 1 is

assigned.

3. In the above weighted graph calculate theminimal weighted path between the end nodes.This is done using a standard shortest-pathalgorithm. The computed path is the secondaryroute.

In the following we prove that this secondary pathshares the fewest number of nodes with the primaryroute. In fact, of all paths that share the fewestnumber of nodes with the primary route, the paththat we compute will have the fewest hops. Eachshared node in the second path adds 2M + 1 to thetotal path weight. This may easily be explained byadding M + 1 to the weight when entering a nodeof the first path and adding M + 1 when leavingsuch a node. This proves that the weight of thecomputed path is exactly 2Mx (the number of sharednodes with the primary path) + the number of linksin the secondary path. As M is chosen to be very

84 I. CIDON AND I.S. GOPAL

large and as we calculated a minimum weightedpath it follows that the computed path has the

above-claimed properties.

Link status monitoring

One of the aspects of the p ARIS system is thatit does not require a general purpose processor tobe part of the link adaptor. Consequently, thenetwork controllers themselves are responsible fordetecting the link status. In the following we describea protocol performed by the network controller todetect failures and reconnection of its adjacent links.

The network controller tests each of its adjacentlinks, whether it is operational or not, by periodicallysending a special message called a BOOMERANGmessage. The BOOMERANG message has a specialANR field causing it to be sent over that particularlink, to be sent back over the same link and thento be switched back to the network controller. Inaddition, the BOOMERANG message is copied atthe adjacent node by the local network controller .As long as the BOOMERANG messages return thelink is considered to be operational. If the controllerdetects that some (the exact number will be opti-mized to the network structure) sequential BOOM-ERANG messages did not make it back, it declaresthe link as not operational.

CONCLUSION

We have presented an overview of a design for ahigh-speed packet system for the transport ofintegrated traffic in a private environment. Wehave demonstrated that by off-loading most of thefunctions to the end points the task of buildinga high-throughput switching node is considerablysimplified. The PARIS architecture decouples thecontrol process from the switching process bylimiting the involvement of the control software toset-up and take-down of streams of packets ratherthan handling individual packets.

A prototype P ARIS network has beenimplemented at the IBM T. J. Watson ResearchCenter. The switching sub-system can support athroughput of l.2Gb/s and uses lOOMb/s privatefibre-optic links. The network control unit isimplemented using a PC. The hardware design isbased exclusively on off-the-shelf components andwe believe that it can be easily enhanced to supporttotal throughput around SGb/s.

ACKNOWLEDGEMENT

The authors thank their colleagues Alan Baratz,Jeffrey Jaffe, Joong Ma, Yoram Ofek, DavidSeidman and James Janniello for their numerouscomments and suggestions regarding this work.

The call set-up protocol

The call set-up protocol in PARIS differs substan-tially from the previous packet switched set-upprotocols such as the one for LPID that is used inReference 10. Unlike LPID routing, there is noneed in ANR for the intermediate node to maintaina logical connection number or the mapping fromthis number to the outgoing link. Consequently, thecall set-up protocols need to involve the intermediatenode in only a peripheral manner. This involvementis only to update the count of the number of callsin progress over each link. This number will be usedto determine whether or not another call will be

accepted.Before the call set-up can be performed the EPP

requests its NCU for a complete route to (and from)the destination end user, specifying the averagecapacity of the connection and a class of serviceparameter that specify the level of burstiness. Uponthe receipt of this request the NCU first locates thephysical destination through an optional directorysearch. Secondly, it computes the best route(s) tothat destination (or blocks the call if such a routedoes not exist). Next the process of the call set-upis initiated from the source via a call set-up message.The basic scheme includes a single end-to-endmessage (that carries the call ID, parameters andthe reverse ANR) that is directly sent to thedestination. This message is acknowledged via asimilar end-to-end message that is copied by allNCUs along the path. (Here we are using thehardware copy feature implemented by the PARISswitching subsystem). If the bandwidth requestedby the call is small compared to the residualbandwidth along the path, the probability of band-width overflow due to simultaneous calls thatattempt to capture the same residual bandwidth issmall and can be ignored. This probability is furtherreduced by the fast update process used in PARIS.13If the call bandwidth requirement is high (forexample for a high-quality video call), we provideenhancements to the basic call set-up protocol. Insuch a case the call set-up message is forced totraverse through some selected NCUs in order toguarantee that no bandwidth overflow occurs. Theset of these NCUs that are neighbours of the bottle-neck links is provided by the local NCU and isembedded in the call set-up ANR.

The call set-up process is being refreshed period-ically in order to hold the guaranteed bandwidth.By default, if no set-up message is received by theNCU after some time-out, the capacity of the callis released just as in the case of a call take-down.The call take-down process is similar to the call set-

up.

REFERENCES

I. D. Cohcn, ' A protocol for packet switching voice communica-

tion', Compuler Nelworks, 2, 320-331 (1978).2. C. J. Wcinstcin and J. W. Forgie, 'Expcricnce with speech

communication in packct nctworks', IEEE I. Selecled Areas;/I Comm., SAC-I, 963-980 (1983).

85PARIS

Authors' biographie.

Israel Cidon received the B.Sc (summa cum laude) andthe D.Sc. degrees from the Technion-lsrael Institute ofTechnology, Haifa, Israel, in 1980 and 1984, respectively,both in electrical engineering.

From 1980 to 1984 he was a Teaching Assistant and aTeaching Instructor at the Technion. From 1984 to 1985he was on the faculty with the Department of ElectricalEngineering at the Technion. From 1985 he is with IBMT .1. Watson Research Center .

His current research interests are in communicationnetworks and distributed algorithms.

Inder Gopal received the B.A. degree in EngineeringScience from Oxford University, Oxford, England, in1977 and the M.S. and Ph.D. degrees in ElectricalEngineering from Columbia University, New York, in1978 and 1982, respectively.

Since 1982, he has been a Research Staff Member inthe Computer Sciences Department at the IBM T .1.Watson Research Center, Yorktown Heights. Currentlyhe is manager of the Integrated Networks group. Hisresearch interests are in communication networks, high-speed packet switching, and in distributed algorithms.

Dr. Gopal is currently an editor for the IEEE Tra,JS-actions on Communications, guest editor for Algorithmica ,and guest editor for IEEE Journal on Selected Areas inCommunicalions. He was formerly a technical editor forIEEE Communications Magazine.

3. D. Conrads and P. Kermani, ' A feasibility study of using

store-and-forward data communication networks to transmitdigitized speech', IBM Research Reporl, RC 9157, October

1981.4. J. D. Atkins, 'Path control-the network layer of system

network architecture', in Paul E. Green (ed.), COmpulerNelwork Archileclure and Prolocols, Plenum, 1982, chap.11.

5. J. M. McOuilliam and D. C. Walden, 'The ARPA networkdesign decisions', COmpuler Nelworks, I, 243-289 (1977).

6. J. S. Turner and L. F. Wyatt, 'A packet network architecturefor integrated services', GLOBECOM, 1983, 2.1.1-2.1.6.

7. J. J. Kultzer and W. A. Montgomery, 'Statistical switchingarchitectures for future services', ISS'84, Aorence, Italy,1984.

8. M. Schwartz and T. E. Stern, 'Routing protocols', inPaul E. Green (ed.), COmpuler Nelwork Archileclure andPrOIOCO/s, Plenum, 1982, chap. 11.

9. M. S. Chen, 'Shortest paths in random graphs', IBM ResearchReporl, RC 11835, April 1986.

10. A. E. Baratz, J. P. Gray, P. E. Green Jr., J. M. Jaffe andD. P. Pozefsky, 'SNA networks of small systems', IEEE J.Selecled Areas in Commun., SAC-3, (3) 416-426, (1985).

11. I. Cidon, I. Gopal, G. Grover and M. Sidi, 'Real timepacket switching: a performance analysis', to appear in IEEEJ. Selecled Areas in Commun.

12. Joong S. Ma, ' A blind voice packet synchronization strategy',

in preparation.13. I. Cidon, I. S. Gopal and S. Kutten, 'New models and

algorithms for future networks', to appear in PODC'88.14. A. E. Baratz and A. Segall. 'Reliable link initialization

procedures', IBM Research Reporl, RC 10032, 1984.15. A. Segall 'Distributed network protocols', IEEE Trans.

Informalion Theory, IT-29, (I), 23-35 (1983).16. C. T. Chou and S. Zaks, 'Issues in route computation', IBM

Research ReDorl. to appear.