Hans Eriksson

: The MulticastBackbone

Tbt' first thing many researcherslike me do when they come towork is read their email. The sec-ond thing on my list is to checkwhat is (Jn the MBone—theMitlticast Backbone, which is avirtual network on "lop" of theInterrrct providing a multicastingfacility to the Internet. There mightbe video from the Space Sbtrttle, aseminar from Xerox, a plenary se.s-sion from an interesting conferenceor a software deinorrstration for theSwettisb prime minister.

It all started in March 1992 whentbe fir.st audiocast on the Internetlook place from the Internet Engi-neering Task Force (IETF) meetingin San Diego. At that event 20 siteslistened to the audiocast. Two yearslater, at the IETF meeting in .Seattleabout biM hosts in 15 countr ies trinedMl to the two parallel broadcastingchannels (audi<j and video) and alsotalked back (audio) and joined tbediscussions! The networking coiritriti-nity now takes it for granted thai ilieIpn F meeiiugs will be distributed viaMBone. MBone has also been used todistribute experimental data from arohot at the bottom o! llic .Sea of ("or-te/ (as will be descr ibct! later) as wellas a late Satui day night feature movieWAX or the Discoveri of Television.4mong the Bees by David Blaii\

.As soou as some cnrcial loots ex-isted, the usage just exploded. Manypeople started using MBone for con-ferences, weather maps, researchexperiments, to follow the SpaceShuttle, for example, .At the SwedishInstitute of Comptiter Science (SI(;S)we saw our contribution to the Swed-ish University Network SUNET. in-crease from 2(if;B per month in Eeh-riiarv 199:i to (i9(lB per mr)nth inMarch 199;i. 1 his was mainly due tomulticast traffic as SICS at that timewas the major connection point be-tween the U.S. and Eirrope inMBone.

e has also (in)directly heerr

the cause of severe problems in theNSFnet backbone, saturation ofmajor international links renderingthem irseless as well as sites beingcompletely fli,sfonii('( ted due toInternet Connection ManagementProtocol (ICMP) responses floodingthe networks. We will expand on thislaler" in this article.

Multicasting BackgroundWhen we talk about MBone we some-times mean the virtual network thatimplements triulticasting. sometimeswe refer to the applications that runon top of MBone (vat. nv, ivs, for ex-ample), and often we mean every-thing. We will conre back to tbe appli-cations that are in use on MBone laterin this article, but for now we will con-centrate on the MBone proper, themulticasting virtual network.

First let us define what is meant bythe different types of "casting." Thetistial way packets are sent on theInternet is unicasting. that is, onehost is sending to airollrer specific sin-gle lH>st. Broadcasting is when onehost sends to all hosts on the samesuhnet. Normally, the routers be-tween one stibnet and another subnetwill not let broacUast packets passthrough. Multicasting is when onehost sends to a group of bosts.

On the link level (e.g., Ethernet)mrdticastirrg has beeir defined lor*some lime. On the iieiwork level (In-terface Protocol or IP) it started withthe work of Steve Deer ing of XeroxPARC when he developed multicastat the IP level |:5|. Ihe IP addressspace is divided into dillcrent classes.An IP address is four bytes and theaddress classes A, B and C divide theaddresses into a network part and ahost part. The difference between tbeclasses is the balance between bits des-ignating network and bosts. Class Aaddresses bave one byte for llie net-work and three tni" host, Ei addresseshave two bytes for each, and class Caddresses bave three bytes for the

network and one for' thf host, lo dif-ferentiate between the classes, startwith 0, 1 or 2 Ijits that are set followedby a zero bit, C:lass A addresses startwith binary "0" and are in the rangeO.O.O.O to 127.255.255.255, class Bstarts with " 10" with a range of128,0.0.0 to 191.255.255.2.'jri. andclass (; starts with "110" with a rangeof 192.0.0.0 to 223.255.255.255. Notall addresses are available fbr hostaddresses, however, as some ar e de-fined for" specific uses (e.g., broadcastaddresses). Class I) is indicaled by"II10" at the start, giving an ad-dress range of 224.0.0.0 to2:i9.2ri3.25.f>.23.'j. This class has beenreserved for nrulticast addiesses.

Wherr a host wishes to join a mirlti-cast group, that is, get packets with aspecific multicast address, the hostissues an Internet (iroup Manage-nrent Protocol (KiMP) reqiiesl. I henruiticast router for that subnet willtben inform the other routers so thatsuch packets will get to this subnetand eventually be placed on tbe local-ar"ea network (IAN) where the host isconnected, Freqrrently, the localrouter will poll the hosts orr the IANif they are still listening to the mrilti-cast group. If not, no more sucbpackets will be placed onto the IAN.

When doing multicasting irtilizingMBoue. tbe sender does not knowwho will receive the packets. Ihesender just setrds to an address and itis up to the receivers to join ihaigroup (i.e., multicast address). .-\n-oihfr- style of nrulticasting is wlrerethe sender" specifres who should re-ceive the multicast. I his gives morecontrol over the distribution, hut onedrawback is that it does rrot scale well.Having thousands of receivers is al-most inrpossihie io handle this way.Ibis second style of multicasting hasbeen rrsed in ST-2 [6, 81,

MBone TodayAs previously mentioned, MBone is avirttial network running on "top" of

llu' Ititcniet. MBotic is composed ofiK'tworks (islatuis) that support tiuilti-cast. On each of these i.slands, there isa im.st that is running' tlic Diniiilt'dniultitast routing tlfiiioti. 1 hemt'outed's are cotitiectecl uitli oticanother via ttnirast tunnels.

Ill Figure I. we have three islandsof Ml^oiie. V.MU isiaiifl loiisists ol' alocal tictwork connecting a nuniher olrlienl hosts ("C") and one host rttn-ning nn'outed ("M"). The mroiited-hosts arc linked with poitit-to-poinittiniiel.s. llie lliick luntiels ate thepiitnaiy lecds with the ihin tunnel asa backup.

Basically, a nitilticast packet will hesent from one client who pins ihepacket on the !(Kal suhnet. Thepacket will he picked up by thenuouted (or that stilinet. Thennouted will c<tnsiilt its toiititig tablesand decide oiilo which tunnels thepacket ought t() he pla(ed. At theother end of the nintiel is anotherinrotited that will teceivc the nuttti-cast packet. It will also examine itsrouting tahles atui decide il thepacket shonld be foi warded otiio anyotiui tuiniels. The mrotut-d will also(lu(k il ilicre is any client oti its sub-tiet that ha.s subscribed to that grotip(tnulticast address) and if so, put itonto the subnet to be picked up bytbe (lieiil.

TunnelsWhen .sctiding tbe multicast patketihiough the ttmnel. tbe multicastp.K kets nitist be repacked. I lu te aretwo methods ol cloitig tins, addingthe Loose Source and Record Route(I.SRR) IP option and encapsulation.The Ihst inipleineiitaiiotis (ilmrouted used the I.SRR II' o|»tion.Mrouted modified the mtilticast data-gram coming frotn a client by ap-pending ati II* KSRR oplioti wherellie inulti(,ist adthess was placed. 1 heII' (U-siination addiess was set to the(luii(ast) addiess ol the mtouted onibe other side of the tunnel. I herehave heen some problettis witb thisapproach (as will be dt s< t ibed later)that pr(nii|)ted the iinplemetitation ofetuapstilation. In tbis mctbod theoiigitial nniltiiast datagram will bepur iiilo ihe data part of a tiortiial II*datagram thai is addressed to thetn routed on the other side oi thetunnel.

The teceiving niiouted will stripoil the encapsulation and forward thedatagi"am apptopriately. Both thesemethod.s are available iu (he (uirentimplementations.

Kach tunnel has a tnetric and athreshold. The tnetric is used fbrrouting anti the threshold to litiiit thedistribution scope for nniiticastpackets.

The metric specifies a touting costthat is ttsed in the Distance VectorMulticasting Rot it ing Protocol(DVMRP). lo itnplemetit the pri-tnary and backup tunnels in Figtire 1,the tneti ics could have been specifiedas I (or ihe thick tutincis and ! for thetliin tunnel. When Ml gels a miiUt-cast packet hom one of its clients, itwill compute the cheapest path toeach of the other M's. The tunnelMl-M:t has a tost of .'i, whereas thecost via the other tunnels is (I -I- 1)2.Hence, the tunnel Ml-M;i is nt)rniallynot used. However, ifany of the othertunnels breaks, the backup Ml-M;iwili be usetl. Howevet, siiue DV'MRPis slow on propagating cbaiiges itinetwork topology, rapid changes willbe a prol)letn.

"!lie llin:\lii)ht is ibe minimum time-to-iive (TII.) tiiat a tnuiticast tiata-gratn needs to be torwartietl onto agiven tuiinei. When sent to the net-work by a ciient, eacii muiticast|>acket is assigned a specific ITl.. l"oreach mitiuted the packets pass, the' !T1. will be decremented by 1. If a|)a( ket's remaining TTL is itjwer tbandie tiiiesiiold of tbe tutinrf tbatDVMRP wants to sentl the [)acketonto, the packet is dropped. Withthat tnechanism we can litiiit tliescope ibr a mtiiticast transmission.

In ihe beginning I here was noprtintug t)f the multicast tree. Ihat is,every multicast datagram is sent toevery mrouletl in MBone if it passesthe diicsiiold limit. Ihe otily pruningis tlone at tbe leaf stibtiets. wbeie ibeiocai mrouted wiii put a datagramonto the iocal tietwork tjniy if there isa (iieiit host tiial iias joined a particu-lar muiti<:ast group/adtiress. This iscailed truncateti ijtoaticast. As theMBone giew, problems surfacedwiiicii we wiii discuss later. "I'heseptoblems protnpted work on properpriming oi 'the tnulticast tree as weiias work t)n other techniques for mni-licasting [i , 5, 9j. Pruning as imple-

mented in the MBone today worksroughly iikc tbis: If a mrouted gets amuiticast packet ftir which it has noreceiving ciients or tutnieis to fot-ward it to, it wili drop the patket luitaiso send a signai upstteam that itdoes not want packets witii that ad-dress. Tile upstream nirt)Uted wiiinotice tiiis aiui slop seiiditig packetsthat way. If tiie (iowtistream mttiutedgets a ciient that joins that prunedmuiticast gtotip. it will sigtial its up-stream neighbtus that it wants tbesepa( kets again. Rtgtilarh ibr inlbi nta-tioti will be flusbetl .tiitl pat kets willflow to every corner of MBt)ne utitilpushed back again.

Managementr i u t e is no "neiwork |)rovider" ofibe MBtMie. In ibe spiril til tbe Ititer-net, MBt)ne is lt)oseiy cotirditialetl viaa mailing list. When end usets wanttt> t:t)nntxt tt) MBone. tiiey are en-couiagetl lo contact their tietworkprovider. If tbat netwt)ik jirovitier isnot participating in MBtine and Ibrst)me reason does nt)t want tti, a tiin-tiei (an be arrangeci tt) another poitititi MBtdie.

From time tt> time, there have beenmajor overhatiis of the ttipology as^ilione has grown. Usually this hasbeeti protiipled l)\ ati upcominglKIF meeting, i iiese tneettngs put abig strain on MBotie. Tbe IETF mul-ticast tralTic lias been about 100 tollOOKi) per second with spikes up tofjOOKii per secotid.

Applications,Siu( (• M iiotie was set up. a number ofwide-ranging appiicatit)ns have sur-facetl. We have seen the astronautsrepairing the Huiibie telescope, iis-tened to semiuLU s aud st-t n tars tomeand gt) at the Btjit. Beranek, antiNewman parking iot in Boston. I willgive an t)veiview of stime t>f tiieevents thai have used MBone in someway. But first I wili mention some oithe iiKJst popuiar ttjtiis ft)r usitig theMBone. This list is by no tneans com-plete as new applications ap|}earregularly,

For atidit) we have vnl (visuai autlit)tooi) hy Steve McCanne and Van Ja-cobsen of Lawrence Berkeiey Labora-tory. The tuiH)! (netwink voice tertni-nai) i>y Henning St iiul/rinne t)fAT^-T/Beii Labt>ratt)ries is anotherautlio tool.

5 6

Video tools are ius (Inria videocon-ferencing system) by Ihieri y Tluirle-iti of INRIA in Sophia Antipoli.s.Krance and nv (network video) byRon Krederifk of Xerox PARCi.

IVh (wbile board) by McC anne andJacobsc'ii piovides a shared thawingspace and is especially useful for pre-sentations over the MBone. Wb tanimport slides in PostScript and thespeaker (an make small annotations(hiring the lectuif.

Kigure 2 depicts tbe sd (session di-recloiy) by McClanne and [acobsen.Sd oilers a convenient way of an-nouncing "sessions" tbat will takeplace on the MBonc. Wben creatinga session, you specify the multicastaddress (an umiserl address is sug-gested by sd) and the various toolsthat are used. Other people can thenjust click "Open" and sd will start alllhe necessary tools with appropriateparameters.

When this snapshot was taken, lheSlCiC'RAPH conference was takingplace. As a special event at tbat con-ference, children were invited lo lalkwith pe()pl<' on lhe MBone. Ihise\ent is bighliglited in the sd .snap-shot, tiding up in the list we ha\cRadio Free Vat. This is the MBcmc"laclio" station where anyone on tlu-MBone can be the "disk jockey."Next up is MBone Audio, which is thrcommon chat channel of the MBoncKveryone is free To join and siari .1discussion about any subjec t, Becau>fMBone spans about 16 time zones,no! everyone is at tbeir workstationwben you ask "Is tbere anybody ouitliere?" [71, but tbere is always some-one out tbeie! lhe (ilobal MappingSatellite (GMS) sessions are pictureshnm a satellite above Hawaii. Thepictures {composite, inhared or \isual spectra) are sent oul tising iwtii(Image Multica.st Client) by WinstonDang of tbe University of Hawaii,Second to the top is the Bellcore Win-dowNet. It yon tuned in to tliis ses-sic)n, you would see the outlook froma window from Bellcore. At the topwe have not a session, but a plea. Asaudio and video consumes a fairamount of bandwidth and MBone isglobal, tebroadcasting your favcmtelocal radio station onto MBone will[)tu .\ hard strain on many networks.\\V will come back to this problemlater in tbis article.

c

c

c

LBL Session Directory v1.19hans@hydra.sics.se

** Please don't start a radio sessionBellcore WindowNetGMS-4 CompositeGMS-4 IRGMS-4 VISMBone AudioRadio Free VatSIGkids @ SIGgraph

Live feed from SIGkids area at

SIGgraph 93.

©224.2.176.182, tt l 127

Lifetime: from 01:00 MET DST until Tue

Aug 10 01:00 MET DST

•

T

•

T

Open New Edit Delete Quit

Not sbowri in ihis particular snap-shot, but a frequent and \ery popularguest on MBone are the Space Shut-tle missions. The NASA select cablecbanne! is broadcast onto the MBoneduring tlie Iliglits. 1 he pictures of theastionauts travel a long wav and tra-verse many dillerent leclinologies

Figure 1. MBone topology—islands, tunnels, mrouted

Figurea. sd—session directory

57

bt'lnie appearing on the screen ol'your workstation. But it works!

A different type of event was men-tioned earlier, the lOOI JASON Proj-eel [4]. Woods Hole OceaiiogiaplikInstitulton provided software for Sunand Silicon Ciraphics workstations soanyone on ihe MBone could followthree luiflerwaler \ehicle.s on theirtours in the Sea of C.o'te/. Positiondata and some pictut es were continu-ouslv disiTibiiied over the MBone.Bt sidi' l)eiiig interesting for scientistsin other fields, it wa.s very valuable loroccanographic researchers to followthe experiments in real time and givefeedback immediately.

The multimedia conference con-trol (mmcr) by Eve Schooler of Uni-versity of Sotithern C^alifornia (USCl)/InlbriiiHtioii Sciences Institute (ISI)goes beyond this simple supportgiven by sd. We will include moreabout this when discussing theM M U S K ; protocol.

The popular Mosaic package fromihe National C^enter Ibr Superrom-piitcr Applications (N( SA) is beingenhanced by people at University ofOslo, Ihe idea is to use Mosai< forlectiu'es atid let the speaker nuilticastcontrol inlbrmation to the Mosaicprograms used by (he student.s.

We also have tbe media-on-demand server created by AndersKlemets of Royal Institute of Tech-nology in Stockholm, Sweden, whichoffers imicasted replays of sessionsihat have heen niulticasted on theMBone.

I his is merely a snapshot of some<jf the developments taking place inthe MBone commtuiity. New ideassurface often and implementationsfollow close behind.

ProtocolsAll lialfic in MBone uses User Data-gram Protocol (IDP) rather than theusual IVansport Control Protocol( rCP). "i'C.P provides a point-to-pointconnection-oriented reliable bytestreatn protocol, whereas I'DP is jitsta transport-level envelope around anIP packet witb almost no controlwhatsoever. One reas<}n for not usingT("P is tbat the reliability and flowcontrol mechanisms are not suitablefor live audiotasting, fbr example.Occasional lo.ss of an audio packet (aswhen using liOP) is usually accept-

able, whereas tbe delay for retrans-mission (when tising TCP) is not ac-ceptable in a interactive conference..Also, TCP does not easily lenfl itsell toinuititasting. One problem thai nnistbe lesolved is that UDP packets maybe duplicated and reordered (besidebeing diopped) when transmittedover the Internet.

On top of U [)P most MBone appli-cations use the Real-Time Protocol(R rP) developed by the Audio-VideoTransport Working Croup within theIKTF. Kach RIP packet is stampedwith timing and ,sec|uencing infbrma-tion, Witli appropriate buffering atthe receiving hosts, this allows theapplications to achieve continuousplayha(k in sj:ite of varying networkdelays.

Each lorin of media can he en-coded and compressed in severalways. Audio is usually encoded inPCM (Ptilse Code Modulation) atHKH/ wilh 8-bit resolution giving()4Kb per second bandwidth foraudio. Induding packet o\eil!ead itlaises to about 73Kb per second. Byusing (Jroupe Special Mobile (CiSM),a cellular phone standard, one canget down to ahoui lHKb per secondincluding oveibead.

Video is more demanding, I be ivstool uses the CC^ITF (ConsultativeCommittee of Internationai Tele-phone and Telegraph) standardH,26I [2] whereas the nv tool uses atinique compression scheme. It ispossible lo limit the amount {)f hand-width that shotild be pioduted inboth toois. The usual baiidwithh set-ting is 128Kh per second. How thistranslates into qnalily depends on thekind of scene thai is captured.

ExperiencesDuring tlu- litciirne of MBone, a fairnumber ol problems have been en-countered. Some are inherent tonuiiticasting in general, and some aremore specilic to the current imple-mentation of MBone,

A numher of prohlems that iiavesurfaced duringoperations of MBonewili be discussed in this section. Someprohiems iiave a (iiiect hearing on tiieMBone implementation, other ptoh-lems have been discovered recentlydm ing the use of MBone.

Bandwidtfitiurrentlv there are three more-or-

iess permanent sessions going on inMBone. There is one audio and videochannel for free-for-aii use and thereis Ratiio Free Vat. in adfiition to theIIVIF meetings, whitli are transmit-ted three times per year, severalmajor conferences and workshopsare being iransmitteti onto the net.such as )ENC-93 and some IETFworking group meetings. We havealso seen President Clinton and Vice-president Core on MBone, and wehave already mentioned the jASONpr()ject and tlie Space Shtittle.

MBone in its present form shouldbe viewed as one singie resource.Oniy in a few piaces can it handlemore tban one video channei to-gether with andio. The IETF tries tomake two video and fbur audio chan-nels but does not always accomplishthis, even if the best "networkers'" itiibe Internet put in tbeir best efforts.So far, we have not had any niajorcollisions of major events. The coHi-sions that iia\e occurred have heenresolved after some biief discussions,Essentiaiiy it is a first-annotince-fhsi-serve scheduiing. .As MBone increasesin popularity, one can expect morecoiiisions and tiie pressute fbr a par-ticular slot wiil increase.

Some of the success of MBone isdependent on the "courtesy" of TC.P.When someitne siarts sending audioonto a fully loaded Internet link, itwiii cause packet iosses for many ofthe connections that are rtuining onthat iink. liiey are ustiatly TCP con-nections and tliey wiil back offvvbenpacket iosses occur. L'DP-based audiodoes not have any such mechanismand wiii effectively take tiie band-width it neecis.

On severai otcasiotis end it.sershave started a video session with ahigh time-to-iive (TTL) and subse-quentiy swamped the network with acontinuous stream of :U)() to ,"i()()Kbper second. Tbese tisers have notbeen maiicious. Sometimes the pro-gram has jitst been started with "-tti I16" itistead of "-tti ifi" with tiie effectthat it reaciies most parts of tlieMBone instead of just the focai part.At other times, the tisers have notreaiiy been aware of what "2.5fiKb persecond" reafiy is netwise. Very fewiinks in the Internet can handle thatload withotit severely disturbing nor-mal traffic, I'sualiy after the mistake

58 l<J94/Vnl.:i7, Nr, H .

has been pointed out. the users havestopped their transmissions. Tileproblem is that with ihe new videoand audio appluatiotis the mistakeshave severe consequences and withiiiiiliicasting in MBone, the consc-(|iien(es are spread glohally. It willtake some time befbre tbe user com-munity gets a tee! fbr bow mucht)andwidth video and audio takes,Kxisting applications like ttp can . ilsouse a lot oi bandwidth, but ifie back-off mecbanism of TCP ensures a fairsplit of resources, wbicb a L'DP-basedapplication does not.

Lacking a tbic-grained resourceallocation mecbanism, a way to put alimit on tbe bandwidtb usage of atunnel could be very belpful. Tbatwould maki' many network providersa lot less nervous aboui letting multi-cast traflk loose.

Thresholds on Tunnelsl..ukiiig mullitiisl iree pruning, tbeonly wiiy to limit tbe scope of a multi-cast datagram is by using tbresbolds.If a datagiam bas a TH. greater thantbe tbieshold. ii will be ioiwardedonto tbe umnel. Ibresholds rangebetween 0 and 255. The tbresboldlevels cbosen on tbe tunnel tries le-llect boih a gfograpbic partitioning(e.g., keeping a local conlerence local)and a cboice of traffic (e.g.. testrictingvideo more than audio). But express-ing two dimensions of cboices in oneinelric always introduces some trade-offs.

'f be guidelines establisb tbat trafficwitliiTi (me site should be sent witb aitl **i' H). witbin one "loinmunity" !i2,ami global traffic sliouki bave 127.n~lie IETF transmission plan is sbownin Table 1.

Hie table says tbat if you only wantto get audio channel t witb tbe (iSM(onipression. your tunnel sbouldbave a tbresbold of 224.

The tbresbolil mecbanism is a verycoar.se method to limit (ralfic. Wilhthe current W'VV plan, tbere is noway y<jn can use your 256Kb per sec-ond link to join tbe session tbat isbroadcast on (baiinel 1. To get P(1Maudio I you will open up for botbCSM andio t hannels, giving a total of= i{)5Kb per second, To get Video Iyou will also get tbe PCM audio 2summing it up to ~.' 10Kb pei- secondIbr virleo and audio from (bannei 1.

Table 1. Time-to-live (TTL) and thresholds from the IhternetEngineering Task Force

Traffic type

GSM audiu 1C.SM ;)iidio '2PCM audio 1PCM audio 2Video 1Video 2locat event audiolocal event video

TTL

255223f911591279563

:n

=:kb per second

15157575

130130

2:250^250

thifshold

22419216012S966432

1

Wben trne pruning gels widely de-ployed in MBone it will be possible toget only what you ask for.

Tunnel Fan-OutSome mrouted bosts bave a fair ninn-ber of tunnels. Tbe top ones bad 1 ftunnels (early June 1993). However,not all of tbem are primary tutmels.For example, liydia.sics.se (a SPARO1+) bas 10 tunnels, but only 5 ofihem are primary. The aggregatetraflic from an IETF tneeting roughlygenerates one packet every 4tns.Looking at how much time it takes toforward a multicast packet we findthat a SPARC 1 -I- needs = 1.0ms and aSPARC: IO needs ==0.6ms. This sug-gests tbat hydra.sics.se is saturateddiuing lEFF sessions. I bis will sbowup not only as dropped packets, butalso as dropped timnels. As hydra willbe busy, in kernel forwarding of niul-ritasi packets, tbe mrouted will notget any cycles to rim its business.Eventually, tbe peers at tbe otherends of tbe tiumels will tbink tbatbydra is dead and bence tbey will re-ioi;ie tbeir packets. Soon after tbeload bas gone down on bydra,mrouted will get some cpu cycles andtalk to its peers and the overloadingwill start again.

A related problem is saturation ofthe local p:thernets. FlX-West bad 15lunnels diuing tbe IF.IK meeting inMarcb 1994. With an IFTF load of=5()0Kb per second it results in= B.5Mb per second pusbed over tbeFIX West Ftbernet. Fven witbout tbeMBone traflit, tbat Ethernet is al-ready busy. Some mrouted operatorsuse the cpu overload as a means tolimit tbe impact on tbe hx'al network.That is. if yon have a SPARC I + as anni)uted bost. it will nol push more

tban 1.000 packets onto tbe net,probably much less. This is a danger-ous practice unless you are the onlyentry point to a part of Mfione. Asstated previously, wlieti your luniielgets declared dead, MBone willchoose another route if possible utililyour mrouted gets its breatb ba<k.Tbis rcsnits in beavy route flapping,wbicb becomes a global problem. Uyou are tbe only patb to take, trafficwill just stop for a wbile. wbicli is alo<al problem only.

MBone Tunnels vs. internet LinksTunnels are set up along tbe links ofthe underlying real tietwork. Bulwben a link fails and tbe underlyingnetwork does a rerouting, tbe tnnnelsstay and become less optimallyplaced. As an example, traffic fromSweden to the United Kingdom (I IK)would normally go via tbe tunnelfrom Stockholm lo Wasbington overa Tl link and tlien take the ttitinelWasbington to London over ariotberLl link. When tbe first link malfunc-tioned, traffic was rerouted via Am-sterdam and London and tben to tbeU.S. 'Lhe multicasi traffic fVotii Swe-den to tbe UK ended up going Stock-holm to London to Washington andtlien back to London. Eventually, wemade a manual reroute of tunnels.Putting multicast routing into tberonters enables the mtilticast louiinglo follow the unicast rotiting. 1 heie isa proposal for an extensioti to theOSPF (Open Shortest Patb First)routing protocol to also incorporatemnlticasting [51.

MBone as a Bug TriggerDuring the lEl F meeting in Wasb-ington, D.C. in November 19*12.tbere were prolilems with tbe NSFnet

imijx l9<H'V.il.:i7. Nr.H 5 9

were due t<t the niiilritast traf-I'u t(»miiij>; iidin the I1-V["K ineeting. Atihai time ilu- iiiniu'ls were all usingihf Icjse soiutc route option (LSRR).In modern router technology, park-els are handled hy the interface cardsas muth as possible. Packets with 11'options, however, are usually for-w;n ded to the main cpu tor handling.A\ that time, the IETF meeting wasgene I at ing two audio thaimels of=7r)KI) per second and two videochannels of = 130Kh per second. Thiswas about 400 packets per secondibat was sent Irom tlie IETF site toihc mroult'd at (iornell. Thatinioutc'd in turn, ted a inunber ollunnels so the traffic from Cornell-into FNSSI3^ at Ilhaca was above1,(100 packets per second that bad tobe bandied by tbe main cpu. Addingto tbat, a great number ol' ICIMP-unrea( bable messages were gener-ated. The cpu was ha\ing a diHiculttime when regular routing updateswere added. Ibi.s lead to routinglimeouts wben other networks batlprol)lems due to excessive MBonenaflii. A number ol actions weretaken lo lix tbese problems. One oltbe immediate actions was tbe disa-bling of one video cbannel to lessenibe load. In tbe alteiniatb, some inel-I'lciencies ol routing updates werelixed, for example, ECiP (External(Gateway Protocol)-derived routes willnow be aggregated into a single B(il'(Border (Gateway Protocol) updateuiessage. I be most important changeill MBone was the use of true encap-sulation Instead of LSRR option foribe tunneling,

lluring the packet x'nko workshopal M C ; N ( ^ Van |ac(>bsc)n observed a

pbenomenon in wbifb It ,seemed thatrouting updates severely impactedthe audio transmissions. The conges-live loss rate was about 0.5% buievery !10 seconds he observed bugelosses {509, to 8ri9f) for about 3 sec-onds, Jatobson (oncluded that it wasdue to tbe I.SRR ojjiion protessing(ompeting witb routing updates. Notonly does this afVect MBone trafHc,but also otber traffic stich as pingsand traceroutes.

Many hosts and routers do nothandle multicast trafiic properly,("jften they respond by sending anK'MP redirect or network unreach-.iblc. I be,se responses are not in ac-

cordance witb tbe IP specifications.I bis is usually not a problem until weliave several sue b liosis reading withICMPs to a ntunber of audio streamsof about 50 packets per second. Thenany network tends to gel Hooded witbICMPs. It has happened that a sitewas disconnected from MBone due toa "screaming" router. Over time, thisproblem bas diminished as routervendors update llu'ir software. Also.wiih the new encapsulation tunnels,the KlMPs will be sent to tbe last tun-nel endpoint, not the entire routeback to tbe original sender.

ConclusionsMulticasting ciu\ be a dangerousbeast, but it also carries tbe promise(if very useful applications. As an in-dication of tins MBone usage is in-creasing very rapidly. As a probabl)'unintentional side effect, it has alsobiougbt out some bugs in some rout-ers and hosts in tbe Internet. BeforeMBone can be provided as a regularwidespread service, some issues baveto be afldressed.

Some of tbese issues are still rlifli-cuit researcb issues, like resourcecontrol and real-time traffic control,(^tber work is directed toward bettermanagement hooks and tools andincorporating multicasting in theInternet routers. Maybe there arebetter technologies for multicastingthan those currently used in tbeMBoue? Tbe IDMR (Inter-DomainMullicasl Routing) working group inIETF is working im this.

MBone has enabled a lot of appli-cations. One problem wben siariingthe a[J|)lications is tbe cjuestion olwhat addresses should be used, Pi(k-ing one randomly will be Hne forquite a while, but eventually whenMBone gets more c rowded .somemet hanism has to be put in place forallocation of multicast addres,ses andport ntimbers.

As MBone is today, the .sender hasno control, or implicit knowlerlge ofwbo is listening otit there. A receivercan just "tune in." like a tadio. Someapjjlications would want some kind ofinforinaiion about wlio is listening,for example by asking MBone whichhosts are ctirrently in a particularmulticast group. Tbere are mecha-nisms in some applications for end-to-end control oiWho is listening (i.e.,

encryption) but there is st) far no((»mmon architecture for this.

Wben tbe going gets riiugb and alot of pac kets are droppect. some ap-plications would be belped by somefeedback on tbe actual perlorinancfof tbe network, A video applicalionconid for example, stop sending rawHD rV data when only 2''/< make it tothe receivers and instead start send-ing slow-scan, heavily compressedpictures.

We look forward to the next roundof flevelopments as ihe MBone con-tinues to evolve. Q

References1. Ballai'dic, A., Krancis, P., (lixmcroft. ),

(^orc-baseti tret'S—An itirhilft tiirc linscalable inter-domain multicast rout-ing. ACM SKkomm 9?. pp. 8,5-9,T.

2. tConsultative (Committee of Interna-tional 'Vclephnnf and IVIegraph(CCriT) Rccomnicndalioii H. tiK

3. DecTinj;;, S. (Insl I'xlensiims lor IP iiuil-ticiiMing. k l - ( ; i l l 2 . Any- HW).

4. Mallei, A. Rcnnut- scit'iuc ovfi tlit*MBimc cUiiiiig ihr lWVi J,\.S()N proj-ect. \V(K>ds Hole Oceanographif Insti-tiition. Woods Hole, Mass.

5. Moy, J. MiiUicasl routing extensionsfill O,SP1-. Commun. ACM 77. H (Aug.] 904),

6. Parliktge, C. and Pink, S, An liiiple-nientalion of ilic Revised [nleinetStream Prciiocol SI-'.*. In/fnifhi'. Rfs.Exp. (March 1!)9'2). pp. 21-'A.

7. Pink,F. Dmit Side of Ihf Mnoti.8. Tnpolcic, C. Kxperiniental 1 nternet

slieam proKKol. Version 1 SI- I I . RFC1190, On , !W)0.

9. Zlunig, L., Deering, S.. Kstrin, D.,Slu-nker, S. and /iippala, I). RS\'I': Anew rfsoiiire Kc'ScTV.ilion Pronxol.IFAA: NHumk (St-pt, lO'tS)-

About the Author:HANS ERIKSSON is H rt-scanher al iheSwedish Instiuitc of Computer Science(SIC,S), localed in Kisla. outside Stock-holm. Current rescaicli interests iTuhifieI'ompuier nehvnikiiiK .uid (list lilin ledmultimedia a[)})lr(:itions. Author's Pres-en( Address: Swedish Institute ot (iom-[jiitct Stteiut:, Box I2(W. Ifi4 28, Kista,Sweden: email: hans(S'sics.se

I'crmis.sLon [ii <'O|)y M'iiliniir It'i' ,ill o r piiii of iliism;iteri;il is ^riinit ' i l pii ividt-d ilcii \\\f lopit-^ :ir't.'iKii IDLKIC III' (listijlnired lor d i i f t t c i i innicicuil[i<Kai)[ag('. ilie ACM copyri^l t i iioiict' a n d ilii'tirif o l ' l l i f |)iLl>iircilion ^Liiil ils ILLIC iipfx-iir. ^iiiiliiiiliK' l\ given dial lopy i i ig i.*' liy pt'iiliissiiiii iililif \ssoi iaiion lin (i i i i i i | i i i i ing M;KliiiU'iy. I nciipy oilicMvisc, i)T I" icpi i l i l i sh . i f t | i i i i f s w It-t-;iii(l/(ir s])f(iti( piTiTiissifui,

©ACM t)002-0782/94/0800 $3.50

60 i<)'t+/v<il.:)J.