16. incremental deployment service of hop by hop multicast routing protocol

14
IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 14, NO. 3, JUNE 2006 543 Incremental Service Deployment Using the Hop-By-Hop Multicast Routing Protocol Luís Henrique M. K. Costa, Member, IEEE, Serge Fdida, Senior Member, IEEE, and Otto Carlos M. B. Duarte Abstract—IP multicast is facing a slow take-off although it has been a hotly debated topic for more than a decade. Many reasons are responsible for this status. Hence, the Internet is likely to be or- ganized with both unicast and multicast enabled networks. Thus, it is of utmost importance to design protocols that allow the pro- gressive deployment of the multicast service by supporting unicast clouds. This paper presents HBH (hop-by-hop multicast routing protocol). HBH adopts the source-specific channel abstraction to simplify address allocation and implements data distribution using recursive unicast trees, which allow the transparent support of uni- cast-only routers. An important original feature of HBH is its tree construction algorithm that takes into account the unicast routing asymmetries. Since most multicast routing protocols rely on the unicast infrastructure, the unicast asymmetries impact the struc- ture of the multicast trees. We show through simulation that HBH outperforms other multicast routing protocols in terms of the delay experienced by the receivers and the bandwidth consumption of the multicast trees. Additionally, we show that HBH can be in- crementally deployed and that with a small fraction of HBH-en- abled routers in the network HBH outperforms application-layer multicast. Index Terms—Multicast, routing, service deployment. I. INTRODUCTION I P MULTICAST is facing a slow take-off although it has been a hotly debated topic for more than a decade. Many reasons are responsible for this status. The IP Multicast architecture is composed of a service model that defines a group as an open conversation from sources to receivers, an addressing scheme based on IP class-D addresses, and routing protocols. In IP Multicast, any host can send to a multicast group and any host can join this group and receive data [1]. The IP Unicast service model is also completely open, but in multicast the po- tential burden caused by unauthorized senders is multiplied by the multicast group size. The IP Multicast architecture is completed by group ad- dressing and routing protocols. A multicast group is identified by a class-D IP address which is not related to any topological information, as opposed to the hierarchical unicast addressing model. Therefore, multicast address allocation is complex, and multicast forwarding state is difficult to aggregate. Currently, Manuscript received June 17, 2003; revised December 3, 2004, De- cember 6, 2004, and June 20, 2005; approved by IEEE/ACM TRANSACTIONS ON NETWORKING Editor R. Cohen. This work was sponsored by CNPq, CAPES/PRODOC, COFECUB, and RNP/FINEP/FUNTTEL. An earlier version of this paper appears in the Proceedings of ACM SIGCOMM’01. L. H. M. K. Costa and O. C. M. B. Duarte are with the Grupo de Teleinfor- mática e Automação-Universidade Federal do Rio de Janeiro, Rio de Janeiro, Brazil (e-mail: [email protected]; [email protected]). S. Fdida is with the Laboratoire d’Informatique de Paris 6-Université Pierre et Marie Curie, 75252 Paris Cedex 05, France (e-mail: [email protected]). Digital Object Identifier 10.1109/TNET.2006.876157 there is no scalable solution to inter-domain multicast routing. The approach used is to connect different domains through MBGP (multiprotocol extensions to BGP-4) [2] and MSDP (multicast source discovery protocol) [3]. MBGP is used to announce different unicast- and multicast-capable routes whereas MSDP exchanges active source information between the domains. The configuration complexity of this solution works against multicast deployment. On the other hand, back- bone operators are currently over-provisioning their networks and thus have one reason less to use multicast. Nevertheless, ISPs (Internet Service Providers) have interest in multicast to face the increasing demand for network resources and content distribution. As a consequence, the Internet is likely to be organized with both unicast- and multicast-enabled networks. Therefore, it is of utmost importance to design protocols that allow the progressive deployment of the multicast service by supporting unicast clouds. Different solutions that simplify the multicast service by re- ducing the distribution model were proposed [4]. EXPRESS [5] restricts the multicast conversation to 1 to (the channel abstraction), simplifying address allocation and data distribution, and still covering most of the current multicast applications. The source-specific multicast (SSM) [6] service is currently in final standardization phase at the Internet Engineering Task Force (IETF). The SSM service is implemented by Version 3 of Internet group management protocol (IGMP) [7] and by a modified ver- sion of protocol independent multicast—sparse mode (PIM-SM) [8], named PIM-SSM. Nevertheless, even if SSM largely sim- plifies the multicast model, it does not allow the progressive deployment of the multicast service. Currently, the only alterna- tive is the use of tunnels to go through unicast-only networks. Different multicast tunneling mechanisms have been pro- posed. One such mechanism is the UDP multicast tunneling protocol (UMTP) [9]. UMTP encapsulates UDP multicast datagrams inside UDP unicast datagrams, so it can be im- plemented as a user-level process at end-hosts. The work in [10] proposes a mechanism to automate the generation of UMTP tunnels. Automatic multicast tunneling (AMT) [11] is an alternative scheme that does not rely on UDP; it provides tunneling capability through pseudo network interfaces that act as default routes to multicast traffic. In this work, we do not propose a new automatic tunneling mechanism to connect the multicast-enabled parts of the Internet, but we propose instead a new multicast routing protocol that inherently supports unicast routers. Additionally, the protocol design takes into account the unicast routing asymmetries that may affect the structure of the multicast distribution tree, especially if unicast-only routers are present. 1063-6692/$20.00 © 2006 IEEE

Transcript of 16. incremental deployment service of hop by hop multicast routing protocol

Page 1: 16. incremental deployment service of hop by hop multicast routing protocol

IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 14, NO. 3, JUNE 2006 543

Incremental Service Deployment Using theHop-By-Hop Multicast Routing Protocol

Luís Henrique M. K. Costa, Member, IEEE, Serge Fdida, Senior Member, IEEE, and Otto Carlos M. B. Duarte

Abstract—IP multicast is facing a slow take-off although it hasbeen a hotly debated topic for more than a decade. Many reasonsare responsible for this status. Hence, the Internet is likely to be or-ganized with both unicast and multicast enabled networks. Thus,it is of utmost importance to design protocols that allow the pro-gressive deployment of the multicast service by supporting unicastclouds. This paper presents HBH (hop-by-hop multicast routingprotocol). HBH adopts the source-specific channel abstraction tosimplify address allocation and implements data distribution usingrecursive unicast trees, which allow the transparent support of uni-cast-only routers. An important original feature of HBH is its treeconstruction algorithm that takes into account the unicast routingasymmetries. Since most multicast routing protocols rely on theunicast infrastructure, the unicast asymmetries impact the struc-ture of the multicast trees. We show through simulation that HBHoutperforms other multicast routing protocols in terms of the delayexperienced by the receivers and the bandwidth consumption ofthe multicast trees. Additionally, we show that HBH can be in-crementally deployed and that with a small fraction of HBH-en-abled routers in the network HBH outperforms application-layermulticast.

Index Terms—Multicast, routing, service deployment.

I. INTRODUCTION

I P MULTICAST is facing a slow take-off although it has beena hotly debated topic for more than a decade. Many reasons

are responsible for this status. The IP Multicast architecture iscomposed of a service model that defines a group as an openconversation from sources to receivers, an addressingscheme based on IP class-D addresses, and routing protocols.In IP Multicast, any host can send to a multicast group and anyhost can join this group and receive data [1]. The IP Unicastservice model is also completely open, but in multicast the po-tential burden caused by unauthorized senders is multiplied bythe multicast group size.

The IP Multicast architecture is completed by group ad-dressing and routing protocols. A multicast group is identifiedby a class-D IP address which is not related to any topologicalinformation, as opposed to the hierarchical unicast addressingmodel. Therefore, multicast address allocation is complex, andmulticast forwarding state is difficult to aggregate. Currently,

Manuscript received June 17, 2003; revised December 3, 2004, De-cember 6, 2004, and June 20, 2005; approved by IEEE/ACM TRANSACTIONS

ON NETWORKING Editor R. Cohen. This work was sponsored by CNPq,CAPES/PRODOC, COFECUB, and RNP/FINEP/FUNTTEL. An earlierversion of this paper appears in the Proceedings of ACM SIGCOMM’01.

L. H. M. K. Costa and O. C. M. B. Duarte are with the Grupo de Teleinfor-mática e Automação-Universidade Federal do Rio de Janeiro, Rio de Janeiro,Brazil (e-mail: [email protected]; [email protected]).

S. Fdida is with the Laboratoire d’Informatique de Paris 6-Université Pierreet Marie Curie, 75252 Paris Cedex 05, France (e-mail: [email protected]).

Digital Object Identifier 10.1109/TNET.2006.876157

there is no scalable solution to inter-domain multicast routing.The approach used is to connect different domains throughMBGP (multiprotocol extensions to BGP-4) [2] and MSDP(multicast source discovery protocol) [3]. MBGP is usedto announce different unicast- and multicast-capable routeswhereas MSDP exchanges active source information betweenthe domains. The configuration complexity of this solutionworks against multicast deployment. On the other hand, back-bone operators are currently over-provisioning their networksand thus have one reason less to use multicast. Nevertheless,ISPs (Internet Service Providers) have interest in multicast toface the increasing demand for network resources and contentdistribution. As a consequence, the Internet is likely to beorganized with both unicast- and multicast-enabled networks.Therefore, it is of utmost importance to design protocols thatallow the progressive deployment of the multicast service bysupporting unicast clouds.

Different solutions that simplify the multicast service by re-ducing the distribution model were proposed [4]. EXPRESS[5] restricts the multicast conversation to 1 to (the channelabstraction), simplifyingaddressallocationanddatadistribution,and still covering most of the current multicast applications. Thesource-specific multicast (SSM) [6] service is currently in finalstandardization phase at the Internet Engineering Task Force(IETF). The SSM service is implemented by Version 3 of Internetgroup management protocol (IGMP) [7] and by a modified ver-sion of protocol independent multicast—sparse mode (PIM-SM)[8], named PIM-SSM. Nevertheless, even if SSM largely sim-plifies the multicast model, it does not allow the progressivedeployment of the multicast service. Currently, the only alterna-tive is theuseof tunnels togo throughunicast-onlynetworks.

Different multicast tunneling mechanisms have been pro-posed. One such mechanism is the UDP multicast tunnelingprotocol (UMTP) [9]. UMTP encapsulates UDP multicastdatagrams inside UDP unicast datagrams, so it can be im-plemented as a user-level process at end-hosts. The work in[10] proposes a mechanism to automate the generation ofUMTP tunnels. Automatic multicast tunneling (AMT) [11] isan alternative scheme that does not rely on UDP; it providestunneling capability through pseudo network interfaces that actas default routes to multicast traffic. In this work, we do notpropose a new automatic tunneling mechanism to connect themulticast-enabled parts of the Internet, but we propose instead anew multicast routing protocol that inherently supports unicastrouters. Additionally, the protocol design takes into account theunicast routing asymmetries that may affect the structure of themulticast distribution tree, especially if unicast-only routers arepresent.

1063-6692/$20.00 © 2006 IEEE

Page 2: 16. incremental deployment service of hop by hop multicast routing protocol

544 IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 14, NO. 3, JUNE 2006

The ability to transparently support unicast routers is themain motivation of the hop-by-hop multicast routing protocol(HBH) we present in this paper [12]. HBH implements multi-cast distribution through recursive unicast trees, an approachoriginally proposed in REcursive UNIcast TrEes (REUNITE)[13]. REUNITE does not use class-D IP addresses for groupidentification, abandoning the IP Multicast addressing model.Additionally, REUNITE faces some problems in the presenceof asymmetric unicast routes.

HBH uses the unicast infrastructure to do packet forwardingwith smaller routing tables, similarly to REUNITE, but usesthe channel abstraction to identify the group. Thus, HBH pre-serves compatibility with IP Multicast as it uses class-D IP ad-dresses for group identification. HBH constructs shortest-pathtrees (SPTs) instead of reverse SPTs as most routing proto-cols do [14]–[17]. Consequently, HBH has the potential to pro-vide better routes in asymmetric networks. Additionally, the treemanagement algorithm of HBH provides enhanced tree stabilityin the presence of group dynamics and reduces tree bandwidthconsumption in asymmetric networks, compared to most of thealternative solutions: shared trees, application-layer trees, andREUNITE. The results obtained through simulation support ourstatements.

This paper is organized as follows. Section II presents the re-lated work, motivations, and basic ideas of HBH, Section IIIdescribes the HBH protocol, and Section IV presents the per-formance comparison of HBH and other multicast protocolsthrough simulation. Finally, Section V concludes the paper.

II. BASIC PRINCIPLES OF HBH MULTICAST

This section presents previous works related to this paper,namely the EXPRESS and REUNITE protocols. We then in-troduce the basic principles of HBH, as well as the problemscaused by asymmetric unicast routing that motivated the designof our protocol.

A. Related Work

EXPRESS [5] provides a simple solution to the multicast ad-dress allocation problem, introducing the channel abstractionthat reduces the multicast conversation from to to 1 to

. A channel is identified by the pair , where is theunicast address of the source and is a class-D multicast ad-dress. The concatenation of a unicast address with a class-Daddress solves the address allocation problem since the unicastaddress is unique. The channel model introduced by EXPRESSalso simplifies group management issues such as sender accesscontrol. Nevertheless, the SSM service [6], which was inspiredby EXPRESS and standardized by the IETF, does not changethe group management support provided by IGMP [7].

REUNITE [13] implements multicast distribution based onthe unicast routing infrastructure. The basic motivation of RE-UNITE is that, in typical multicast trees, the majority of routerssimply forward packets from one incoming interface to only oneoutgoing interface, because only a few routers are branchingnodes [18]. In other words, for the Internet topology, the mul-ticast protocol acts as a unicast protocol in most of the nodes.

Nevertheless, all multicast protocols keep per-group informa-tion in all routers of the multicast tree. Then, the key idea ofREUNITE is to separate multicast routing information in twotables: a multicast control table (MCT), which is stored in thecontrol plane, and a multicast forwarding table (MFT), whichis installed in the data plane. Nonbranching routers simply keepgroup information in their MCT, whereas branching nodes keepMFT entries which are used to recursively create packet copiesto reach all group members.

REUNITE identifies a conversation by a pair, whereis the unicast address of the source and is a port number.

Class-D IP addresses are not used. As receivers join the groupREUNITE populates its tables to construct the distribution tree,using two control messages: and . messagestravel upstream from the receivers to the source, whereasmessages are periodically multicast by the source to refresh thesoft-state of the tree. Only the branching nodes for the group

keep entries in their MFTs. The control table,MCT, is exclusively used for tree construction, not for packetforwarding. Nonbranching routers in the tree haveMCT entries for but no MFT entry.

B. Multicast Distribution Through Recursive Unicast

The basic idea of recursive unicast is that packets have unicastdestination addresses. The routers that act as branching nodesfor a specific multicast group are responsible for the creationof packet copies with modified destination address in such away that all group members receive the information. Fig. 1(a)gives an example of the recursive unicast data distribution inREUNITE. In this figure, the source is , is a receiver, and

is a REUNITE router. The source sends data in unicast to thefirst receiver that joined the group. At a branching node, , in-coming packets are addressed to the first receiver, , that joinedthe group in the subtree below . The receiver is stored ina special MFT entry, . The router creates onepacket copy for each receiver in its MFT. The destination ad-dress of each packet copy is set to the unicast address of thereceiver. The original packet is also forwarded to . In the ex-ample of Fig. 1(a), produces data packets addressed to , andthese packets reach unchanged. The router creates onepacket copy and sends it to . Since is a nonbranching node,it simply forwards the packets without consulting its MFT. Therouter creates one packet copy to , and finally createscopies to and .

Fig. 1(b) shows how the recursive unicast data distributionworks for our protocol, HBH.1 In this figure, is an HBHrouter. The source sends data addressed to . The router

creates two packet copies and sends them to and(the next downstream branching nodes). The router simplyforwards the packets in unicast. receives the data and sendsa modified packet copy to and . Finally, creates onepacket copy to , , and . Data distribution is symmetric onthe other side of the tree.

The recursive unicast technique allows the progressive de-ployment of the multicast service because data forwarding is

1In the rest of the paper, we interchangeably use hSi and hS; P i to refer toREUNITE’s multicast group and hSi and hS;Gi to refer to HBH’s multicastchannel.

Page 3: 16. incremental deployment service of hop by hop multicast routing protocol

COSTA et al.: INCREMENTAL SERVICE DEPLOYMENT USING THE HOP-BY-HOP MULTICAST ROUTING PROTOCOL 545

Fig. 1. Data distribution in the recursive unicast approach. (a) REUNITE tree. (b) HBH tree.

based on unicast addresses. Unicast-only routers in the distribu-tion tree are transparently supported. These routers are unableto be branching nodes of the tree, but can still forward data sinceunicast destination addresses are used.

C. Risks of Asymmetric Unicast Routing

Asymmetric routing means that the unicast path from tomay differ from the path from to . In the Internet, it maybe due to different reasons [19]. The simplest case is that ofasymmetric or unidirectional links (e.g., ADSL lines or satellitelinks). There are also less obvious sources of asymmetric routes:routing misconfiguration and routes “intentionally” configuredasymmetric. One such phenomenon is known as “hot-potatorouting” and occurs because of economical reasons. For ex-ample, suppose two ISPs, and , both provide connectivitythrough the US territory. Traffic generated on the east coast innetwork and destined to a customer on the west coast con-nected to will be routed to network as soon as possible, i.e.,in a peering point located on the east coast. This way, avoidsusing its own links to cross the country since these links are amore scarce resource. In the other direction, uses the samestrategy, causing routes between and to be asymmetric.

Measurements in the Internet have shown a high percentageof asymmetric routes. The analysis in [19] evaluated about10 000 pairs of sites. Only major routing asymmetries wereconsidered, where the virtual paths differ by one city or au-tonomous system (AS). About half of the measures revealedroutes that differ by one city or more. In a different level ofgranularity, about 30% of the routes were asymmetric with atleast one AS of difference, which is still high.

Asymmetric unicast routing affects multicast routing sincemost multicast routing protocols construct reverse SPTs [14],[16], [17]. Data packets from the source to a receiver follow theunicast route used to go from the receiver to the source. If these

paths have different characteristics, the use of the reverse SPTmay be a problem to any QoS mechanism which collects infor-mation about the path from the data source to the receiver, e.g.the path loss. The ability to construct SPTs is therefore advan-tageous for a multicast routing protocol.

REUNITE differs from previous protocols because it tries toconstruct SPTs. (multicast OSPF [20] is the only Internet pro-tocol that constructs SPTs.) This is possible because themessages that travel from the source to the destination nodes in-stall the forwarding state and not the messages that followthe inverse direction. Nevertheless, REUNITE may fail to con-struct shortest-path branches in the presence of unicast routingasymmetries. A second undesirable behavior of REUNITE isthat the route for one receiver may change after the departure ofanother receiver. This is undesirable if some QoS mechanism isto be implemented.

D. Problems With the Operation of REUNITE

Fig. 2 illustrates the tree construction mechanism of RE-UNITE and gives an example where it fails to construct a SPT.Suppose the following unicast routes: ;

; ; .Suppose the following events: joins , joins ,and leaves the group.

Receiver joins the multicast group by sending amessage to . This message reaches since

there is no previous tree state for this group in the network.We say that joined the group at . The sourcethen starts sending messages to (in unicast).These messages install soft-state for in the routerstraversed downstream. The routers and create aentry in their MCTs. Now joins the group. Thetravels in the direction of reaching the tree at . The router

drops the message, creates an with

Page 4: 16. incremental deployment service of hop by hop multicast routing protocol

546 IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 14, NO. 3, JUNE 2006

Fig. 2. REUNITE’s tree construction mechanism. (a) r , then r join the group. (b) r leaves the group. (c) r is connected to S. (d) Tree after r left.

as , adds to , and removes from its MCT.The router becomes a branching node and will consequentlyforward messages downstream, upon the receptionof messages from upstream. We say that joinedthe group at . Data packets sent to the multicast group, whichare addressed to , are now duplicated at and addressedto . Subsequent messages sent by and refresh theMFT entries at and , respectively.

In this configuration, receives data from through theshortest path, but not the receiver . Because the unicast routesbetween and are asymmetric, and because intercepts the

message, data follows the path, the same as messages from down to [Fig. 2(a)].MCT and MFT states are soft. Receivers periodically send

messages and the source periodically multicasts amessage. A receiver simply stops sending mes-

sages to leave the multicast group. When the tree structure isstable, a message refreshes the MCT entries andthe entries down the tree. The mes-sages refresh the entry in the MFT of the node where joined

. For example, in Fig. 2, refreshes the entryin the MFT of and refreshes the entry in theMFT of .

Now, leaves the group: it stops sending mes-sages. As the entry in ’s MFT is not refreshed, after the ex-piration of timer , the entry becomes stale. A second timer,

, is created and will eventually destroy the entry. Asis stale, now sends messages [Fig. 2(b)].The stale messages indicate that the data flow ad-dressed to will soon stop, thus the tree portion based onhas to be reconfigured. At the branching nodes, MFT tables thathave become stale as the stale travelsdown the tree. At nonbranching nodes, the reception of a stale

causes the destruction of any MCT entries. Con-sequently, messages are no more intercepted by(as its is stale) and reach . The receiver now joins

at [Fig. 2(c)]. Eventually times out resulting in thedeletion of from ’s and ’s MFTs. As stops receiving

messages, its is destroyed [Fig. 2(d)]. Now,receives data through the shortest path from .

Asymmetric routing may also lead REUNITE to unneededpacket duplications on certain links. In fact, this possibility alsoexists when the network has pure unicast routers or when theREUNITE router is overloaded. In both cases, the branchingnode must migrate to another router and may cause packet du-plications in some links. For a more detailed description, thereader is referred to [13].

Page 5: 16. incremental deployment service of hop by hop multicast routing protocol

COSTA et al.: INCREMENTAL SERVICE DEPLOYMENT USING THE HOP-BY-HOP MULTICAST ROUTING PROTOCOL 547

Fig. 3. Packet duplication due to asymmetric routes in REUNITE.

Fig. 3 gives an example of packet duplication due to asym-metric routes. The first receiver sends a that fol-lows the path . Themessages follow the route .Suppose now that joins and that follows

. The (produced by ) andthe (created at ) both traverse the link .As does not receive messages from these receivers, itis not identified as a branching node. The source creates datapackets to and creates packets to . Therefore, there aretwo packet copies on the link .

We define the cost of the distribution tree as the number ofcopies of the same packet that need to be generated so that allreceivers get the packet. Therefore, the cost of a regular multi-cast tree, such as the PIM-SSM tree, is equal to the number oflinks in the tree. On the other hand, for application-layer mul-ticast protocols, REUNITE, HBH, and when tunneling is used,there may be more than one instance of the same packet beingtransmitted on a link. Our cost definition takes the additionalcopies into account. The number of copies of the same packettransmitted on a single link is often called link stress. As a con-sequence, the cost of a REUNITE tree may be larger than thatof a source tree constructed by a classic multicast routing pro-tocol such as PIM-SM [16], because the reverse path forwarding(RPF) algorithm ensures that only one packet copy is ever trans-mitted over each network link.

The first simulations with REUNITE showed that in somecases its algorithm uses a huge number of control messages. Theinvestigation revealed a temporary loop creation, which needs tobe solved by an additional check, e.g. the list of routers traversedby a control message. HBH, as shown in the next section, doesnot suffer from temporary loops.

III. HBH MULTICAST PROTOCOL

The HBH multicast protocol has a tree construction algorithmthat is able to better deal with the pathological cases due toasymmetric unicast routes. HBH uses two tables, an MCT andan MFT, that have nearly the same function as in REUNITE. Thedifference is that one entry table in HBH stores the address of anext branching node instead of the address of a receiver, exceptthe branching router nearest the receiver. The MFT has noentry. Data received by a branching router, , has unicast des-tination address set to (in REUNITE, data are addressed to

). This choice makes the tree structure more stablethan in REUNITE.

A multicast channel in HBH is identified by , whereis the unicast address of the source and is a class-D IP ad-dress allocated by the source. This definition solves the addressallocation problem while being compatible with SSM’s channeldefinition. Therefore, HBH can support IP Multicast clouds asleaves of the distribution tree.

The tree structure of HBH has the advantage of an enhancedstability of the table entries when compared with REUNITE.The tradeoff is that in HBH each data packet received by abranching node produces modified packet copies while in RE-UNITE it produces modified packets, assuming that isthe number of outgoing multicast links. The tree managementscheme of HBH minimizes the impact of member departuresin the tree structure. This is possible because the MFT receiverentry is located at the branching node nearest the receiver. Forexample, the departure of in Fig. 1(a) has a greater impacton the tree structure of REUNITE than on that of HBH. ForREUNITE, the routing tables of , , , and are modi-fied, whereas only the table of is modified for HBH. In theworst case, HBH may need one more change than REUNITE,when a branching node becomes a nonbranching one (e.g., afterthe departure of ). In this example, there is no route changesfor other members when a member leaves the group becausethe unicast routes are symmetric. Nevertheless, tree reconfigu-ration in REUNITE may cause route changes to the remainingreceivers, as for in the example of Fig. 2. This is avoided inHBH.

A. Tree Management in HBH

HBH has three control messages: , , and .messages are periodically unicast by the receivers in the

direction of the source and are used to refresh the forwardingstate (MFT entry) at the router where the receiver was con-nected to the tree. A branching router itself “joins” the multicastchannel at the next upstream branching router. The mes-sages may be intercepted by the branching routers, which mustsign themselves messages, and filter the messagesreceived from downstream nodes. The source periodically mul-ticasts a message that refreshes the tree structure.messages are sent by potential branching routers and constructthe distribution tree together with the messages.

The Appendix gives a detailed description of the controlmessages processing rules of HBH. The basic ideas are: thefirst issued by a receiver is never intercepted, reachingthe source, and the messages are periodically multicast bythe source. These messages are combined with

Page 6: 16. incremental deployment service of hop by hop multicast routing protocol

548 IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 14, NO. 3, JUNE 2006

messages, sent by potential branching nodes, to construct andrefine the tree structure.

B. Routing Tables

The HBH protocol uses two routing tables, an MCT and anMFT. Consider a multicast channel implemented by thesource . Each HBH router in the distribution tree of the source

has either a or an .The state kept in HBH’s multicast routing tables is soft and

periodically refreshed by the protocol control messages. Eachreceiver periodically sends a in unicast to thesource. The source periodically multicasts a message foreach channel.

Routers implementing HBH which are in the distribution treeof have entries in their routing tables. Normally, non-branching routers have entries in their MCT. Branchingrouters have entries in their MFT. Nevertheless, a non-branching router may also have an MFT, in some configurations,to cope with asymmetric routes (see Section III-B3).

1) MCT Entries: An HBH router in ’s distribution tree mayhave an . The has a single entry, whichcan be fresh} or stale. Therefore, two timers are associated to theMCT entry, and . At the expiration of the MCT becomesstale and at the expiration of the MCT is destroyed.

MCT entries are not used to replicate data. Depending onits state, different actions are taken upon reception of controlmessages. These actions are described in detail in the Appendix.

2) MFT Entries: An HBH router in the distribution tree ofthe source may also have an . The MFT entriesare also soft-state. Two timers, and , are associated witheach entry in . When times out, the MFT entrybecomes stale and it is destroyed when expires.

The MFT entries stored for channel may be in one ofthree states:

• default—The default state of an MFT entry is to be fresh(not stale) and unmarked. In default state, the MFT entryis used for both data and control forwarding.

• stale—If the MFT entry is stale, it is used to forward data(avoiding service disruption during tree reconfiguration)but it is no longer used to forward control messages (thereis no notion of stale message in HBH).

• marked—The MFT entry may also be marked. As opposedto a stale entry, a marked entry is used to produce con-trol messages but does not participate in data replication.

3) Branching Versus Nonbranching Nodes: There is onepeculiarity of HBH regarding branching nodes. A branchingnode is a router that has more than one outgoing multicast link,which is the common definition of a branching node. There-fore, an HBH branching router has MFT state. Nevertheless,a nonbranching router may also keep MFT state, in certainscenarios, as a consequence of the algorithm we devised to copewith asymmetric routes. The difference between a “regular”nonbranching router, which only has MCT state, and a non-branching router which has an MFT is that the first one simplyforwards packets in unicast, whereas the second one modifiesthe destination addresses of the data packets it forwards. Theexample in Fig. 5 illustrates one such scenario.

C. Data Forwarding

HBH uses IP unicast destination addresses for data packets.Data packets are replicated at HBH branching routers, in sucha way that all the receivers connected to the multicast channelreceive the data.

Data forwarding in HBH works as follows. Suppose a source,, and the source’s first-hop router . The source produces IP

packets which have the IP source address set to ’s IP addressand the IP destination address set to ’s IP address. The router

creates copies of the data packet, according to its MFT. Thecopies have the IP destination address set to the unicast addressstored in the MFT (typically, that is the address of the next-hopHBH router). Packets are recursively replicated according to thedistribution tree, until they get to the tree leaves.

There are two possibilities for the tree leaves. The leaf maybe the receiver’s first-hop router (the last-hop router from thesource) or the receiver itself. In the first case, the receiver useIGMP to tell the first-hop router which multicast channel it iswilling to listen to, just as in classical IP Multicast. HBH re-quires IGMPv3 because the multicast group and the source IPaddresses must be passed to the local HBH router. The otherpossibility is for the receiver to connect itself to the multicasttree. In that case, its local HBH router would have one entry in itsMFT for each receiver. The first solution is the preferred config-uration, because it reduces the number of MFT forwarding en-tries in the local router and because it does not need any modifi-cation in the receiver’s operating system—the only requirementis version 3 of IGMP.

Similarly, there are two choices for the source. The first oneis for the source, to produce data packets with unicast destina-tion address set to . In that case, there is no modification in theoperating system of the source station, but the application mayneed modification to allow two unicast destination addresses forthe channel (normally, the application would use a multicastdestination address, say, , and the source’s unicast addressfor the channel). The other possibility is to make no modifica-tions at all in the station. In that case, the application producesdata addressed to , just as it would for SSM. The first-hopHBH router is responsible for modifying the data packets withthe next hop’s unicast destination address. This configuration isprobably better, since it requires no modification in the station(there are no changes in the service interface) but only in thefirst-hop router, which anyway is modified to implement HBH.

D. Operation of HBH

To illustrate the operation of HBH, we repeat the examplesof Section II-C. First, Fig. 4 repeats the scenario of Fig. 2. Thereceiver joins the multicast channel at the source , whichstarts producing messages. These messages create a

containing at routers and [Fig. 2(a)]. Whenjoins the group, by sending the first , this mes-

sage is not intercepted and reaches (the first messageis never intercepted). The produced by the sourcecreate state at router [Fig. 4(b)]. Both receivers areconnected to the source through the shortest path. This can only

Page 7: 16. incremental deployment service of hop by hop multicast routing protocol

COSTA et al.: INCREMENTAL SERVICE DEPLOYMENT USING THE HOP-BY-HOP MULTICAST ROUTING PROTOCOL 549

Fig. 4. HBH’s tree construction mechanism. (a) r , then r , subscribe the channel. (b) r is connected toS. (c) r subscribes the channel. (d) r leaves the channel.

be guaranteed because the first join message always reaches thesource .

Suppose now that the receiver (unicast routes:and ) joins the channel. It

sends a to , which starts sending mes-sages. As receives two different messages, it sends a

to the source. The reception of themessage causes to mark the and entries in its MFT andto add to it. In the same way as , receivesand messages and thus sends a tothe source [Fig. 4(c)]. ’s MFT now contains and . Subse-quent messages are intercepted by and refreshthe marked entry in ’s MFT. The messagesrefresh the MFT entry at . The source sends data ad-dressed to that forwards it addressed to . The routersends copies to and . Afterwards, as receives no more

or messages, its corresponding MFTentries time out and are destroyed. The final structure is shownin Fig. 4(d). In this way, HBH discovers the ideal branchingpoint for the distribution tree.

The messages used by HBH cope with the problemof Fig. 3. The first will reach the source. After re-ceiving different messages for and , router sends

a to (Fig. 5). Subsequentand messages will be intercepted by . At itsturn, router receives two different ’s and sends a

upstream. In this case, however, it willnever receive messages issued by receivers and .The consequence is that ’s entry in will be kept staleand and entries will be fresh, but marked. Thus, data willbe produced to as control will be addressed to and .The design choice of HBH imposes some control overhead butminimizes data duplication.

Temporary loop creation is avoided in HBH because the cre-ation of forwarding state is always a consequence of messagesthat travel downstream, the messages sent by the source tothe receivers. messages can also instantiate MFT state,but as the production of messages is itself consequenceof different message reception, and because the “fusion”process is done hop by hop, the produced forwarding structureis always a SPT.

IV. PERFORMANCE ANALYSIS

We used Network Simulator (NS) [21] to compare HBH toother routing algorithms from the viewpoint of the constructedtrees. We analyzed the average delay experienced by all of the

Page 8: 16. incremental deployment service of hop by hop multicast routing protocol

550 IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 14, NO. 3, JUNE 2006

Fig. 5. Avoiding packet duplication in HBH.

receivers of the group and the number of copies of the samepacket that are transmitted to reach all of the receivers.

A. Simulation Scenario

The first topology used in our simulations has 18 nodes and istypical of a large ISP network [22]. Without loss of generality,we suppose that only one receiver is connected to each node inthe topology. The presence of one or many receivers attached toa border router through IGMP [7] does not influence the cost ofthe tree, so we do not consider the aggregation provided by themulticast service at the local network level. Thus, in the 18-nodetopology, nodes 0–17 are routers, whereas nodes 18–35 are po-tential receivers of the multicast channel.2 The second topologywas randomly generated, with 50 nodes and higher connectivity(8.6 versus 3.3). Additionally, we used nem [23], which is atopology generator based on Internet map sampling. Nem ran-domly extracts a subgraph of a network map, keeping its originalproperties, such as: node degree, mean distance, mean eccen-tricity, and topology diameter. Magoni and Pansiot [23] showthat the topologies generated by nem respect the power laws es-tablished on real Internet maps. The third topology we used has500 nodes and was produced by nem.

As the NS does not implement BGP, we produced asym-metric routes by the use of different link costs. We associate withevery link a cost in each direction. Hence, for the link com-posed by the nodes and , we associate costs and

that are integers randomly chosen in the interval [1],[10]. Simulations consider one multicast group from one sourceto receivers. The source node is fixed. A variable number ofrandomly chosen receivers join the channel. The plots show theaverage of 500 simulation runs per protocol per group size.

2REUNITE uses the term multicast group whereas HBH and SSM use mul-ticast channel. In this section, we also use channel for REUNITE.

We compare HBH to REUNITE and two classical multicastprotocols. NS has a routing protocol which is able to constructshared trees and source trees with the same structure as thetrees constructed by the PIM-SM protocol. The difference isthat the NS implementation is centralized, and the change fromthe shared tree to the source tree is triggered through an explicitcommand and not automatically as in the original PIM-SM [16].Therefore, PIM-SM in our simulations is a protocol similar tothe real implementation of PIM-SM but that exclusively con-structs shared trees, whereas PIM-SSM is a protocol that onlyconstructs source trees. The tree structure PIM-SSM is a re-verse SPT. In addition to HBH, we implemented REUNITE ac-cording to [13]. All routers implement the multicast service inthe first set of experiments, i.e., all routers implement HBH orREUNITE. No tunneling through unicast clouds was needed.

B. Tree Cost Analysis

We first evaluated the cost of the trees constructed by the dif-ferent multicast routing protocols. We define the cost of a treeas the number of copies of the same packet that are transmittedin the network links. This cost definition takes the additionalcopies, or link stress, into account. The cost of a PIM-SM orPIM-SSM tree is equal to the number of links in the tree. Never-theless, the tree cost for a REUNITE, HBH, or application-layermulticast tree may be different from the number of links in thetree, because one packet may be sent more than once on thesame link. The recursive unicast technique may send more thanone copy of the same packet over a specific link. This maybe due to routing asymmetries (as shown is Section II-C) butalso to unicast routers inside the network that are not able to bebranching nodes. In this case, the location of a branching nodemay not be ideal. The same is valid when we use automatic tun-neling or application-layer multicast, because the physical net-work topology is not known by the end systems. In our exper-iments all routers are multicast capable, therefore extra packetcopies produced by REUNITE or HBH are a consequence ofrouting asymmetries.

Fig. 6 shows the average cost of the multicast trees con-structed by the different protocols as the number of receiversvaries. For the 18-node ISP topology, PIM-SM constructsthe trees with the highest cost in most cases. This result wasexpected since PIM-SM constructs shared trees. As we simu-lated the distribution from one source to many receivers, theutilization of a shared tree is disadvantageous since the tree iscentered on a rendezvous point (RP). With a high probability,this tree has a higher cost than the equivalent source tree.HBH and PIM-SSM construct the cheapest trees. This result isexpected since PIM-SSM constructs source trees based on theRPF algorithm, which guarantees that, at the maximum, onecopy of the same packet is transmitted over each link, and thateach receiver is connected to the source through the reverseshortest-path. HBH performs similar to PIM-SSM becausein HBH each receiver is connected to the source through theshortest path. Using this path or the reverse shortest path doesnot influence tree cost.

The REUNITE curves in Fig. 6 demonstrate that the treeconstruction mechanism of REUNITE suffers from the patholo-gies produced by asymmetric unicast routing, investigated in

Page 9: 16. incremental deployment service of hop by hop multicast routing protocol

COSTA et al.: INCREMENTAL SERVICE DEPLOYMENT USING THE HOP-BY-HOP MULTICAST ROUTING PROTOCOL 551

Fig. 6. Tree cost. (a) 18-node IS topology. (b) 50-node random topology.(c) 500-node Internet-like topology.

Section II-C. The phenomenon is less frequent with a smallnumber of receivers, since the probability that two receiversshare the same link in the multicast tree is smaller. For the ISPtopology, the problem is also less severe when the number of re-ceivers is huge (receiver distribution is dense) since a large per-centage of network links is used in the distribution tree anyway.

Nevertheless, this is not the case for the 50-node topology. Thistopology has a much higher connectivity, which means that asmaller percentage of network links is used. In this topology, theadvantage of HBH increases with the group size. The PIM-SMshared trees also outperform REUNITE. This is a consequenceof badly placed branching nodes, which lead REUNITE to use-less packet duplication. With the 500-node topology, the gainsof HBH over REUNITE are approximately constant and around5%. The difference is almost constant because the number ofreceivers represents a smaller percentage of the total number ofnodes than for the other two topologies. Furthermore, the con-nectivity of the 500-node Internet-like topology is similar to the18-node ISP topology, giving percent gains in the same order ofmagnitude.

The analysis of the HBH curve shows the enhanced efficiencyof the tree construction mechanism of HBH. In terms of treecost, the advantage of HBH over REUNITE, averaged gain overall group sizes, is as large as 5% for the 18-node ISP topology,18% for the 50-node topology, and 5% for the 500-node In-ternet-like topology. We conclude that HBH potentially pro-vides a better bandwidth utilization than REUNITE for asym-metric networks.

C. Delay Analysis

Fig. 7 presents the average delay experienced by the receiversin the multicast channel for the same set of simulations. Thecurves show that HBH is effectively able to generate betterquality routes than REUNITE in the presence of asymmetricunicast routing.

Fig. 7(a) shows two unexpected results. First, PIM-SMperforms better that PIM-SSM in terms of delay for the ISPtopology. In other words, this result means that the shared treeshave a better delay performance than the source trees. This factis explained because PIM-SSM tree is a reverse SPT and not aSPT and, as a consequence, delay is not minimized. Delay isnot minimized either in the shared tree constructed by PIM-SM.But, in the shared tree, all of the paths from the source to thedifferent receivers have one common portion, namely, the pathbetween the source and the RP. As data are encapsulated inunicast between the source and the RP, delay is minimizedbetween the source and the RP. Hence, paths in the PIM-SMtree have two parts: one from the source to the RP, where thedelay is minimized, and another from the RP to the receiver,where the delay is not minimized since it is a reverse shortestpath. This explains the advantage of PIM-SM over PIM-SSM.The same was not observed for the 50-node topology becausethis topology is larger and has a higher connectivity. Therefore,going through the RP is likely to result in a longer path thangoing directly from the source to the receiver. As expected, theshared tree has the worst delay performance in this case.

The second important remark for the 18-node ISP topologyis that the effect of the network asymmetries in the quality ofREUNITE trees may be strong, as REUNITE performed worsethan PIM-SM when the receiver set is large. REUNITE per-forms better than PIM-SM in the 50-node topology because thistopology has a higher connectivity.

The delay performance of HBH is better than that of RE-UNITE for all group sizes, and for all topologies. The advantage

Page 10: 16. incremental deployment service of hop by hop multicast routing protocol

552 IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 14, NO. 3, JUNE 2006

Fig. 7. Receiver average delay. (a) 18-node ISP topology. (b) 50-node randomtopology. (c) 500-node Internet-like topology.

becomes larger as the number of receivers grows, being of 14%in average for the 18-node ISP topology. The absolute valuesfor the 50-node random topology are smaller as this topologyhas a higher connectivity. On the other hand, the advantage ob-tained by HBH over REUNITE for this topology is larger, 30%

in average. This is a consequence of its richer connectivity, asthe routing protocol has more choices to construct the distribu-tion tree, and is consequently more vulnerable to unicast routingasymmetries. For the 500-node Internet-like topology, the ad-vantage of HBH over REUNITE is of 8% in average. The dif-ference is smaller because the connectivity of this topology issimilar to the 18-node ISP topology and because the maximumnumber of receivers is a small fraction of the total number ofnodes in this topology. The delay gains increase with the groupsize.

D. Message Cost Analysis

In this section, we compare the message processing over-heads of HBH and REUNITE, i.e., the amount of controlmessages used by the protocol to keep the multicast distributiontree. To make a fair evaluation, both protocols use the samejoin refresh periods (the interval between the production ofconsecutive messages by a receiver), as well as the sameperiod for message production and timeout values (usedto “timeout” the table entries and eventually destroy them).Additionally, two sets of experiments were made. In the firstone, routes are asymmetric as in all previous experiments. Inthis situation, REUNITE may produce temporary loops forcertain scenarios and create a huge number of messages. Insuch scenarios HBH largely outperforms REUNITE becausethe average processing overhead of REUNITE is increased.Even though the actual Internet scenario is of asymmetricroutes, we forced a second type of scenario, where all unicastroutes are symmetric, to measure the overhead of the algorithmof HBH over REUNITE.

These experiments were conducted with the ISP topology.Fig. 8(a) shows the results obtained with asymmetric routes.They confirm our statements on the problems REUNITE mayincur. As we did not add any loop avoidance mechanism to RE-UNITE, the number of useless messages produced during thetemporary loop is a function of the link bandwidth and prop-agation delay. This is because a message received by arouter is immediately forwarded in our implementation, and, iftwo routers form a “ loop,” the number of messages pro-duced is only limited by the physical path connecting them. Forsmall groups, REUNITE performs the same as HBH, becausethe possibility of loop creation is low. The same holds for largegroups, because almost every router in the network is part of thetree.

Fig. 8(b) shows the results obtained with symmetric routes. Inthis case, HBH does show a message processing cost larger thanREUNITE. This is explained by the more complex algorithmof HBH. To guarantee the construction of a SPT, the first joinmessage issued by a receiver always reaches the source. Fur-thermore, in some cases HBH will continue to producemessages as a way to avoid useless data-packet duplication, forexample, router in Fig. 5. Fig. 8(b) shows that the controloverhead of HBH compared to REUNITE is about 11%.

We can conclude that HBH is a promising approach. TheHBH algorithm outperformed REUNITE by 5% in tree cost and14% in delay paying 11% in control overhead, considering theunfavorable scenario of symmetric routes.

Page 11: 16. incremental deployment service of hop by hop multicast routing protocol

COSTA et al.: INCREMENTAL SERVICE DEPLOYMENT USING THE HOP-BY-HOP MULTICAST ROUTING PROTOCOL 553

Fig. 8. Average number of control messages—18-node ISP topology.(a) Asymmetric routes. (b) Symmetric routes.

E. Incremental Deployment Analysis

As a consequence of the difficulties involved in deployinga router-based multicast service in the Internet, different ap-proaches were proposed to implement the multicast service atthe application layer [24], [25]. Implementing the multicast ser-vice at the application layer has the advantage of not needing tomodify router implementations. On the other hand, the pointswhere packets are replicated are limited to the stations, nor-mally located at the boundaries of the network. Therefore, ap-plication-layer multicast (ALM) is a contender to router-basedmulticast, as implemented by IP Multicast routing protocols, oralternative routing protocols such as EXPRESS, REUNITE, orHBH.

A differential characteristic of HBH is its ability to be incre-mentally deployed. Routers have to be modified to implementHBH, but not all routers in the network have to implement HBH.The protocol can be partially deployed because of the recursiveunicast technique. In this section, we compare HBH with ALM,with different degrees of HBH deployment.

Different ALM systems exist. They have in common thefact that replication is done at the stations (end-points of thenetwork). Therefore, we decided to compare HBH to what

Fig. 9. Tree cost for different HBH deployment levels. (a) 18-node ISPtopology. (b) 50-node random topology. (c) 500-node Internet-like topology.

we called end system multicast (ESM), not to any specificALM system. Hence, ESM is not a specific application-layerprotocol, but a protocol which simulates a generic applica-tion-layer protocol. The characteristic we analyze to compareHBH with ESM is the cost of the multicast tree, which is thesame metric used in the previous analysis. To be fair, ESM is a

Page 12: 16. incremental deployment service of hop by hop multicast routing protocol

554 IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 14, NO. 3, JUNE 2006

Fig. 10. Message processing in HBH. (a) Join message. (b) Fusion message. (c) Tree message.

protocol which simply builds SPTs, with the restriction that theonly possible branching nodes of the multicast distribution treeare the end systems. Routers do not participate in the multicastdistribution. This is the main characteristic of the different ap-plication-layer protocols, therefore the one we chose to modelby implementing ESM.

The simulation scenario uses the same topologies of the pre-vious analysis. Nevertheless, all links are symmetric in this sce-nario because we aim to isolate the effect of different degrees

of deployment over the cost of HBH trees. We varied the per-centage of HBH routers in the network from 20% up to 100% ofthe total number of routers. We vary the group size and measurethe cost of the distribution tree, as the total number of packetssent.

Fig. 9(a)–(c) show the results. In the figures, HBH- meansthat of the routers in the network implement HBH, whereasthe other of the routers are unicast-only. HBHrouters were randomly placed in the network. In all figures, we

Page 13: 16. incremental deployment service of hop by hop multicast routing protocol

COSTA et al.: INCREMENTAL SERVICE DEPLOYMENT USING THE HOP-BY-HOP MULTICAST ROUTING PROTOCOL 555

can see that the advantage of HBH over ESM increases with thepercentage of HBH routers deployed. Fig. 9(a) shows the re-sults for the smallest topology, with 18 nodes. With symmetriclinks, HBH is 7% better than ESM when all routers are HBH-en-abled, and with only 20% of HBH routers, HBH already per-forms 3% better than ESM. For the 50-node random topology,on the other hand, HBH was 10% better than ESM with 100%of HBH-enabled routers, but with other percentages of deploy-ment, HBH performed almost the same as ESM [Fig. 9(b)].The most promising results were obtained with the 500-nodetopology, generated from a real Internet map (see Section IV-A).With only 20% of HBH routers deployed in the network, HBHperformed 14% better than ESM. With 40% of HBH-enabledrouters, the gain is as large as 30%, and on the other hand,HBH is 35% better than ESM when all routers implement HBH.Therefore, it is not needed to implement HBH in all routers, and,in close to 100% of HBH routers in the network, the relativegains are small.

V. CONCLUSION

In this paper, we analyzed HBH, a multicast routing protocolthat implements multicast distribution through recursive unicasttrees. HBH allows the incremental deployment of the multicastservice because unicast routers inside the network are transpar-ently supported. The main goals of HBH are: to support unicastclouds, allowing incremental deployment; to have a stable treestructure, by minimizing the impact of receiver departures; andto construct low-cost trees.

The tree management algorithm of HBH uses three controlmessages to construct an SPT. messages are periodicallysent to the source by the receivers. The source periodically pro-duces messages that are multicast to the receivers. As the

messages travels in the tree, the intermediate nodes maygenerate messages that are responsible of refining thetree structure.

HBH is able to construct SPTs even if unicast routing is asym-metric. HBH also provides better network utilization becauseit constructs recursive unicast trees minimizing packet duplica-tion. This is a great advantage when the network bandwidth isscarce. Moreover, HBH provides better delay performance thanother routing protocols, favoring interactive applications.

The simulation results show that HBH is a promising ap-proach. Its tree construction algorithm outperformed REUNITEin terms of the tree cost and the delay experienced by the re-ceivers. The advantage of HBH grows with larger and more con-nected networks.

We also analyzed the incremental deployment of HBH. Theperformance of HBH improves with the number of HBH-en-abled routers in the network, as should be expected. Never-theless, the interesting results are that with only 20% of HBHrouters in the network, HBH already presents significant gains.Additionally, with 60% of HBH routers, the performance ofHBH is close to the one obtained with 100% of HBH routers.Therefore, there is no need to implement HBH at all routers inthe network to take advantage of the protocol. Thus, a point thatdeserves investigation is the placement of HBH routers in thetopology.

APPENDIX

MESSAGE PROCESSING IN HBH

Fig. 10 presents the processing rules of the three messagetypes used by HBH to construct the distribution tree. Eachreceiver periodically sends a in unicast to thesource. The source periodically multicasts a message foreach channel.

message [Fig. 10(a)]—When router receives a, it should forward this message unchanged if

has no MFT (1) or if is not in ’s MFT (2). Only if ’sMFT has a entry, does intercept the and send a

afterwards. This means that is a branching nodeof the channel (3).

message [Fig. 10(c)]—A message receivedby router is treated and forwarded in multicast. If is abranching node, it may receive a message addressed to .In this case, discards the message and sends a messageto each node present in its MFT (1). If is a branching node and

is not addressed to , there are two possibilities: isa new receiver (in which case inserts in its MFT and sendsa message upstream) (2) or is present in the MFT of

—which means that does not receive messagesfrom —therefore simply has to refresh the entry in itsMFT and to send a message upstream (3). If is nota branching node, there are two possibilities: was not in ’sdistribution tree in which case creates an MCT containing(4) or was already in the tree but was not a branching node(in which case has an ) (5). If was already in ’sMCT, nothing has changed and simply refreshes its MCT (6).If is not present in the MCT, then maybe the MCT is stale inwhich case replaces the previous MCT entry, or the MCT is upto date which means that there is a second receiver that will re-ceive data through , so becomes a branching node. This im-plies the creation of an MFT, the destruction of the MCT, and a

message to be sent upstream (8). The messagesproduced by contain all the nodes that has in its MFT—thenodes for which is branching node.

message [Fig. 10(b)]—Suppose router receives afrom node . If the message is not ad-

dressed to , then simply forwards it upstream (1). If the mes-sage is addressed to , then marks the receiver entries in itsMFT that are listed in the message (2). A marked entryin the MFT is used to message forwarding but not for datadistribution. is added to ’s MFT if it was not previouslypresent. In addition, ’s timer is expired— becomes stale(3). Consequently, will be used for data forwarding, but notfor message forwarding. If was already present in ’sMFT, then ’s timer is refreshed (it avoids the destructionof the entry), but its timer is kept expired (4).

If afterwards (the node that produced the for) receives the messages produced by any of, it intercepts them and send a upstream.

In this case the entries in ’s MFT will eventuallytimeout and be destroyed, while the entry in ’s MFTis refreshed by the . If does not receivemessages from , the emission of messagescontinues since there is another node upper in the tree that

Page 14: 16. incremental deployment service of hop by hop multicast routing protocol

556 IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 14, NO. 3, JUNE 2006

receives the and periodically produce themessages to these receivers. Nevertheless,

this node will not forward data to these receivers, but toinstead since the receivers’ entries are marked. The node isresponsible for data duplication. In this way, HBH solves thesecond asymmetry problem presented in Section II-C.

REFERENCES

[1] S. Deering, “Host Extensions for IP Multicasting,” RFC 1112, Aug.1989.

[2] T. Bates, R. Chandra, D. Katz, and Y. Rekhter, “Multiprotocol Exten-sions for BGP-4,” RFC 2283, Feb. 1998.

[3] D. Meyer and B. Fenner, “Multicast Source Discovery Protocol(MSDP),” RFC 3618, Oct. 2003.

[4] C. Diot, B. N. Levine, B. Liles, H. Kassem, and D. Balensiefen, “De-ployment issues for the IP multicast service and architecture,” IEEENetwork, vol. 14, no. 1, pp. 78–88, Jan. 2000.

[5] H. W. Holbrook and D. R. Cheriton, “IP multicast channels: EX-PRESS support for large-scale single-source applications,” in ACMSIGCOMM, Sep. 1999, pp. 65–78.

[6] S. Bhattacharyya, C. Diot, L. Giuliano, R. Rockell, J. Meylor, D.Meyer, G. Shepherd, and B. Haberman, “An Overview of Source-Spe-cific Multicast (SSM),” RFC 3569, July 2003.

[7] B. Cain, S. Deering, I. Kouvelas, B. Fenner, and A. Thyagarajan, “In-ternet Group Management Protocol, Version 3,” RFC 3376, Oct. 2002.

[8] B. Fenner, M. Handley, H. Holbrook, and I. Kouvelas, “Protocol In-dependent Multicast—Sparse Mode (PIM-SM): Protocol Specification(Revised),” Work in progress, <draft-ietf-pim-sm-v2-new-10.txt>, Jul.2004.

[9] R. Finlayson, “The UDP Multicast Tunneling Protocol,” Work inprogress, <draft-finlayson-umtp-09.txt>, Nov. 2003.

[10] P. Liefooghe and M. Goossens, “An architecture for seamless accessto multicast content,” in Proc. IEEE Conf. Local Computer Networks,Nov. 2000, pp. 488–494.

[11] D. Thaler, M. Talwar, A. Aggarwal, L. Vicisano, and D. Ooms, “IPv4Automatic Multicast Without Explicit Tunnels (AMT),” Work inprogress, <draft-ietf-mboned-auto-multicast-02.txt>, Feb. 2004.

[12] L. H. M. K. Costa, S. Fdida, and O. C. M. B. Duarte, “Hop by hopmulticast routing protocol,” in Proc. ACM SIGCOMM, Aug. 2001, pp.249–259.

[13] I. Stoica, T. S. E. Ng, and H. Zhang, “REUNITE: A recursive unicastapproach to multicast,” in Proc. IEEE INFOCOM, Mar. 2000, vol. 3,pp. 1644–1653.

[14] S. Deering, D. L. Estrin, D. Farinacci, V. Jacobson, C.-G. Liu, andL. Mei, “The PIM architecture for wide-area multicast routing,”IEEE/ACM Trans. Netw., vol. 4, no. 2, pp. 153–162, Apr. 1996.

[15] C. Diot, W. Dabbous, and J. Crowcroft, “Multipoint communication:A survey of protocols, functions and mechanisms,” IEEE J. Sel. AreasCommun., vol. 15, no. 3, pp. 277–290, Apr. 1997.

[16] D. Estrin, D. Farinacci, A. Helmy, D. Thaler, S. Deering, M. Handley,V. Jacobson, C. Liu, P. Sharma, and L. Wei, “Protocol IndependentMulticast-Sparse Mode (PIM-SM): Protocol Specification,” RFC 2362,Jun. 1998.

[17] D. Waitzman, C. Partridge, and S. Deering, “Distance Vector MulticastRouting Protocol,” RFC 1075, Nov. 1988.

[18] R. C. Chalmers and K. C. Almeroth, “On the topology of multicasttrees,” IEEE/ACM Trans. Netw., vol. 11, no. 1, pp. 153–165, Feb. 2003.

[19] V. Paxson, “End-to-end routing behavior in the Internet,” IEEE/ACMTrans. Netw., vol. 5, no. 5, pp. 601–615, Oct. 1997.

[20] J. Moy, “Multicast Extensions to OSPF,” RFC 1584, Mar. 1994.[21] K. Fall and K. Varadhan, “The ns Manual,” UC Berkeley, LBL, USC/

ISI, and Xerox PARC, Dec. 2003 [Online]. Available: http://www.isi.edu/nsnam/ns/ns-documentation.html

[22] G. Apostolopoulos, R. Guerin, S. Kamat, and S. K. Tripathi, “Qualityof service based routing: a performance perspective,” in Proc. ACMSIGCOMM, Sep. 1998, pp. 17–28.

[23] D. Magoni and J.-J. Pansiot, “Internet topology modeler based on mapsampling,” in Proc. IEEE Int. Symp. Comput. Commun., Jul. 2002, pp.1021–1027.

[24] A. El-Sayed, V. Roca, and L. Mathy, “A survey of proposals for analternative group communication service,” IEEE Network, vol. 17, no.1, pp. 46–51, Jan. 2003.

[25] P. Radoslavov, C. Papadopoulos, R. Govindan, and D. Estrin, “A com-parison of application-level and router-assisted hierarchical schemesfor reliable multicast,” IEEE/ACM Trans. Netw., vol. 12, no. 3, pp.469–482, Jun. 2004.

Luís Henrique M. K. Costa (M’99) received theElectronic Engineer degree and the M.Sc. degree inelectrical engineering from the Federal Universityof Rio de Janeiro (UFRJ), Rio de Janeiro, Brazil, in1997 and 1998, respectively, and the D.Sc. degreefrom the University Pierre et Marie Curie (Paris 6),Paris, France, in 2001.

In 2002, he was a Post-Doctoral Researcher withLaboratoire d’Informatique de Paris 6, Paris. Then,he was awarded a research grant from CAPES (anagency of the Brazilian Ministry of Education) and

joined COPPE/UFRJ. Since August 2004, he has been an Associate Professorwith UFRJ. His major research interests are in the area of routing, especiallyon wireless networks, group communication, quality of service, multicast, andlarge-scale routing.

Dr. Costa has been a member of the Association for Computing Machinerysince 2001.

Serge Fdida (M’88–SM’98) received the Doctoratde 3eme Cycle and the Habilitation a Diriger desRecherches from the University Pierre et MarieCurie, Paris, France, in 1984 and 1989, respectively.

He has been a Professor with the University Pierreet Marie Curie since 1991. From 1989 to 1995, he wasa Full Professor with the University Rene Descartes,Paris, France. His research interests are in the areaof high-speed networking, multicast communication,resource management, and performance analysis. Heheads the Network and Performance Group of the

Laboratoire d’Informatique de Paris 6, Paris. He is also involved in two IFIPworking groups on networking. He is also the Co-Director of EURONETLAB,which is a joint laboratory established in 2001 between the University Paris 6,CNRS, THALES, and 6WIND, developing research and development work onQoS routers, radio routers, and security.

Dr. Fdida is a member of the Association for Computing Machinery.

Otto Carlos M. B. Duarte received the ElectronicEngineer degree and the M.Sc. degree in electricalengineering from the Federal University of Rio deJaneiro (UFRJ), Rio de Janeiro, Brazil, in 1976 and1981, respectively, and the Dr. Ing. degree fromENST/Paris, Paris, France, in 1985.

Since 1978, he has been a Professor with UFRJ.From January 1992 to June 1993, he was with MASILaboratory, University Paris 6, Paris. In 1995, hespent three months with the International ComputerScience Institute (ICSI), University of California,

Berkeley. In 1999 and 2001, he was an Invited Professor with the UniversityParis 6. He heads the computer network group (Grupo de Teleinformática eAutomação-GTA), UFRJ. His major research interests are in multicast, QoSgarantees, security, and mobile communications.