BitTorrent Unified Tracker (article)

8
Design and Implementation of a BitTorrent Tracker Overlay for Swarm Unification alin-Andrei Burloiu, R˘ azvan Deaconescu, Nicolae T , ˘ apus , Automatic Control and Computers Faculty Politehnica University of Bucharest Emails: [email protected],{razvan.deaconescu,nicolae.tapus}@cs.pub.ro Abstract—The deployment of the BitTorrent protocol in the early 2000s has meant a significant shift in Peer-to-Peer tech- nologies. Currently the most heavily used protocol in the Internet core, the BitTorrent protocol has sparked numerous implementa- tions, commercial entities and research interest. BitTorrent uses the term “swarm” to define all peers taking part in an actual file transfer session. The swarm revolves around a “.torrent” file and one or more trackers. Although trackerless updates are common in DHT network overlays, most sessions still use a tracker to mediate communication among peers. In this paper we present a mechanism that allows integration of disparate swarms that share the same content. We’ve designed and implemented a novel inter-tracker protocol, dubbed TSUP, that allows trackers to share peer information, distribute it to clients and enable swarm unification. The protocol forms the basis for putting together small swarms into large, healthy ones with reduced performance overhead. Index Terms—Peer-to-Peer, BitTorrent, tracker, unification, TSUP, swarm I. I NTRODUCTION The continuous development of the Internet and the increase of bandwidth capacity to end-users have ensured the context for domination of content-distribution protocols in the Internet. Currently, most Internet traffic is content Peer-to-Peer traf- fic, mostly BitTorrent. Peer-to-Peer protocols have positioned themselves as the main class of protocols with respect to bandwidth usage. The arrival of the BitTorrent protocol in the early 2000s has marked a burst of interest and usage in P2P protocols, with the BitTorrent protocol currently being accounted for the biggest chunk in Internet traffic [3]. Modern implementations, various features, focused research and commercial entities have been added to the protocol’s environment. As its level of success grew, the BitTorrent protocol was also found to posses weaknesses in terms of free-riding [15], anonymity, the existence of centralized trackers, weak support for streaming. Various remedies for the before mentioned inconveniences have been researched and engineered, such as moderation, private/closed swarms, DHT, PEX, streaming extensions etc. In this paper we address the issue of unifying separate swarms that take part in a session sharing the same file. We propose a tracker unification protocol that enables disparate swarms, using different .torrent files, to converge. We define swarm unification as enabling clients from different swarms to communicate with each other. The basis for the unification is a “tracker-centric convergence protocol” through which trackers form an overlay network send peer information to each other. The protocol, in a manner similar to the OSPF network routing protocol, uses periodic updates to let all trackers in the overlay network, and subsequently, all peers in the P2P network, receive required information about other peers in the swarm. A. BitTorrent Keywords A peer is the basic unit of action in any P2P system, and, as such, a BitTorrent system. It is typically a computer system or program that is actively participating in sharing a given file in a P2P network. A peer is generally characterized by its implementation, download/upload bandwidth capacity (or lim- itation), download/upload speed, number of download/upload slots, geolocation and general behavior (aggressiveness, entry– exit time interval, churn rate). The context in which a BitTorrent content distribution session takes place is defined by a BitTorrent swarm which is the peer ensemble that participate in sharing a given file. It is characterized by the number of peers, number of seeders, file size, peers’ upload/download speed. A healthy swarm, one that allows rapid content distribution to peers, is generally defined by a good proportion of seeders and stable peers (peers that are part of the swarm for prolonged periods of time). A swarm consists of two types of peers: leechers and seeders. A seeder is a peer that has complete access to the distributed content and is, thus, only uploading data. A leecher has incomplete access to distributed content and is both uploading and downloading. The BitTorrent negotiation protocol uses a form of tit-for-tat that forces peers to upload in order to download (though this has been proven to be abused [15]) therefore ensuring fairness and rapid distribution. A good number of healthy seeders is a requirement for healthy swarms that allow rapid distribution of data. A swarm is given birth by an initial seeder which is the peer sharing a file it has complete access to. The seeding/sharing process within a swarm is started through a metafile, a .torrent file, that defines swarm trackers and piece hashes to ensure content integrity. Typically, a peer uses a web server to download a .torrent file and then uses a BitTorrent client to interpret it and take part into a swarm. It is possible to create different swarms for the same content by using different trackers in a .torrent file.

description

In this paper we present a mechanism that allows integration of disparate BitTorrent swarms that share the same content. We’ve designed and implemented a novel inter-tracker protocol, dubbed TSUP, that allows trackers to share peer information, distribute it to clients and enable swarm unification. The protocol forms the basis for putting together small swarms into large, healthy ones with reduced performance overhead.

Transcript of BitTorrent Unified Tracker (article)

Page 1: BitTorrent Unified Tracker (article)

Design and Implementation of a BitTorrent TrackerOverlay for Swarm Unification

Calin-Andrei Burloiu, Razvan Deaconescu, Nicolae T, apus,Automatic Control and Computers Faculty

Politehnica University of BucharestEmails: [email protected],{razvan.deaconescu,nicolae.tapus}@cs.pub.ro

Abstract—The deployment of the BitTorrent protocol in theearly 2000s has meant a significant shift in Peer-to-Peer tech-nologies. Currently the most heavily used protocol in the Internetcore, the BitTorrent protocol has sparked numerous implementa-tions, commercial entities and research interest. BitTorrent usesthe term “swarm” to define all peers taking part in an actual filetransfer session. The swarm revolves around a “.torrent” file andone or more trackers. Although trackerless updates are commonin DHT network overlays, most sessions still use a tracker tomediate communication among peers.

In this paper we present a mechanism that allows integrationof disparate swarms that share the same content. We’ve designedand implemented a novel inter-tracker protocol, dubbed TSUP,that allows trackers to share peer information, distribute it toclients and enable swarm unification. The protocol forms thebasis for putting together small swarms into large, healthy oneswith reduced performance overhead.

Index Terms—Peer-to-Peer, BitTorrent, tracker, unification,TSUP, swarm

I. INTRODUCTION

The continuous development of the Internet and the increaseof bandwidth capacity to end-users have ensured the contextfor domination of content-distribution protocols in the Internet.Currently, most Internet traffic is content Peer-to-Peer traf-fic, mostly BitTorrent. Peer-to-Peer protocols have positionedthemselves as the main class of protocols with respect tobandwidth usage.

The arrival of the BitTorrent protocol in the early 2000s hasmarked a burst of interest and usage in P2P protocols, with theBitTorrent protocol currently being accounted for the biggestchunk in Internet traffic [3]. Modern implementations, variousfeatures, focused research and commercial entities have beenadded to the protocol’s environment.

As its level of success grew, the BitTorrent protocol wasalso found to posses weaknesses in terms of free-riding [15],anonymity, the existence of centralized trackers, weak supportfor streaming. Various remedies for the before mentionedinconveniences have been researched and engineered, suchas moderation, private/closed swarms, DHT, PEX, streamingextensions etc.

In this paper we address the issue of unifying separateswarms that take part in a session sharing the same file. Wepropose a tracker unification protocol that enables disparateswarms, using different .torrent files, to converge. We defineswarm unification as enabling clients from different swarms tocommunicate with each other. The basis for the unification is a

“tracker-centric convergence protocol” through which trackersform an overlay network send peer information to each other.The protocol, in a manner similar to the OSPF network routingprotocol, uses periodic updates to let all trackers in the overlaynetwork, and subsequently, all peers in the P2P network,receive required information about other peers in the swarm.

A. BitTorrent Keywords

A peer is the basic unit of action in any P2P system, and, assuch, a BitTorrent system. It is typically a computer systemor program that is actively participating in sharing a givenfile in a P2P network. A peer is generally characterized by itsimplementation, download/upload bandwidth capacity (or lim-itation), download/upload speed, number of download/uploadslots, geolocation and general behavior (aggressiveness, entry–exit time interval, churn rate).

The context in which a BitTorrent content distributionsession takes place is defined by a BitTorrent swarm which isthe peer ensemble that participate in sharing a given file. It ischaracterized by the number of peers, number of seeders, filesize, peers’ upload/download speed. A healthy swarm, one thatallows rapid content distribution to peers, is generally definedby a good proportion of seeders and stable peers (peers thatare part of the swarm for prolonged periods of time).

A swarm consists of two types of peers: leechers andseeders. A seeder is a peer that has complete access tothe distributed content and is, thus, only uploading data. Aleecher has incomplete access to distributed content and isboth uploading and downloading. The BitTorrent negotiationprotocol uses a form of tit-for-tat that forces peers to upload inorder to download (though this has been proven to be abused[15]) therefore ensuring fairness and rapid distribution. A goodnumber of healthy seeders is a requirement for healthy swarmsthat allow rapid distribution of data.

A swarm is given birth by an initial seeder which is the peersharing a file it has complete access to. The seeding/sharingprocess within a swarm is started through a metafile, a .torrentfile, that defines swarm trackers and piece hashes to ensurecontent integrity. Typically, a peer uses a web server todownload a .torrent file and then uses a BitTorrent clientto interpret it and take part into a swarm. It is possible tocreate different swarms for the same content by using differenttrackers in a .torrent file.

Page 2: BitTorrent Unified Tracker (article)

Communication of peers in a swarm is typically mediatedby a BitTorrent tracker or several trackers which are definedin the .torrent file. It is periodically contacted by the peersto provide information regarding piece distribution within theswarm. A peer would receive a list of peers from the trackerand then connect to these peers in a decentralized manner. Asit uses a centralized tracker, the swarm may suffer if the trackeris inaccessible or crashes. Several solutions have been devised,such as PEX (Peer EXchange) [?] or DHT (Distributed HashTable) [12]. The tracker swarm unification protocol presentedin this article enables redundancy by integrating multipletrackers in a single swarm.

B. Tracker Swarm Unification Protocol

The goal of the tracker swarm unification protocol is theintegration of peers taking part in different swarms that sharethe same file. These swarms, named common swarms, usethe same content but different trackers.

We have designed and implemented a tracker networkoverlay that enables trackers to share information and integratepeers in their respective swarms. The overlay is based on theTracker Swarm Unification Protocol (TSUP) that allows up-date notification and delivery to trackers from the overlay. Theprotocol design is inspired by routing protocols in computernetworks. We have used XBT Tracker [11] as the basis forour current implementation.

At this point, as proof of concept, the tracker overlay isdefined statically. All trackers know beforehand the host/IP ad-dresses and port of the neighboring trackers and contact themto receive required information. The integration of dynamictracker discovery is set as future work. Each tracker may actas a “router”, sending updates to neighboring trackers that maythemselves send them to other trackers. We are thus using theterm “convergence” to signal the moment in which all trackershave access to all information.

The rest of the paper is structured as follows. In Section IIthe basic concepts behind swarm unification are presented. TheTSUP is described in Section III and the current implemen-tation is detailed in Section IV. Results and our experimentalsetup are presented in Section VI and Section V respectively.We conclude with final remarks and aspects to be investigatedin the near future in Section VII.

II. TRACKER UNIFICATION

A. Motivation

We define a healthy swarm as a swarm with a good percent-age of stable seeders. A client in a non-healthy swarm doesn’treach the maximum potential speed because the swarm has asmall number of stable seeders. It is also possible that differentusers create different .torrent files, but with the same files forsharing. If the .torrent files don’t refer the same tracker, eachone will represent another swarm. The peers from differentswarms do not know each other and integration/convergenceis not accomplished. This is quite different from other P2Pprotocol such as eDonkey or Gnutella where all peers are partof the same large P2P network.

By unifying swarms the communication between peers ispossible. Every client will have the opportunity of increasedconnections to other peers, increasing the download speed anddecreasing the download time.

B. Solution

The protocol proposed in this article, named TrackerSwarm Unification Protocol (TSUP), renders possible theunification of the swarms which share the same files, by em-ploying a tracker network overlay. A tracker which implementsthis protocol will be referred here as an unified tracker.

Figure 1. Unified trackers

According to the BitTorrent protocol specification [1], the.torrent file contains metainformation encoded into a dictionarystructure (keys ans values). There are two mandatory keysand some optional others. The first mandatory one, namedannounce, contains the announce URL of the coordinatingtracker and the second one, named info, contains as its value adictionary with information about the file(s) shared. The SHA1hash of the info dictionary, referred in the specification asinfo hash, is sent by the client in the announce requests andit is the only item which uniquely identifies the torrent file inthe tracker’s data structures.

In swarms which share the same file(s) (common swarms),peers will announce the same info hash. Therefore, TSUPcapable trackers can “unify” them by communicating witheach other in order to change information about peers whichcontribute to shared files with the same info hash. In orderto accomplish this, unified trackers send periodic updates toeach other, containing information about the peers from thenetwork.

III. TSUP

As mentioned above, TSUP is the acronym for TrackerSwarm Unification Protocol. TSUP is responsible with thecommunication between trackers for peer exchange informa-tion in common swarms.

Page 3: BitTorrent Unified Tracker (article)

A. Protocol Overview

For transport layer communication, the protocol uses UDPto reduce resource consumption, as a tracker already possessesTCP connections for communication with other peers. Thesimplicity of the UDP protocol gives an advantage of a smallercommunication overhead. Reliability is, however, absent fromUDP’s design, so TSUP must implement reliability mecha-nisms.

In TSUP’s operation the following processes may be iden-tified:

• Virtual connections establishment process: A three-way-handshake responsible with establishing a UDP“connection” between two linked trackers at the appli-cation layer. The process is started by a SYN packet(synchronization packet).

• Unification process: The trackers exchange unificationpackets (named SUMMARY packets) during a three-way-handshake in order to find out which are the commonswarms.

• Updating process: The trackers exchange peers fromcommon swarms, encapsulated in UPDATE packets.

• Election process: The establishment of a swarm leaderwhich is responsible with receiving all updates for fromthe neighboring linked trackers, aggregating them andsending the results back.

The design of the TSUP is heavily inspired by networkrouting protocols [7].

The unification process includes an updating process in itsthree-way-handshake, such that the two operations, unificationand update, are run in pipeline. Whenever a new torrent fileis registered to the tracker, a new swarm is created. Theunification process is triggered and a SUMMARY packet isimmediately sent to each neighboring tracker, informing theothers of the new swarm.

The updating process is started periodically, such thatUPDATE packets are sent at a configurable interval of time toeach tracker in a common swarm. A typical update interval is30 seconds.

In order to maintain the virtual connections between track-ers, HELLO packets are sent periodically, acting as a keep-alive mechanism. A typical HELLO interval is 10 seconds,but its value may be changed from protocol configuration. Ifno HELLO packet is received during a configurable interval,called disconnect interval, the virtual connection is droppedand the virtual connection establishment process is restartedfor that link by sending a SYN packet.

Some packets, such as UPDATE packets, must be acknowl-edged. If no answer or acknowledgement is received, thepacket is retransmitted. For example, UPDATE packets areresent at each hello interval until an acknowledgement isreceived.

Periodically sent packets, the keep-alive mechanism, ac-knowledgements and retransmissions contribute to the reli-ability of the TSUP. They help exceed the drawbacks ofthe UDP transport protocol, and also give a more efficient

communication than a TCP one.

B. Tracker Awareness

Tracker communication is conditioned by awareness. Forthis purpose, in the current version of the protocol, each trackeris configured statically with a list of other communicatingtrackers. Each element of the list represents a link which isidentified by the tracker host name (URL or IP address) andport. Other parameters for the link may be configured; someof them may be specific to the implementation. If the virtualconnection establishment process is successful for a link, itbecomes a virtual connection, which is conserved with keep-alive packets (HELLO packets).

A future version of the protocol will incorporate the designof a tracker discovery mechanism capable of generating the listof communicating trackers for each tracker dynamically, withthe benefit of scalability and reduction of the administrativeoverhead.

C. Tracker Networks

To improve TSUP’s scalability, trackers may be groupedtogether in networks named tracker networks. Connections inall tracker networks are full mesh. Two networks are connectedwith the aid of border trackers (see Figure 2).

Figure 2. Tracker Networks

To configure a topology which contains multiple networks,each link of each tracker must be set as an internal link or anexternal link (see Figure 2). Trackers connected with internallinks are part of the same tracker network; trackers connectedwith external links are part of other networks. However, theones from the first category may also be classified as belongingto an internal network and the ones from the second categoryas being from an external network. A complete graph, usinginternal links as edges is an internal network, and a completegraph with external links as edges is an external network (seeFigure 2). A tracker which has both internal and external linksis a border tracker. Peer information received from an internalnetwork originates from internal peers while information

Page 4: BitTorrent Unified Tracker (article)

received from an external network originates from externalpeers. Peers connected to a tracker via TCP, via the BitTorrentprotocol, are called own peers. The links between trackers ina network, whether internal or external, must be full mesh.

In order to use a scalable and low resource consumingcommunication within a network, trackers are organized ingroups depending on the unified swarm. Therefore a trackermay belong to more than one group at the same time, thenumber of groups it belongs being equal with the number ofswarms present on that tracker. Each group contains a swarmleader responsible for sending peer updates to peers in thegroup. The other group members, instead of sending updatesto other members on a full mesh graph (as seen in Figure 3(a)), send updates to the swarm leader on a tree graph (as inFigure 3 (b)). These updates propagate to other peers in thegroup.

Figure 3. (a) A full mesh update in tracker network; (b) An update in atracker network with the aid of a swarm leader (tree graph)

The number of updates sent in a swarm without the swarmleader mechanism (full mesh) may be computed by using theabove formula:

UPDATESfullmesh = 2 · n(n−1)2 = n(n− 1)

The number of updates sent within a swarm using the swarmleader mechanism (tree) are:

UPDATESswarmleader = 2(n− 1)

As may be observed from the formulas above the complex-ity decreases from O(n2) in a full mesh update scheme toO(n) with the swarm leader scheme.

Each swarm contains two swarm leaders, one for the internalnetwork (which sends updates through internal links), calledinternal swarm leader and one for the external network (whichcommunicates updates through external links), named externalswarm leader.

As connections are full mesh in an internal network, theinternal peers (received from other trackers from the internalnetwork) are distributed to other internal trackers only bythe internal swarm leader and in no other circumstance by

another tracker. Through analogy, in an external network, peers(received from other trackers from the external network) aredistributed to other external trackers only through the externalswarm leader. On the other hand, internal peers are distributedto external trackers and external peers to the internal trackers.Own tracker peers are distributed both to the internal and theexternal network.

Swarm leaders are automatically chosen by trackers duringthe election process which is started periodically. There aremetrics used in order to choose the most appropriate leader.The first and most important one prefers as swarm leader atracker which possesses the smallest number of swarm leadermandates. The number of mandates is the number of swarmswhere a tracker is swarm leader. This balances the load of thetrackers – as the number of mandates of a tracker increases,its load also increases. In the current version of the protocolthe grouping of trackers into networks and the selection ofborder trackers is done manually (statically) by the systemadministrator.

When two network trackers A1 and A2 are connectedindirectly through other network trackers Bj , if A1 and A2 usea common swarm and Bj doesn’t use this swarm, then the Ai

trackers cannot unify unless the border trackers are specified inthe configuration. This happens because the configured bordertrackers must unify with any swarm, although the network theyare part of do not contain that swarm.

Grouping trackers in networks increases system scalabilityand network convergence time. The update timers can be setto a lower value for border trackers to limit convergence over-head. The system administrator should opt between scalabilityand convergence and adapt the protocol parameters to thespecific topology.

IV. IMPLEMENTATION DETAILS

The TSUP is currently implemented in the popular XBTTracker [11], developed by Olaf van der Spek in C++. Thedecision was based on its efficient implementation using apowerful and flexible programming language. The trackeris cross platform, containing directives for compiling andrunning it both on Linux and on Windows environments.The extended TSUP capable tracker was dubbed XBT UnifiedTracker.

The original XBT Tracker implements an experimental UDPBitTorrent protocol known as UDP Tracker. As TSUP alsouses UDP and as communication takes place using the sameport, TSUP-specific packets use the same header structureas the UDP Tracker, enabling compatibility. The unificationprotocol is mainly implemented in the UDP Tracker part ofthe XBT Tracker source code, with some functionality in theserver’s main loop.

XBT Tracker uses a MySQL database [10] for configurationparameters [9] and for communication with a potential frontend. Visigod implemented a web front end in Joomla forthis tracker [8]. XBT Unified Tracker adds parameters forconfigurations that are specific to TSUP; it also uses a newtable in order to remember links with other trackers and

Page 5: BitTorrent Unified Tracker (article)

their parameters. Tracker awareness, as described in III-B, isimplemented in the database. A front end may be used tosimplify tracker linking.

Besides the HTTP announce and scrape URLs, the originaltracker uses other web pages for information and debuggingpurpose. The unified tracker adds two extra information webpages for monitoring. The trackers web page shows detailsabout every link and the state of the connection for that link.For every swarm, the swarms web page shows the list of peersand the list of trackers connected for that swarm.

Other implementations of TSUP, running on different plat-forms, are possible. Each platform will provide specific param-eters and different ways for configuration of tracker awarenessestablishment, monitoring and operation.

V. EXPERIMENTAL SETUP

In order to test and evaluate the tracker overlay and theTracker Swarm Unification Protocol, we have used a vir-tualized infrastructure and a Peer-to-Peer testing frameworkrunning on top of it. We were able to deploy scenariosemploying various swarms, ranging from a 4-peer and 1-tracker swarm and a 48-peer and 12-tracker swarm. Apartfrom testing and evalution, the infrastructure has been used tocompare the proposed tracker overlay network with classicalswarms using a single tracker and the same number of peers.We will show that a unified swarm has similar performancewhen compared to a single tracker (classical) swarm.

The infrastructure is constructed on top of several commod-ity hardware systems in the NCIT cluster from the UniversityPolitehnica of Bucharest. In order to deploy a large numberof peers we have used a thin virtualization layer employingOpenVZ [5]. OpenVZ is a lightweight solution that allowsrapid creation of virtual machines (also called containers).As an operating system-level virtualization solution, OpenVZenables creation of a large number of virtual machines (30virtual machines are sustainable on a 2GB system), each ofwhich is running a single BitTorrent client instance.

All systems are identical with respect to hardware andsoftware components: 2GB RAM, 3 GHz dual-core CPU,300GB HDD, 1Gbit NIC running Debian GNU/Linux 5.0Lenny. The deployed experiments used a single OpenVZcontainer either for each tracker or peer taking part in a swarm.A virtualized network has been build allowing direct linklayer access between systems – all systems are part of thesame network; this allows easy configuration and interraction.Peers and trackers are started and managed through the P2Ptesting framework. A commander station (typically an unusedcontainer) is used to start required applications and collectresults.

The experiments made use of an updated version of hrktor-rent [2], a lightweight application built on top of libtorrent-rasterbar [4]. Previous experiments [14] have shown libtorrent-rasterbar outperforming other BitTorrent experiments leadingto its usage in the current experiments. The hrktorrent has beenupdated to make use of bandwidth limitation facilities providedby libtorrent-rasterbar. The experiments we conducted used a

limitation typical to ADSL2+ connections (24 Mbit downloadspeed limitation, 3 Mbit upload speed limitation).

An automatically-generated sample output graphic, describ-ing a 48 peer session (12 seeders, 36 leechers, 12 trackers)sharing a 1024 MB file is shown in Figure 4 (TODO: changethe picture with a new one). The image presents downloadspeed evolution for swarm peers. All peers are limited to 24Mbit download speed and 3 Mbit upload speed. Such graphicshave been used as basis for comparison with classical swarms.

tsup−mta−3−2011.01.22−02.39.51

tsup−mta−3−15: a test swarm ( 12 Seeders, 36 Leechers, 12 Trackers),

Time(s)

Dow

nlo

ad

sp

ee

d (

Mb

it/s

)

0.00.51.01.52.02.53.03.54.04.55.05.56.06.57.07.58.08.59.09.5

10.010.511.011.512.012.513.013.514.014.515.015.516.016.517.017.518.018.519.019.520.020.521.021.522.022.523.023.524.0

0 500 1000 1500 2000 2500 3000

Figure 4. Sample Run Graphic

VI. SCENARIOS AND RESULTS

In order to test the overhead added by TSUP to BitTorrentprotocol, we have made a set of test scenarios which comparethe average download speed for a swarm with unified trackersand for another swarm with just one non-unified tracker, butthe same number of leechers and seeders. Each test scenariois characterized by the shared file size, the number of peersand, in the case of tests with unified trackers, by the numberof trackers. We shared 3 files of sizes 64MB, 256MB and1024MB. In the test scenarios with unified trackers for eachfile we tested the swarm with 1, 2, 4, 8 and 12 trackers. Oneach tracker there were connected 4 peers, from which 1 is aseeder and 3 are leechers. So, for example, in a scenario with8 trackers there are 8 seeders and 24 leechers, totalizing 32peers. Having 3 files and 12 trackers in the biggest scenario weneeded to create 36 .torrent files, because for each shared filewe made a .torrent file for each tracker. In the correspondingtest scenarios with non-unified trackers (classical trackers),there is just one .torrent file for each shared file. Here, becausethe number of trackers remains constant, we had to vary thenumbers of seeders and leechers connected to the tracker sothat they have the same cardinality with the correspondingunified trackers scenarios.

Each test scenario has been repeated 20 times in order toallow statistical relevance. The average download speed wascalculated as an average value from the 20 sessions.

The set of scenarios with unified trackers used the XBTUnified Tracker implemented here and those with non-unified

Page 6: BitTorrent Unified Tracker (article)

trackers used the original XBT Tracker, developed by Olaf vander Speek.

Results may be seen in the table at the end of this article,which depicts the results for each file size, in the top (64MB),middle (256MB) and bottom part of it (1024MB), respectively.Under the “Single Tracker” header, there are the measuredvalues from the tests with non-unified trackers and under theheader titled “Unified Trackers”, the values from the tests withunified trackers can be seen. For each of this two situations themean download speed (“mean dspeed”) and relative standarddeviation (“rel.st.dev.”) is depicted. In the right part, titled“perf. Decrease” (performance decrease), shows what is thepercent of download speed decrease induced by the overheadof the TSUP. In the left side of the table the number of seedersand leechers for each scenario is shown. All download speedsare expressed in kilobytes per second (KB/s). The percentagevalue for download speed decrease is computed using thestandard formula:

decrease = 100% · dsSingleTracker−dsUnifiedTrackers

dsSingleTracker,

where ds is an abbreviation from mean download speed.

If the download speed decrease percentage is negative, itthat there is an increase instead of a decrease. All positivepercentage values from the “perf. decrease” header mark adecrease of performance caused by the TSUP overhead.

The trackers were started at the beginning of each scenario,whether there was XBT Tracker or there were more XBTUnified Trackers.

The unification which takes place in the XBT UnifiedTracker introduces an overhead in the BitTorrent protocolin comparison to XBT Tracker. In theory the performancedecrease must always be positive. But there are situationswhere the percentages are negative, suggesting that the TSUPincreases the speed, thus reducing the download time. Theperformance increase is not due to TSUP. In all scenarios,at the beginning all peers are started almost simultaneously,creating a flash crowd. In the Single Tracker experiments thecommunication between peer will start immediately; whenmultiple trackers are unified, the TSUP imposes a delaybefore each peer finds out of all the others, because of theconvergence time. This fact avoids the appearance of a flashcrowd. It is known that sometimes it is better when peersenter the swarm later [13]. We believe that this is the reasonof the performance decrease, which has little relevance for ourexperiment set.

By looking at the numerical values from the results, twothings may be observed. The TSUP overhead becomes moreinsignificant, on the first hand, when the number of peersincreases (and proportionally the number of seeders) and, onthe other hand, when the file size increases. When the overheadis insignificant, the percentages have lower values. The relativestandard deviation is generally increasing with the numberof peers, but is decreasing when the file size increases. Weconsider the values to be normal, considering the number ofpeers that are part of a swarm.

Due to the small values of performance decrease and relativestandard deviation, we concluded that TSUP overhead isinsignificant for small to medium-sized swarms (less than 50peers) which share big files (1GB). BitTorrent is generallyused for sharing large files and TSUP allows the increase ofswarms size; these two factors come as an advantage for thistechnology.

VII. CONCLUSION AND FURTHER WORK

A novel overlay network protocol on top of BitTorrent,aiming at integrating peers in different swarms, has been pre-sented. Dubbed TSUP (Tracker Swarm Unification Protocol),the protocol is used for creating and maintaining a trackernetwork enabling peers in swarms to converge in a singleswarm. Each initial swarm is controlled by a different tracker;trackers use the overlay protocol to communicate with eachother and, thus, take part in a greater swarm.

We have used an OpenVZ-based Peer-to-Peer testing infras-tructure to create a variety of scenarios employing an updatedversion of the XBT Tracker, dubbed XBT Unified Tracker. Theprotocol incurs low overhead and overall performance. Theunified swarm is close to the performance of single-trackerswarm consisting of the same number of seeders and leecherswith the benefit of increased number of peers. The increasednumber of peers provides the basis for improved informationfor various overlays (such as social networks) and allows ahealthier swarm – given enough peers, if some of them decideto leave the swarm, some peers will still take part in thetransfer session.

Current results show that the addition of new trackers andan overlay protocol allows similar performance to a classicalswarm. Experiments have involved different swarms, withrespect to the number of peers and trackers, and differentfile sizes using simulated asymmetric links. The table at theend of the article shows a summary of results. Performancewas shown to be similar with the additional benefit of a largeswarm for the unified tracker specific scenarios. As expected,the result dispersion is higher in case of larger swarms and/orincreased number of trackers.

At this point each tracker uses a statically defined pre-configured list of neighboring trackers. One of the maingoals for the near future is to enable dynamic detection ofneighboring trackers and ensure extended scalability. We arecurrently considering two approaches: the use of a trackerindex where trackers’ IP/host addresses and ports are stored orthe use of a completely decentralized tracker discovery overlaysimilar to DHT’s discovery methods. At the same time we planto address scalability issues that may arise from contacting alarge number of trackers and storing peer/tracker details.

As proof of concept, our test scenarios have focused onhomogeneous swarms. All peers in swarms are using the sameimplementation and the same bandwidth limitation. All peersenter the swarm at about the same time, with some delay untilswarm convergence, in case of the unified tracker protocol.We plan to create heterogeneous swarms that use differentclients with different characteristics. The number of seeders

Page 7: BitTorrent Unified Tracker (article)

and leechers in initial swarms are also going to be altered andobserve the changes incurred by using the unification protocol.We also plan to test the scalability limit of the TSUP, byvariating the number of connected trackers and measuring theirimpact on the swarm performance and BitTorrent protocolmessages delays.

As with real swarms, we plan to add diversity in peers’behavior, such as using different upload slots and introducechurn. The effects of using peers with diverse behavior aregoing to be investigated.

ACKNOWLEDGMENT

The authors would like to thank Alex Heris, anu for kindlyproviding access to the NCIT cluster systems we have beenusing throughout our experiments.

Special thanks go the the P2P-Next [6] team who is workingenthusiastically to deliver the next generation peer-to-peercontent delivery platform. Their dedication, professionalismand vision are a constant factor of motivation and focus forour work.

REFERENCES

[1] BitTorrent Specification. http://wiki.theory.org/BitTorrentSpecification.[2] hrktorrent. http://50hz.ws/hrktorrent/.[3] ipoque Internet Studies. http://www.ipoque.com/resources/internet-

studies/internet-study-2008 2009.[4] libtorrent (Rasterbar). http://www.rasterbar.com/products/libtorrent/.[5] OpenVZ. http://wiki.openvz.org/.[6] P2P-Next. http://www.p2p-next.org/.[7] RFC 2328 - OSPF Version 2. http://tools.ietf.org/html/rfc2328.[8] The XBT Tracker frontend. http://www.visigod.com/xbt-tracker-

frontend.[9] XBT Configuration Options . http://www.visigod.com/xbt-

tracker/configuration.[10] XBT Table Documentation. http://www.visigod.com/xbt-tracker/table-

documentation.[11] XBT Tracker by Olaf van der Spek. http://xbtt.sourceforge.net/tracker/.[12] H. Balakrishnan, M. F. Kaashoek, D. Karger, R. Morris, and I. Stoica.

Looking up data in p2p systems. Commun. ACM, 46:43–48, February2003.

[13] A. R. Bharambe, C. Herley, and V. N. Padmanabhan. Understandingand Deconstructing BitTorrent Performance. Technical Report MSR-TR-2005-03, Microsoft Research, 2005.

[14] R. Deaconescu, G. Milescu, B. Aurelian, R. Rughinis, , and N. T, apus, . AVirtualized Infrastructure for Automated BitTorrent Performance Testingand Evaluation. International Journal on Advances in Systems andMeasurements, 2(2&3):236–247, 2009.

[15] T. Locher, P. Moor, S. Schmid, and R. Wattenhofer. Free Riding inBitTorrent is Cheap. HotNets, 2006.

Page 8: BitTorrent Unified Tracker (article)

Sin

gle

Tra

cker

Unifi

ed

Tra

ckers

seeders

leechers

meandspeed

(KB/s)

rel.st.dev.(%

)trackers

meandspeed

(KB/s)

rel.st.dev.(%

)perf.decrease

(%)

size

=64M

B 13

337.

731.

281

335.

112.

870.

78

26

472.

270.

876

396.

5516

.64

16.0

3

412

475.

131.

874

463.

4212

.60

2.46

824

476.

106.

068

497.

3511

.10

-4.4

6

1236

470.

539.

6112

496.

4811

.48

-5.5

2

size

=25

6MB 1

335

8.72

0.45

135

6.55

1.07

0.6

26

476.

590.

266

407.

5514

.67

14.4

9

412

477.

560.

454

477.

4510

.73

0.02

824

492.

406.

648

500.

568.

49-1

.66

1236

486.

048.

3312

494.

9312

.38

-1.8

3

size

=10

24M

B 13

365.

270.

191

365.

570.

31-0

.08

26

477.

470.

156

437.

099.

128.

46

412

477.

540.

184

466.

035.

932.

41

824

423.

716.

198

418.

698.

731.

18

1236

407.

714.

4712

418.

766.

27-2

.71

Tab

le1:

Sin

gle

Tra

cker

vs.

Unifi

edT

rack

ers

Com

par

ison