My Second Bike: A TV-enabled social andinteractive riding experience

Jaewoo Chung∗, Kuang Xu∗†, Andrea Colaco∗, Chris Schmandt∗, Victor O.K. Li†∗Media Laboratory, Massachusetts Institute of Technology

†Department of Electrical & Electronic Engg., The University of Hong Kong

Abstract—In this paper, we propose a novel concept for a socialTV application targeting the demographic of viewers enjoyinglive sports events, such as road bicycle racing. We intend toenhance the viewing experiences of spectators with sensor-fittedbikes tied to an interactive biking environment on television. Thesystem enables a new form of personalized, physical, and virtual-reality interaction between viewers and a TV program, as well asinteractions within or between communities of friends. We alsodescribe a prototype we have implemented to demonstrate thefeasibility of our idea. The prototype, My Second Bike, uses a 3Dmirrored world environment (Google Earth) to visually representparticipating spectators, competing athletes and outdoor bikers.We contend that the system has the potential to attract andsupport a large user base on account of its scalability, ease ofdeployment and ability to promote audience participation in livesports events on TV.

I. INTRODUCTION

Shared experiences around a television have been a drivingforce for co-viewing content. Sport-related events, moreover,tend to create a huge impact, bringing together large audiencesto participate both physically and socially. People engage insports for recreation, exercise, and also as spectators. Veryoften spectators vicariously participate in the experiences ofthe athlete by following the event closely - either live or onscreen. It is not uncommon to find oneself imitating the actionsof an athlete on screen or even repeating the lines of a favoritesitcom character. These aspects of television-related contenthave potential to be channeled into areas that can benefit fromsustained engagement and interactivity. My Second Bike isa platform to create a social riding experience by providinga meeting space for bikers physically riding for recreation,bikers competing in sporting events and bikers exercisingat home. The platform provides each rider a representationon screen and visual cues of nearby riders. By creating asense of participation via direct visual/audio feedback and co-presence, My Second Bike aims to have a reinforcing effect inachieving sustained participation. Drawing from the positiveeffects that competitive gaming and sports events have, MySecond Bike supports riders challenging each other to racesand competitions. The framework allows biking related andpresence information of real-world riders to be fed into thescreen environment. This allows home-riders to challenge theirreal-world counterparts to biking races or even an exploratoryride. To capture the strong element of enjoying a competitivesporting event on television and creating a feel of participatingin it, My Second Bike has a provision for riders to navigate

the terrain a bike-racing team may currently be climbing. Allthese aspects of the system can potentially create a communityof bikers each with their own preference for riding all linkedby a common platform.

Further, the system is developed using easily accessible3D environments such as Google Earth, rendering it scalableand deployable to large audiences. This could potentiallycreate similar effects as gaming consoles and merge into oneapplication the key sociability and collaborative aspects ofgaming with television screens - a sociable riding experience.One can imagine a typical user scenario using this system aselaborated below:

Matt is a high school student who commutes to schoolby bike everyday and likes hiking during the weekends withhis friends. Today, he is watching TV featuring the Tour deFrance. He opens a gadget in the TV, which lists a menuto choose which riding group he would like to join. Mattselects his school’s community group that his friends have justjoined. On the left of the TV screen, he sees video streamingfrom France and on the right half of the screen, he sees amirrored world of 3D model with bike avatars that representhis bike, athletes’ bikes and others who are currently partof the group. He also recognizes other bikes that represent anearby rival school’s biking community. Not only competingwith the athletes and participants of his group, but Matt is alsocompeting as a team with other groups. Matt turns on thedevice attached on his bike (mounted on a bicycle trainer),which detects rotation of his back wheel and the angle ofthe handle. The device is connected to his TV via Bluetoothautomatically. The viewers are not simply enjoying the Tourde France, but also participating in the very activity theyare watching on television. They’re also able to connect andshare this experience and activity with members of their socialnetworks and online communities.

In sections to follow, we describe background work in thisarea (Section II) followed by system overview (Section III)and implementation details (Section IV); we also discusskey features different from existing interactive applicationsand limitations of our system (Section V). We concludewith possible additional useful features to the current system(Section VI).

II. BACKGROUND

There has been an impressive surge in social and interactiveTV applications. The television experience has shifted from

being a passive medium for distribution of professionallyauthored narrative-rich content to a combination of narrativityand interactivity as well as the ability to manipulate content.Further, opportunities for interacting with screen content andpeople co-viewing the same content are increasingly being pro-vided. In many such applications common forms of engagingviewers include voting on a topic, conversations via instantmessaging on issues emerging from on-screen content, aware-ness of remote viewers and discussions in online communities.With these trends in place viewers are increasingly interestedin interacting with their screen content and as a result a numberof new social television applications have been seen in themarket in the past few years.

2BeOn [6] is an interpersonal communication service forTV users to communicate with their peers watching thesame TV show as themselves. It was an attempt to reinforcesocialization between viewers and took detailed account of thelimitations of TV as an output terminal as well as the role ofcontent in promoting conversations. Along similar lines, Real-ity Instant Messaging [9] provides services for conversationsthat involve groups of users added to a bot interface rather thanfocusing on desktop-like single user applications. Additionally,it also ties this social network service with real-time sportsevents (in their example, golf). The system sends out syncedinformation about events in the game and participants candiscuss these via the bot interface. Cabo Interactiva [11]in Portugal offers a number of interactive TV applicationsincluding shopping, banking, communicating and sport-relatedapplications. Most popular among these was their Sports TVchannel’s interactive football that offered video stream selec-tion, participating in debates and polls, instant shopping forfootball products, etc. It turned out that the forums witnessed alot of activity, with active comments, cheering and competitionamong supporters of opposing teams. Similarly, many othersolutions have been proposed and implemented to capture andrepresent a person’s social network while watching television,sharing multimedia messages, finding friends and their favoriteprograms and interacting while viewing content.

Several TV applications specifically based on live sportsevents have been proposed. [14] aims at enabling real-timeinteraction between audiences and the video director, so thatthe video director can schedule future broadcasts based onviewers’ feedback about which parallel sports event they enjoymore. [10] endeavors to facilitate personalized video deliveryby enabling a viewer to customize the live sports videoaccording to his own interest. Based on multi-camera angleselection, athletes/objects matching the personal preferencescan be automatically tracked. To enhance the collective view-ing experience, [12] further proposes a model that learns anddevelops rules for group preferences by clustering, improvingsocial communication and sense of belonging to a group,among spectators. However, our work is different in that,we intend to engage viewers to physically interact with andparticipate in a TV program, thereby increasing the sense ofrecreational and social connectedness among them.

On the other hand, the concept of using a real bike as

Figure 1. Topology of data communication architecture between content-provider, service provider and clients of TV viewer.

controller of a virtual bike in Virtual Reality environments hasbeen explored many times during the last decade [7] [8] [13].In recent years, many console based arcades and on-line gamesalready have made sophisticated multi-user based interactiveracing games, but, to date, we have not encountered a projectthat utilizes the two parts together – interactive TV and virtualreality games – to allow viewers to participate in the sameactivity as the live event being broadcast through TV channelse.g, live sports games.

III. SYSTEM OVERVIEW

Implementing the system described in the scenario (in Sec-tion I) requires the following three main components: content-providers (or broadcasters), service-providers, and clients (orcontent consumers). The main task for the broadcaster is tosend two types of information, video and audio (TV content),and metadata that is associated with the content. In our case,we simulated the data portion that is streaming the metainformation to viewers via IP. A Service provider server servesas an aggregator of metadata from an event broadcaster andregistered users. The gathered data will be re-distributed tothe registered users so that the client side at each user canvisualize all the registered users’ and athletes’ bikes. Themain functions for the clients are to exchange metadata withthe server and visually represent this metadata. In detail, thetasks are: i) collecting control data from the bike via a devicethat detects rotation of the back wheel and heading anglefrom the handle, ii) generating and sending out metadataassociated with a given user’s bike, and iii) based on thereceived streams of metadata from the Service-provider Server,simulating and visually representing biking participants on TV.This visualization serves as a direct feedback of the user’sperformance compared with other avatars’ relative positionsand performance on the TV screen.

As Fig. 1 depicts, the network connections between server-server and server-clients are essentially HTTP requests overthe Internet. Between the content providers and server-client

Figure 2. An evaluation scenario of the system prototype.

sides, the data network can be one of many options viz. satel-lites, TV cable lines, Internet infrastructure, etc. that supporttransmitting metadata from broadcasters. Multiple servers canhost clients, and each server can connect to any numbersof peer servers and exchange metadata over the Internet byopening a socket connection - an example connection is shownbetween Server B and Server C in Fig. 1. The athletes’ datacan be collected via short distance radio transmitted from aGPS equipped device to nearby outdoor broadcasting van thatis following the athletes. Alternatively, a GPS equipped phonecan transmit GPS coordinates, Athletes’ IDs, and headinginformation to content providers over GPRS. In our case, weused the latter method.

IV. IMPLEMENTATION

We have implemented a working prototype of our proposedsystem. The prototype demonstrates the core functionality ofthe system for multiple users. It also highlights the feasibilityof deploying and scaling this system. Fig. 2 shows an evalu-ation scenario of the system prototype, in which two indoorviewers ride their own bikes enjoying the same TV program ofroad bicycle racing while physically based in different places.On the TV screen, the left part displays the TV program andthe right part shows the presence of viewers and athletes asavatars in a 3D mirrored world utilizing Google Earth’s API.

The main components that make up the system includethe Content-provider Server (Broadcaster), Service-providerServer (Application server), Clients (content consumers) thatconsist of a desktop computer connected to an HDTV, andsensor-fitted Bikes. In the following we describe these partsin detail.

A. Content-provider Server (Broadcaster)

The Broadcaster in My Second Bike is the televisioncontent provider - any digital TV broadcaster could poten-tially function as a content-provider server. In the prototype,we equip the bike of an outdoor biker with a cell phone(Motorola i870) with Assisted Global Positioning System (A-GPS), which provides not only the GPS coordinates, butalso the heading information. We have implemented a light-weight J2ME application that runs as a background process to

Figure 3. The metadata object.

retrieve GPS coordinates periodically to gather information forgenerating metadata, which contains essential information forour system to work. Fig. 3 depicts the content and format ofthe metadata. During our experiment, we observe the Altitudeitem in the metadata is inaccurate as provided by GPS; so wedo not transmit this information through the network, and wecalculate it at the client side.

The application sends the metadata to the Content-providerServer via UDP (User Datagram Protocol) over IntegratedDigital Enhanced Network (iDEN) [2]. During the experiment,we observe the average delay for data transmission was around220ms. The Content-provider Server listens to this incomingmetadata and synchronously passes this information on toany Service-provider Server connected via TCP/IP over theInternet. In our implementation of the Content-provider server,we have omitted the transmission of video and audio.

B. Service-provider Server

The Service-provider Server hosts a database of real-timemetadata, serving as an intermediary between different clients,and between clients and Content-provider Server. We utilizeJava Servlets to realize the server-side function: a TCP daemonis set up to initiate connection to the Contentprovider Serverand to listen for incoming metadata from the Content-provider.For communication with clients, we utilize the Direct WebRemoting (DWR) [1] toolkit which supports the reverse-AJAXfeature. We use the Piggyback mode such that when a clientbrowser requests to upload its status to the Service-providerServer, the server sends the most recently updated metadataof other clients along with the response that the client isexpecting. This also reduces unnecessary network transmissioncost and eases the server load. Metadata received by the server(from both content provider and clients) is stored in a databaseand communicated to intended clients.

To minimize network resource consumption, clients canfilter metadata information of other participants by settingproximity boundaries - the server would then have to sendmetadata of clients within a boundary specified. Additionally,there is an audio component in this system design whichsends out audio alerts informing clients about bikers in theirvicinity, as well as updates about their performance relative tocompeting bikers.

Before exchanging metadata between the Service-providerServers, each server computes metadata that holds an averageof the locations of the connected clients. This allows eachserver to not only reduce the amount of metadata but alsomake a Service-provider Server represent a particular team ofparticipants. Thus, participants can compete not only individ-ually but also as a team member of a community, e.g. myhigh school team represented by a Service-provider Server ornearby rival school server. This promotes each server to serve

as a local/online community. In practice, with large-scale userparticipation, this community-based service can be hosted byany social networking website such as Facebook.

C. Client

The client consists of a desktop computer connected to anHDTV. It uses a 3D model rendered by Google Earth. Theclient requires a browser that is compatible with Google EarthAPI, supports JavaScript and OpenGL. The Google Earthmodel is linked to our Service-provider Server and presenceof riders is updated and visually represented as avatars inthe rendering. Using Google Earth has many advantages,important among them are:• reasonably accurate 3D models - thereby offering a huge

number of places for users to choose from.• open source API and hence customizable and easily

accessed by clients at no cost.• dynamic and continuous refreshing of the 3D model,

which can be exploited by other components of thesystem to update location information.

• large user base.• ease of scalability - even a large number of clients

concurrently accessing the system does not overload ourService-provider Server as they are handled directly bythe Google Earth server.

Further, both Google Earth and our Service-provider Serveruse AJAX to update client information and synchronize in-formation between clients. This is an added advantage toour system in that clients do not have to make a requestto the server and wait for a response; rather, the serverasynchronously sends client information whenever availableand necessary. The metadata is communicated between clientand server using JavaScript Object Notation (JSON) format.These JSON packets being object-based are easily re-usableand save sending multiple packets in response to multiplerequests. It also aids in solving multi-client synchronizationsince the metadata of all the involved clients are packed inone JSONArray for transmission. In the prototype, each clientuploads its metadata periodically (every half second).

D. Sensor-fitted Bike

The Smart Bike in our system is an instrumented bikeretrofitted with sensors and encoders, capable of communicat-ing information to our Service-provider Server. The connectionbetween the Client (simulated in desktop PC) and sensor-fitted Bike is through virtual COM-port which is created fromBluetooth-pairing between bike and TV client in MicrosoftWindows OS. However, communication between COM portand web browser using JavaScript is not usually supported.Our solution to this is to utilize ActiveX to tunnel a channelthat allows JavaScript to communicate to COM port [4]. Thisenables receiving sensor data by web browser with JavaScriptthat allows visualization of Google Earth 3D model andgenerates metadata for sending it to the Service-provider everysecond.

Figure 4. The installation of powerless rotary encoders on a bike.

In the prototype, the bike is mounted onto a bicycle trainerso that a user can ride it in front of the TV screen. Therequirement is that the bike should be able to continuouslymonitor its motion status, such as heading direction anddistance traveled, as if the user is riding in the physical outdoorscenario, and upload them to the Service-provider server inreal time. We utilize two powerless 60-step rotary encodersas sensors that are wired to a button-battery powered microBluetooth controller on the bike. One sensor is mounted onthe shaft of the bike knob, to detect the angle of the knobwhen a user makes a turn. Another one is installed on the hubof the bike’s back-wheel, which is responsible for reading therotation status of the wheel. Our rotary encoders have 60 stepsgranularity that provides resolution of 6 degrees. The micro-controller board collects the sensor data every millisecond. Itpre-processes this data to calculate the heading direction andspeed of the bike, and transmits the result to the TV clientthrough Bluetooth radio every 30 milliseconds, so that the rateof updating user status is higher than the frame refreshing rateof HDTV. We find that the 6 degree resolution works wellfor detecting both handle-turning and wheel-rotation. Fig. 4shows the installation of sensors on a bike. In practice, if wecustomize the control-board and utilize standard UART portinstead of Bluetooth for data communication between bike andTV client, the cost of manufacture is less than $20.

V. DISCUSSION

My Second Bike has proposed and prototyped a newparadigm for social interaction with televisions. Watchingsports-related television content is an inherently social activitythat engages large audiences and encourages participation.Our implementation, as described, leverages these aspectsof sports-spectating. Therefore, we anticipate the system toincrease the sense of social connectedness among viewers,promote an immersive experience and better represent remotepresence in television environments. This system differs from

current forms of viewing and has potential to support a largeuser base and sustain interactivity.

Additionally, using Google Earth as a 3D riding environ-ment can also contribute positively to the user experience as itmirrors the real world in contrast to other virtual environmentssuch as Second Life. The familiarity with a particular realworld space may make the navigating experience easier andmore engaging. Alternatively, an unknown area that maps toan interesting area in the real world could also engage usersto undertake exploratory challenges while riding.

However, in implementing My Second Bike some technicalchallenges were encountered that could impact the overall userexperience as well as scalability.• GPS accuracy. Imprecise GPS coordinates can result in

misplacement of avatars in the 3D model, which mayplace the avatar off the road. This problem can beovercome by snapping GPS coordinates to the race coursewhich would place avatars correctly on the riding path.

• AJAX same origin policy. Due to security (data confiden-tiality and integrity) requirements imposed by modernWeb-browsers, certain elements such as XMLHttpRe-quest object in AJAX suffer from the same origin policywhich prevents the application from connecting to do-mains different from its domain of origination. However,there are several effective ways of coping with the sameorigin policy, such as, installing a proxy on the consumerWeb server [5], utilizing dynamic 〈script〉 tags at theclient side, configuring Apache’s mod rewrite propertyat the server side, or modifying the preference options ofclient browser (on the trade-off of data security).

• Network cost. The Service-provider Server broadcastsmetadata of all the athletes to all the spectators, whichwould not scale well with a large number of participants.This can be countered by providing a dynamic profile-matching mechanism for the system so that participantsare provided metadata only of athletes they choose to ridewith.

VI. CONCLUSION

In this paper, we have developed a prototype, My SecondBike, to actively engage television spectators in on-screensports events. My Second Bike promotes a shared riding expe-rience with athletes and outdoor bikers by providing a sharedriding space in 3D mirrored worlds such as Google Earth.By introducing an option for community riding and visuallyrepresenting riders, this system can create a social experiencewhile riding even in your living room. The current phaseof implementation of My Second Bike offers a platform fordevelopment of new features that can improve user experience.• A desirable feature could be superimposing users’s bike

avatars on live TV video streams using Augmented Re-ality (AR) technology. Having visual feedback by fittingone’s avatar directly in the TV video stream may createa deeper sense of participation in the activity.

• This system can be extended to track and field athletics,swimming, car racing etc. For such activities, we can

envision a virtual “track/lane” for the participant byapplying AR technology to draw this track and representthe avatar directly in the environment.

• Riding My Second Bike can be made a more realexperience by transmitting sensory effects to participants,e.g, force feedback, variable resistance to represent uphilland downhill riding conditions, rough and bumpy terrainetc. MPEG RoSE [3] is a potential candidate that couldfit in our system framework to use sensory informationto actuate appropriate physical sensors to enhance userexperience.

• Creating an audio and video channel between participantscould be another interesting feature. Our current systemcan be used for team races such as relay-riding. This formof riding would require participants to communicate witheach other as a rider takes over the previous virtual rider.Audio communication would prove very valuable in thiscase. It would also promote interactivity among partic-ipants. Further, it could give a better sense of distancebetween virtual participants by providing distance-basedvolume control.

ACKNOWLEDGMENT

We thank Dr. Ig-Jae Kim of MIT Media Laboratory for hiscomments throughout the paper. We are also grateful for thesupport received from Samsung Electronics Co., Ltd.

REFERENCES

[1] Direct web remoting. http://directwebremoting.org/dwr/index.html.[2] http://webaugur.com/matt/files/nextel/techover.pdf.[3] http://www-itec.uni-klu.ac.at/˜timse/research/publications/rose.pdf.[4] http://www.activexperts.com/activcomport/howto/html/.[5] Javascript: Use a web proxy for cross-domain xmlhttprequest calls.

http://developer.yahoo.com/javascript/howto-proxy.html.[6] J. Abreu, P. Almeida, and V. Branco. 2beon: interactive television sup-

porting interpersonal communication. In Proc. of the Sixth EurographicsWorkshop on Multimedia 2001, pages 199–208, 2002.

[7] D. C. Brogan, R. A. Metoyer, and J. K. Hodgins. Dynamicallysimulated characters in virtual environments. IEEE Comput. Graph.Appl., 18(5):58–69, 1998.

[8] G. U. Carraro et al. The peloton bicycling simulator. In Proc. of the 3rdSymposium on Virtual Reality Modeling Language, pages 63–70. ACM,1998.

[9] M. Chuah. Reality instant messaging: injecting a dose of reality intoonline chat. In CHI ’03 Extended Abstracts on Human Factors inComputing Systems, pages 926–927. ACM, 2003.

[10] S. Poslad et al. Directing your own live and interactive sports channel.International Workshop on Image Analysis for Multimedia InteractiveServices, 0:275–279, 2009.

[11] C. Quico. Are communication services the killer applications forinteractive tv? In Proc. of European Conference on Internet Television:from Viewers to Actors, 2003.

[12] Z. Wang, S. Poslad, and A. Pearmain. A collective director for highlyinteractive viewing of live sports events. In Adjunct Proc. of EuroITV2009 Networked Television, 2009.

[13] R. C. Waters et al. Diamond park and spline: Social virtual reality with3d animation, spoken interaction, and runtime extendability. Presence,6(4):461–481, 1997.

[14] C. M. Williams and R. Wages. Video conducting the olympic games2008: the itv field trial of the eu-ist project live. In Proc. of the 3rdInternational Conference on Digital Interactive Media in Entertainmentand Arts, pages 436–440. ACM, 2008.