DTV: A Client-Based Interactive DTV Architecture

Edward Y. ChangDepartment of Electrical Computer Engineering

University of California, Santa Barbaraechang@db.stanford.edu

Abstract

In this study we present a client-based architecturethat supports time-shift Digital-TV (DTV) features(i.e., pause, replay and fast-forward) and interac-tive applications. We call this architecture DTVfor Personalizable Interactive Digital-TV architec-ture. Our study focuses on devising effective tech-niques for managing the client’s data (e.g., textual,binary, and video/audio data) under the local re-source constraints. We outline the challenges that

DTV faces and sketch data and resource man-agement policies that maximize the client’s QoSadaptively based on each individual viewer’s qual-ity requirement. We present a prototype that havebeen built and describe an array of applications that

DTV can help realize.

1 Introduction

Many studies have proposed server-based interactive DTVarchitectures (e.g., [1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12]), inwhich a server schedules its resources (e.g., memory anddisk bandwidth) to service interactive VCR-like requests,such as pause, replay and fast-forward. In a server-basedarchitecture, the clients are assumed to be passive—simplyreceiving bits and rendering frames. However, because ofthe typical long end-to-end transmission delay between aserver and a client and the Interet’s limited bandwidth, it ispractically infeasible for the server to support “real-time”

Copyright 1999 by the Association for Computing Machinery, Inc. Permis-sion to make digital or hard copies of part or all of this work for personalor classroom use is granted without fee provided that copies are not madeor distributed for profit or commercial advantage and that copies bear thisnotice and the full citation on the first page. Copyrights for components ofthis work owned by others than ACM must be honored. Abstracting withcredit is permitted. To copy otherwise, to republish, to post on servers, or toredistribute to lists, requires prior specific permission and/or a fee. Requestpermissions from Publications Dept, ACM Inc., fax +1 (212) 869-0481, orpermissions@acm.org.

VCR-like interactive operations for tens of thousands ofsimultaneous users. Furthermore, on a broadcast channel(e.g., CNN), one simply cannot request the server to pauseor to replay a program.

In this study, we present a client-based interactive DTVarchitecture. We call this architecture DTV, for Person-alizable Interactive DTV. Equipped with a large and inex-pensive disk, a DTV client can cache a large amount ofmedia data [4]. This economical caching together with therandom access capability of the disk enables a DTV clientto support time-shift operations. A viewer can pause a liveDTV program to take a break from viewing while the broad-cast stream continues arriving and being written to the localdisk. The viewer can resume watching the program afterthe pause with a delay, or fast-forward the program to getback in synch with the broadcast stream. These time-shiftfunctions allow one, for example, to do one’s own instantreplay during a sports program or to watch a remote lectureat one’s own pace. One can also record programs at multiplechannels at the same time (which a tape-based VCR cannotdo), filter out unwanted content and produce a customizedprogram.

Adding an Internet back-channel in addition to the disk toa DTV client allows TV content to be customized in manyways. While a given broadcast stream remains the same forall viewers, various data objects can be transmitted to indi-vidual viewers via the Internet channel. Data objects can betextual (e.g., an HTML page), binary (e.g., a java appelet), orcontinuous (e.g., a video/audio stream). Image-based ren-dering techniques can overlay a number of data objects withthe broadcast stream to provide a customized program. Forinstance, a viewer can query and change the attributes (e.g.,color, shape, position and history) of these data objects. Abroadcaster can target individual families with tailored ad-vertisement via the back-channel. Many other applicationsexist (more examples are listed in Section 5).

Managing heterogeneous DTV data objects (e.g., video,audio, texts, 3D graphics objects, etc.) at the client sideto support interactive DTV features faces two major chal-lenges: 1) different clients may have different availableresources and 2) different viewers may have different view-ing preferences. A resource manager at the client side mustallocate and schedule available resources adaptively to max-

imize each individual viewer’s quality requirement. In thispaper, we thus propose techniques to manage DTV dataobjects under the constraints of the local resources to maxi-mize the client’s QoS. We start with proposing a quantitativemodel that describes the size and latency characteristics foreach data object. A viewer can assign a QoS factor to adata object to convey that object’s scheduling priority to theresource manager. The resource manager then schedulesthe local resources for the data objects in an order that max-imizes the total QoS. Through quantitative analysis, exper-iments and implementation we show that our client-basedarchitecture can support interactive DTV features effectivelywith very low memory requirement and hence at a low cost.We believe that this client-based approach is a practical ar-chitecture to realize interactive DTV for both point-to-pointand broadcast networks.

The rest of this paper is organized as follows: Section 2depicts a typical DTV pipeline. In Section 3 we character-ize DTV data objects. Section 4 describes how resourcesare scheduled to maximize QoS under constraints. Section 5presents the digital-VCR prototype that we have bulit. Fi-nally, we offer our conclusion in Section 6.

2 System Overview

Internet

Data

Obje

cts

. . .

Broadcast

Receiv

er

Memory

CP

U Dis

pla

y

D T V B o x

Figure 1: The DTV Pipeline

Figure 1 depicts a DTV pipeline. On the left-hand side ofthe figure, DTV data objects are transmitted via broadcastand point-to-point channels to a DTV box. The DTVbox receives network packets in its main memory. If thedata are not needed shortly (e.g., the viewer paused theplayback), the data are written to the box’s local disk toconserve memory. The data are made available to the CPUbefore they are needed. The CPU processes (e.g., computes,decodes, renders, etc.) the data in main memory, and finallythe processed data are played back to the viewer on theright-hand side of the figure.

A DTV pipeline consists of two major components:DTV data objects and system resources, including CPU,

memory, and a local disk. (We assume that a DTV boxhas only one local disk for economical reasons.) In thefollowing sections we characterize DTV data objects andsystem resources in more detail.

Delay Sensitivity

Low High

Dat

a O

bjec

t Si

ze

S

mal

l

L

arge

Images

Graphic Objects

HTML pages

Audios

Videos

JAVA Applets

Captions

Figure 2: The Characteristics of DTV Data.

3 Characterizing Data Objects

Data objects can be continuous, textual, binary, and so forth.They are best characterized by their sizes and latency con-straints. Continuous data are audio/video streams, whichcan be voluminous (video streams) and delay sensitive. Tex-tual data include captions, HTML and XML documents, etc.Textual data in general occupy far less space than contin-uous data and are not delay sensitive. Other data, such asimages and applets, can be very copious or scant and mayor may not be delay sensitive, depending on the nature ofthe application. Figure 2 shows the characteristics of thesedata objects. The -axis in the figure represents the delaysensitivity from low (on the left-hand side) to high (on theright-hand side); the -axis gives the size of an object. Atone extreme, videos are large and very sensitive to delay.At the other extreme, captions are small in size and not verysensitive to delay.

The size of a data object is measured by the amountof storage space it needs and is denoted by , wherestands for the object. (We show how is computedfor a continuous data object in [4].) To quantify the delaysensitivity of a data object, one can specify a value functionfor it. The function can be defined in many ways: Figure 3shows four representatives. The -axis in the figure depictsdelay; the -axis depicts the value of the object normalizedto from zero (worthless) to one. Function depicts anobject that can tolerate delay up to a period of time, ,then becomes worthless. Function shows the value ofan object that decays linearly after . Functions and

have other value decay patterns. Function is a goodvalue function for an audio frame. Function may be agood value function for a video frame since slight delayof a frame display may be tolerable. Function may bea good value function for a subtitle. Function , whichshows that a user’s patience decays exponentially after ,may be a good value function for a Web page. Withoutlosing generality, we define the value function for object

as , where denotes the span between the time theobject is requested and the time the object is available inmain memory for processing.

In short, we characteristize data objects as follows:

V a

l u

eV

a l

u e

Fc

FbFa

Fd

0

1

1

0

t0 0 t

T i m e

t t

Figure 3: Delay Function Examples.

represents the object’s storage requirement, anddepicts the object’s delay sensitivity.

4 Scheduling Resources

The local resources include CPU, memory, disk bandwidthand network bandwidth. The local resources may not beadequate to support a particular interactive scenario. There-fore, it is critical for the resource manager to be adaptiveto the resource constraints and to degrade, if degradation isunavoidable, in a graceful manner. Given limited resources,the design objective of a DTV box is to prioritize resourceallocation in order to maximize the user’s satisfaction.

To measure a user’s satisfaction, we must allow the userto have a say about what is important and what is not. Wethus associate each data object with a QoS parameter. Aviewer can assign a QoS value to each data object, eitherimplicitly or explicitly. For instance, if a user decides toturn off all interactive features, the resource manager canassign a QoS factor of zero to data objects that are neededto support interactive features. Regarding the data objectsthat are needed to support the playback, higher QoS can beassigned to mission-critical data objects, such as broadcastvideo and audio streams, and lower QoS can be assigned tooptional data objects such as captions and applets.

Given requested data objects and available mem-ory, the goal of the resource manager is to schedule dataobjects ( ) to maximize the total QoS under theresource constraints. We use to denote the QoS require-ment assigned to the data object ( ) andto denote the latency needed to stage the data object inmain memory. (The value of depends on the local diskbandwidth and network bandwidth, and the size of the dataobject. We show how to determine in the full version ofthis paper.) Let represent the scheduling order for dataobjects. We would like to schedule the service to thesedata objects in the order that maximizes the sum of the QoS.The objective function can be written as:

(1)

Figure 4: A Digital-VCR Screen Shot.which is subject to the memory constraint

We show how the optimal schedule can be solved efficientlyin the full version of this paper.

5 A Digital-VCR Prototype

To demonstrate that the DTV architecture is practical, aDigital-VCR has been built. The prototype is implementedon a Pentium II MHz Windows workstation witha Quantum Viking II disk and MBytes DRAM. Themain memory and disk are linked with a SCSI fast bus andmanaged by an Adaptec AIC- based PCI Ultra2 SCSIcontroller, both of whom have much higher data transferrates than the disk.

TheDigital VCR supports VCR-like interactive functionsincluding pause, fast-forward and instant replay. Figure 4shows a screen shot of the user interface (i.e., codec) anddisplay. At the bottom of the figure is the codec throughwhich a viewer issues interactive commands and monitorsthe system. To issue a command, a viewer selects andpresses one of the control buttons on the upper-left cornerof the codec (described shortly). To monitor the system, thecodec uses trackbars to display various statuses. The fivetrackbars from the top to the bottom on the codec keep trackof the amount of memory use, the amount of disk use, theaccumulated number of hiccups, the playback progress, andthe broadcast progress.

When a viewer watches a TV program live, memory isused to buffer the mismatch between the data transmissionrate and the playback rate. The memory trackbar keepstrack of the current memory use. The disk use is zero whena program is watched live. When the pause button is pressed,the excessive data that the DRAM cannot stage is writtenonto the disk. Using the disk as a part of the cache, the VCRcan allow the viewer to pause for a substantial amount of

time (a GBytes disk can buffer more than one hour ofMbps HDTV program). After returning from the break,

the viewer can press the play button to resume the playback.At this time, the decoder consumes data from the disk whilethe broadcast data continue arriving and being appended tothe tail of the disk file. The viewer has the option to performfast-forward to catch up with the live program. When thefast-forward button is pressed, the Digital VCR retrievesdata from the disk at a faster pace and the decoder decodesand displays key frames only. The data cached on the diskare eventually used up and the playback returns to the livemode.

The viewer can replay from the start of a recorded videosequence by pressing the reply button. We envision that tosupport user-friendly replay, the Digital VCR should auto-matically generate thumbnails to help the viewer select a de-sired video sequence (e.g., a sequence showing McGwire’srecord breaking homerun). Implementing video indexingand enhancing the replay function are in the project plan.

Potential Applications

The Digital-VCR prototype supports VCR-like featureswithout the need to interfere with the server. This is be-cause the large local disk provides an inexpensive bufferto cache a large amount of data to support interactive fea-tures. The followings are some applications that can alreadytake advantage of this prototype (without an Internet back-channel):

Self-paced distance learning: One can view a lecture atone’s own pace.Personalized instant replay: One can replay a video or anaudio segment of a live program.Personalized news program: One can record news play-ing on multiple channels at the same time and composea customized program of ones desire (e.g., skipping allfinancial news).

As discussed in Section 1, with image-rendering tech-niques, DTV can allow the viewer to influence TV contentat the frame level. For example, a virtual museum programmay deliver data at very high rates but the client decodes,stitches, and displays only the frames that the user views.Children would be able to change the attributes (e.g., col-ors, size, position and even the personality) of the “bunny,”“big bird” and other characters in the Sesame Street pro-gram. A grade school student can learn American states byinteracting with a map. Adding an Internet back-channelin addition to a large disk to the DTV box makes manymore applications possible: e.g., customized news, soaps,ads, and enhanced home-shopping channels.

6 Conclusion

We have presented a client-based DTV architecture thatsupports personalizable interactive DTV features. The

DTV architecture adds a large disk and a back-channel to

the broadcast DTV channels at the client side and managesdata objects under local resource constraints to maximizeeach client’s quality requirement. The Digital-VCR proto-type shows that DTV can support interactive operationseffectively. Its data management and resource schedulingcomponents are being enhanced currently to support someadvanced applications mentioned in the paper. We believethat DTV is a practical approach that can turn the passiveTV viewing experience into a personalizable and interactiveone.

7 Acknowledgements

I would like to thank professors Hector Garcia-Molina andPatrick M. Hanrahan at Stanford University for their helpfulcomments and suggestions. I would also like to thank SergeRutman and Milton Chen at Intel for their hardware/softwaresupport in building the digital-VCR prototype.

References

[1] E. Chang and H. Garcia-Molina. Bubbleup - Low latencyfast-scan for media servers. Proceedings of the 5th ACMMultimedia Conference, pages 87–98, November 1997.

[2] E. Chang and H. Garcia-Molina. Effective memory use ina media server. Proceedings of the 23rd VLDB Conference,pages 496–505, August 1997.

[3] E. Chang and H. Garcia-Molina. Reducing initial latencyin media servers. IEEE Multimedia, 4(3):50–61, July-September 1997.

[4] E. Chang and H. Garcia-Molina. Medic: Memory and diskcache for media servers. Proceedings of IEEE InternationalConference on Multimedia Computing and Systems, pages493–499, June 1999.

[5] E. Chang, H. Garcia-Molina, and C. Li. 2d BubbleUp -Managing parallel disks for media servers. Proceedings ofthe 5th Foundations on Data Organization, pages 221–230,November 1998.

[6] M.-S. Chen and P. Yu. Storage and retrieval methods tosupport fully interactive playout in a disk-array-based videoserver. ACM Multimedia Systsmes, 3(3):126–35, July 1995.

[7] S. Ghandeharizadeh. Continuous retrieval of multimediadata using parallelism. IEEE Transactions on Knowledgeand Data Engineering, 5(4):658–69, 1993.

[8] T. Johnson and A. Zhang. A framework for support-ing quality-based presentation of continuous multimediastreams. Proceedings of the 4th IEEE Conference on Multi-media Computing and Systems, pages 169–176, June 1996.

[9] R. Ng and J. Yang. Maximizing buffer and disk utiliza-tions for news on-demand. Proceedings of the 20th VLDBConference, pages 451–462, 1994.

[10] B. Ozden, A. Biliris, R. Rastogi, and A. Silberschatz. A low-cost storage server for movie on demand databases. Proc.VLDB, September 1994.

[11] B. Ozden, R. Rastogi, and A. Silberschatz. A frameworkfor the storage and retrieval of continuous media data. Proc.IEEE Multimedia, pages 2–13, May 1995.

[12] Y. Wang, J. Liu, D. Du, and J. Hsieh. Efficient video file allo-cation schemes for vod services. ACM Multimedia Systems,5(4), September 1997.