Broadcast Technology No.51, Winter 2013 ● C NHK STRL 9

Hybridcast is a technology platform for implementing services that use both broadcasting and telecommunica-tions delivery channels. It combines the merits of broad-casting and telecommunications to realize a variety of services. In this article, we introduce the specialized tech-nologies developed for Hybridcast.

1. Introduction Hybridcast®*1 is a technology platform for implement-

ing sophisticated services linking broadcasting and tele-communications. It combines the merits of broadcasting and telecommunications to offer services such as data broadcasting and video on demand (VOD) that are be-yond the capabilities of current broadcasting. For exam-ple, a service enabled by Hybricast could be to receive content through a telecommunications network and display it on the TV screen during a broadcast program to make it more interesting or to provide additional in-formation tailored to the individual interests of viewers. In this article, we introduce technologies that were devel-oped to provide these new functionalities to Hybridcast.

2. New Technologies of Hybridcast2.1 Services made possible through Hybridcast

The various services of Hybridcast are implemented with application software that is executed on the receiver. Currently, we assume that each time a service is used, the application will be downloaded through telecommunications networks. Below, we describe the process of implementing such a service. First, the receiver receives application control information*2 through broadcast or telecommunications delivery channels. This information includes commands for launching and terminating an application and information about the servers that distribute it. On the basis of this information, the receiver downloads the application from the specified servers. Once the downloaded application launches, it can communicate with various servers on the network. Through the communication capability with servers, the processing required for the service can be performed not only by the application on the receiver, but also high-performance servers. The processing load distribution allows Hybridcast to be flexible in what kinds of services it supports. A mechanism for linking server communications functions with broadcast

programs is also provided so that information obtained through telecommunications delivery channels can be synchronized with broadcast programs. This mechanism can be used to implement services that integrate broadcast and telecommunications functions more tightly.

2.2 New Technologies in Hybridcast and their RolesTo implement Hybridcast, four new technologies were

developed. They are application control technology, application management technology, synchronized presentation technology, and device link technology.

Application control technology controls the launching and terminating of applications. Hybridcast provides a variety of methods for launching and terminating applications to support various new services.

Application management technology enables both broadcast programs and applications to coexist on the receiver, while maintaining convenience for viewers. Hybridcast will support services offered by third-party service providers who are not broadcasters, so the application behavior must be managed in order to protect interests of both the broadcasters and the viewers.

Synchronized presentation technology synchronizes streams sent over multiple sources spreading over broadcasting and telecommunications. It makes possible services that tightly integrate content obtained through both broadcasting and telecommunications delivery channels.

Device link technology enables communications between the receiver and other devices such as tablets. This technology enables various user interfaces and ways of presenting information. Applications use the device link and synchronized-presentation technologies by calling Application Programming Interfaces (APIs) in the receiver.

In the next chapter, we will discuss these four new technologies in detail.

3. Application Types and Control TechnologiesHybridcast applications can be classified into program

related applications, which are associated with a broadcast program, and independent applications, which are not.

Program related applications are associated with a broadcast channel*3 and are an effective way to provide services that make broadcast programs richer in content

*1 Hybridcast is a registered trademark of NHK-ES, Inc. *2 Information for notifying the receiver whether there is an asso-

ciated application and to control launching and terminating the application.

Technologies Supporting Hybridcast®

*3 A three-digit logical channel number. For example, the NHK General channel is 011, and the NHK Educational channel is 021.

Broadcast Technology No.51, Winter 2013 ● C NHK STRL10

and more enjoyable to watch. Figure 1 shows the timing for launching and terminating an application that is related to the broadcast channel being watched, and an independent application. A program related application runs only while the broadcast channel for the application is selected, and it is terminated if the channel is changed.

An independent application is not associated with a broadcast channel and can be used to provide services

that are not related to a particular program or broadcast channel. As shown in Figure 1, the application continues to operate regardless of whether the broadcast channel changes. Generally, launching, terminating and other controls of independent applications are performed by the user.

Figure 2 shows examples of launch sequences for Hybridcast applications. In one example, the Top Menu

Start of Program 1End of Program 1Start of Program 2

Broadcast channel 1Program 1

Application 1

Application 3

Independent application not related to broadcast channel or program

Application 2

Application related to program 1 Application related to program 2

Broadcast channel 1Program 2

Broadcast channel 2Program 1

Selected channel

Independent application

Program related applications

Change of channel

Signal to launch application 1 from the broadcast

Launched whenever the user desires

Stopped whenever the user desires

Execution continues even when the program ends

Execution continues even if the channel is changed

Signal to launch application 2 from the broadcast

Signal to stop application 1 from the broadcast

Stop of application 2 due to change of channel

Figure 1: Relationship between broadcast and applications

Power on

“Menu”

Data broadcast

Program related application

Independent application

Digital broadcast

Application Menu

Automatic launch or select “launch program related app”→Launch of application specified in the broadcast signal

Program related applications automatically stopped by operations such as channel change

Independent applications stopped manually by user

Various program related applications

Launch other applications Launch other

applications

"Top Menu" Application

Select application from menu

"Application Menu" screen- List launchable program related applications and independent applications- Select and launch applications using remote- Implemented as a receiver function

Transition from a data broadcast

Figure 2: Example sequence for launching a Hybridcast application

Broadcast Technology No.51, Winter 2013 ● C NHK STRL

Feature

11

Application, is launched by a launch operation or a transition due to a data transmission. This application is a linked one that launches other linked applications. The linked applications that are launched through it are automatically terminated by operations such as changing channels.

In the other example, the user launches and terminates an unlinked application by bringing up an application menu and selecting it. The control of each type of application is described below.

3.1 Control of Program related ApplicationsA program related application is launched and

terminated automatically in accordance with current broadcast program or broadcast channel. To control the application in this way, information on the application including the application name, the application ID, control commands to launch and terminate, and the location (URL, etc.) of the server distributing the application is needed. The broadcaster distributes this application control information, including the above components associated with the broadcast channel.

3.2 Control of Independent ApplicationsThe receiver retrieves information for the independent

application list from the servers that distribute the independent applications. This list information includes the application control information, or the location of the control information (URL, etc.). The user can display the distributed list on the TV screen, and run the desired application from the list. Independent applications can be launched or terminated whenever the user desires.

3.3 Other Ways to Launch ApplicationsOther ways of launching Hybridcast applications

include launching from a data broadcast and launching from another application that is already running. Enabling applications to be launched in a variety of ways allows seamless transitions across different types of broadcasting and telecommunications services.

4. Application Management TechnologyHybridcast enables third-party service providers other

than broadcasters to provide services. This carries a problem wherein it may be difficult for broadcasters to know the behavior of applications of other providers. To protect interests of broadcasters as well as those of viewers, Hybridcast includes schemes to authenticate applications, to control the display on the screen, and to control the access to the receiver APIs.

4.1 Authenticating ApplicationsThe application authentication scheme for Hybridcast

authenticates applications running on the receiver. It verifies that applications are legitimate and protect interest of both the content and the viewers. Figure 3 is an overview of this scheme. Broadcasters distribute private keys to application developers, and these keys are used to generate signatures that certify the legitimacy of applications. The public keys needed to verify these signatures are stored in the Hybridcast receivers prior to the time of authentication. The developer attaches an application ID to each application, generates a signature using the private key, and registers the application, with the ID and signature, on the server that will distribute it.

Application developer

ID=10011001

ID=0010101

Application distribution

server

Hybridcast receiverPrivate key

Public key

ID=00101011Authentication by signature and/or ID

Generation of signature

Broadcaster

Application registration

Application registration

Acquisition of application

Acquisition of CRL (broadcast or telecommunications)

Figure 3: Application authentication scheme

Broadcast Technology No.51, Winter 2013 ● C NHK STRL12

Broadcasters register applications they developed on a distribution server in the same way. Hybridcast receivers acquire an application from a distribution server and then consult to the Certification Revocation List (CRL)*4

and verify the signature and ID by using the public key. The broadcaster distributes the CRL beforehand through either broadcast or telecommunications delivery channels. Only applications that pass the verification are permitted to execute. One possible method for the application authentication scheme1) is to use Key-Insulated signatures*5 of which has key management is easy and safe.

4.2 Controlling Presentation on the ScreenServices provided by third-party service providers may

be overlaid on the broadcast screen, which potentially causes to hide information in the broadcast that needs to be conveyed reliably to viewers, such as emergency bulletins. To reliably convey information in a situation such as a disaster while also catering to the needs of viewers, we developed a screen presentation control scheme2) that enables the broadcaster to transmit its intentions to the receiver and give priority to display important information such as emergency bulletins. Figure 4 is an overview of the screen display control scheme. The control scheme developed describes permissions by the broadcaster, such as allowing applications to partition the screen or overlaying. The broadcaster sends the permissions for each broadcast channel or individual program within the broadcast

signal, and the receiver controls presentation of broadcast program and applications on the basis of the permissions. In addition, emergency warning signals such as emergency warnings and earthquake early warning signal are considered that they contains the pre-defined permissions for presentation to be displayed reliably and quickly. This scheme was implemented in prototype Hybridcast receivers, and operational tests confirmed that the application presentation methods control reliably according to the content of the broadcast.

4.3 Controlling Access to Receiver APIsTo provide services that are tightly linked to broadcasts,

applications use the APIs built into the receiver in order to access Program Specific Information and Service Information (PSI/SI)*6 that are multiplexed with broadcast signal and to access channel selection information. However, allowing applications for unconditional access to all of these APIs raises concerns about security risks such as disclosure of someone’s viewing history. To mitigate the risks, a scheme to control access to APIs according to application type and viewer intentions was developed3) to protect viewer privacy and broadcast content rights on Hybridcast receivers. In this scheme, an access control list based on the authentication result and ID of each application is managed on the Hybridcast receiver, and each time the application calls an API, the access control list is checked to determine whether the API can be used. Applications that are not verified as legitimate are not permitted to execute the API. Also, even if an application is permitted to access the API, the viewer can permit or deny access to APIs for reasons such as protecting privacy.

Earthquake Early Warning

Broadcaster

Application is turned invisible in response to something like an earthquake early warning signal

Normally, application and broadcast program are displayed at the same time

Prototype Hybridcast receiver

Application

緊急地震速報

Figure 4: Screen presentation control scheme

*4 A list of disabled applications[npk3].*5 A signing scheme that enables multiple signatures generated

with different private keys to be verified using a single public key.

*6 For example, information such as the electronic program guide (EPG).

Broadcast Technology No.51, Winter 2013 ● C NHK STRL

Feature

13

5. Technology for Synchronized Presentation of Broadcast and Telecommunications Content

Hybridcast makes it possible to synchronize information obtained through telecommunications delivery channels with a broadcast program and present them on the same screen or view them on different terminals. Multi-view services, multi-language audio or closed-captioning services are examples of such services. In particular a multi-view service could be one that lets the viewer choose between different angles of sports coverage while synchronizing and composing that image on the screen with the broadcast video.

In such a service, the timing of presentation of both the broadcast content (program audio and video) and the telecommunications content (video, audio, closed captions, meta-data, etc.) shall be accurately synchronized. Compared with telecommunications delivery channels, latency of digital broadcast delivery channel is small and constant. We developed a scheme for the receiver to synchronize multiple pieces of content sent via multiple transmission paths with different transmission characteristics. Figure 5 illustrates the structure of this transmission and reception system. As the broadcast is being transmitted, the network server streams the additional content, together with an associated Presentation Time Stamp (PTS) that is based on the Program Clock Reference (PCR) of the broadcast stream. The receiver provides a buffer of adequate size to make presentation of broadcast content wait to absorb the delays of the telecommunications content and synchronizes them by comparing their timestamps.

Tests of the prototype equipment confirm that broadcast video could be synchronized with closed captions and video obtained through telecommunications delivery

channels to frame level accuracy (33ms)4)5). The tests also show that addition of synchronization control software to the existing receiver hardware with multiple decoders can achieve this accuracy6).

In Hybridcast receiver model, functionality for synchronized presentation is embedded in the middleware layer*7 of the receiver, and applications call it through the APIs. The main functions of the synchronization APIs are as follows.

• Synchronize the playback of multiple streams by comparing timestamps

• Retrieve the timestamp value of a stream• Delay the presentation of a broadcast stream for

specified period

6. Device Link TechnologyThe device link technology allows collaborative

presentation and interaction by the receiver and other devices such as tablets and smartphones. It enables information to be presented in a variety of ways with user-friendly interfaces. The link is established between applications running on a tablet or other terminal and the Hybridcast receiver or application running on the receiver. As shown in Figure 6, devices are connected through a home network. (LAN) Communication across the devices is achieved by device link APIs provided in the receiver, which allows receiver functions and applications to involve functionalities of other entities. The device link APIs are categorized based on the following three

Broadcast

Audio

Application (control commands)

Presentation composition

Receiver

RequestBroadcaster

Tuner/Decoder

Synchronization buffer (for delay)

Network interface/decoder

User profile

Internet

Various content Synchronization bufferNetwork server

System clock value

with presentation timestamps based on the broadcast clock

Video Graphics Texts Screen rendering/Video/Graphics/Text

Re-s

ynch

roni

zatio

n

Shared system clock

Figure 5: Structure of transmission and reception systems enabling synchronization of broadcast and telecommunications signals

*7 Software within the receiver implementing most of the receiver functions.

Broadcast Technology No.51, Winter 2013 ● C NHK STRL14

purposes; • Provision of information by receiver functions for

other devicesThis category is intended to provide information

mostly handled by receiver functions such as Service

Information (SI), program-related information, information embedded in event messages*8, and the playback position (timestamp).

Device link API

(3) VOD playback

(1) Keywords retrieval (2) Search

Keyword search Playback of a VOD content related to a scene

Server on the Internet

Hybridcast receiver

(1) Retrieval of playing time position information

(3) Play related VOD content (2) Scene related Web page

Figure 7: Example of a service using the terminal-linking technology

Tablet

Receiver functions

Device link API

Hybridcast receiver

Application

TV

Home network (LAN)

Servers

Smartphone

Collaboration

Device link functions

Collaboration with an application on the mobile terminal

STB

・Device ⇒ Receiver- VOD playback control, channel

changing - Intuitive user operations・Receiver ⇒ Device- Information in the broadcast signal

(program outline, scene information)- Timing information for the program

being watched

Information retrieval from the server in accordance with progress of broadcast program

Figure 6: Terminal linking technology

*8 A trigger signal sent by the broadcaster using the data broad-cast mechanism.

Broadcast Technology No.51, Winter 2013 ● C NHK STRL

Feature

• Controlling the Hybridcast receiver from other devices

This category is intended to control Hybridcast receiver functions by giving control information (changing channels, VOD playback, etc.) from other devices corresponding to user operations.

• Communicating between applications on the Hybridcast receiver and the devices

This category is intended for high level communication between applications on multiple devices including Hybridcastw receiver. Bi-directional communication can be used for cross-authentication and relaying user input (keyboard input, etc.).

The device link APIs make it possible to offer various services by combination of functions and information spreading out the Hybridcast receiver and other devices. For example, Figure 7 shows a service in which a tablet receives keywords related to the current scene in a program on the TV and uses it to retrieve Web pages or other information related to the program from servers.

7. ConclusionWe have introduced the technologies that support

Hybridcast. Hybridcast makes it possible to provide services that are beyond conventional broadcasting by combination of the content from multiple sources in sophisticated ways and acceptance of thrird-party service providers. To achieve this, Hybridcast needs technology different from those of conventional broadcasting. These new technologies, of course, should be able to be implemented in commercial receivers easily. Our goal with Hybridcast is to realize services consisting of deeply

combined broadcasting and telecommunications content and functions in the near future, and we will continue work to optimize and standardize the technologies.

(Masaru Takechi)

References1) G. Ohtake and K. Ogawa: “Application Authentication for Hybrid Services of Broadcasting and Communications Networks,” Proc. of WISA2011, LNCS 7115, pp. 171-186 (2011)

2) Otsuki, Ohmata, Fujii, Majima, Inoue: “Application Presentation Control Method for Services Linking Broadcasting and Communications,” ITE Annual Conference, 11-11 (2011) (Japanese)[npk2]

3) Ohmata, Otsuki, Majima, Matsumura, Takechi: “An Access Control Method for Web Applications on Hybridcast,” IEICE General Conference, B-7-34 (2012) (Japanese)

4) K. Matsumura, M. J. Evans, Y. Shishikui, A. McParland: “Personalization of Broadcast Programs using Synchronized Internet Content,” International Conference on Consumer Electronics 2010 (ICCE), 4-1.5 (2010)

5) Matsumura, Kanatsugu, Hamada: “Prototype Program-Extension Services by Synchronizing Broadcasts and IP Streaming,” ITE Annual Conference, 10-5 (2010) (Japanese)

6) Baba, Matsumura, Mitsuya, Takechi, Kanatsugu, Hamada: “A Receiver able to Synchronize and Present Broadcast and IP Content,” ITE Winter Conference, 10-1 (2010) (Japanese)

15