Jongchurl Park(jcpark@nm.gist.ac.kr)

Networked Media LaboratoryDept. of Information and Communications

School of Information & MechatronicsGwangju Institute of Science & Technology (GIST)

System Enhancements and Extension of Uncompressed HD Media Transport System

Lab. SeminarMay 30th, 2009

School OF INFO. & Mecha., GIST

Introduction◦ Uncompressed HD media transport system◦ Networked tiled display & 3DTV

Enhanced uncompressed HD media transport sys-tem◦ Preview video & bi-directional transport system◦ Networked display interface

Extension of uncompressed HD media transport system◦ Multiple streams for free-viewpoint◦ Views in 3D effects

Conclusion & Future works

Contents

School OF INFO. & Mecha., GIST

Introduction

School OF INFO. & Mecha., GIST

No need encoding and decoding proce-dures◦ Highly interactive (i.e., low-delay) media services

Video format◦ 720p: progressive, 60 fps, 1280x720, 20 bits/sample◦ 1080i: interlaced, 30 fps, 1920x1080, 20 bits/sample

Requied bandwidth: 1.2~1.5 Gbps Delay: ~200msec. Interactive application

◦ Video conference◦ Remote lecture (i.e., e-learning)◦ Interactive remote performance (e.g., DancingQ)

Introduction (1/3)-Uncompressed HD media transport system

School OF INFO. & Mecha., GIST

Standard Definition Television (SDTV)◦ Resolution - 480i: 720 x 480, 60 interlaced fields per second

◦ Uncompressed Bandwidth: ~200Mbps Delay: ~100msec.

◦ MPEG2 Bandwidth: 2~6Mbps Delay: 1~2 sec.

High Definition Television (HDTV)◦ Resolution

1080i: 1920 x 1080, 60 interlaced fields per sec. 720p: 1280 x 720, 60 frames per sec. (progressive scan)

◦ Uncompressed Bandwidth: 1.2 ~ 1.5 Gbps Delay: ~100msec.

◦ MPEG2 Bandwidth: 20~80Mbps Delay: 1~2 sec.

Limitation◦ Uncompressed HD video transport system on raw is not enough for display-

ing correct HD videos◦ More realistic presence is required like three dimension

Introduction (2/3)-SD vs. HD

School OF INFO. & Mecha., GIST

Networked tiled display◦ Features

Monitors of Lattice shape , users can use as if tiled display is single monitor Cost-effective ultra-high resolution device (more than 4K) Simultaneously run various applications on local or remote clusters, and share

them by streaming the pixels of each application Users freely move and resize each application's imagery in run-time Color-space

Compatible with RGB16, RGB24 and RGB32

◦ Applications Scalable Adaptive Graphics Environment (SAGE)

The role of display middleware to visualize any kind of pixel-stream to a tiled display color-space

SMeet One Display (SMOD) Realized after SAGE, Integrating collaboration services that include media delivery scheme to offer A/V communication among participants, high-resolution display

service

3DTV◦ “Realistic” ghost-like illusions has always been a goal◦ A new generation of broadcast

Immersive 3D video display in real-time

Introduction (3/3)-Networked Tiled Display & 3DTV

School OF INFO. & Mecha., GIST

Enhanced Uncompressed HD Me-dia Transport System

(Preview Video Support &Bi-directional Transport System)

School OF INFO. & Mecha., GIST

System Overview

The criterion of uncompressed HD media transport system◦ Hardware capture and software display

Specification◦ Ultragrid (USC/ISI) extension◦ XENA HS (AJA Video systems) – Rx & Tx ◦ 1080i (25,50,29.97,59.94,30,60 fps)◦ 6-channel audio (24-bit/48KHz)◦ Video out

XENA HS, SDL, xVideo◦ Audio out

XENA HS, ALSA◦ Platform

Redhat 9, Mandrake 10.1, and Fedora Core 5

6-Channel Audio

Sony HVR-Z1N

5.1 Channel Speaker

Preview

HD Display

1G

1G

High Speed Network

HW Capture

SW Display

Uncompressed HDAV Transport System

School OF INFO. & Mecha., GIST

Preview◦ users can preview and see the current stage of the process before producing into a final form. ◦ lets users to visualize current/final product and correct possible errors easily before finalizing the

product. ◦ Interactive video conference and online forums allow users to preview their place environment

before transport.

Particularly useful on sites with complex markup, where it serves as an op-portunity to identify and correct video quality, light and so on.

Preview Video

A/D Converter

Xena HS Capture

cardGigabit Ethernet

Uncompressed HD

A/D ConverterXena HS Capture

card

Uncompressed HD

A/D Converter

Receiver/Sender ApplicationSender/Receiver Application

Send

Receive

NIC

Speaker

A/D Converter

Uncompressed Audio

Speaker

NIC

Uncompressed Audio

School OF INFO. & Mecha., GIST

1G NIC

Memory

CPU

CPU

HDD (RAID)

Xena card[Capture]

DMA

1G NIC

Memory

HDD (RAID)

TransferModule

[re-organized]

Xena card[Display]

Xena card[Capture]

Xena card[Display]

DMA

CPU

CPUTransferModule

[re-organized]

Node #1 Node #2

Uncompressed HD Media Transport System

HW-based Bi-directional Transport

Alternating system configuration◦ Supporting multiple network interfaces

Verified for bi-directional streams

◦ Multiple HD-SDI interfaces support Separating functions to capture and display Simplifying system handling with bi-directional transport

Hardware performance analysis◦ Requires both HD-SDIs and Network Interface Cards support 1.3Gbps bandwidth

To exploit hardware overload from bulky traffic

◦ Hardware bandwidth requires more than PCI-X 100Mhz extended slot 133Mhz extended slot can support maximum 1 GByte/s

School OF INFO. & Mecha., GIST

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

outgoing video stream (single load)

imcoming video stream (single load)

outgoing video stream (dual loads)

incoming video stream (dual loads)

Frame per second

Occ

urr

ance

rate

Occu

rren

ce

rate

Frame per second

Comparison of frame rate◦ Outgoing/incoming video stream (single load)

61.5% of frame rate maintain at 29.97 fps

◦ Outgoing/incoming video stream (dual load) 99.3% of frame rate maintain at 29.97 fps

Dual load paths improved and stabilized performance than single load network

Experiment Result

School OF INFO. & Mecha., GIST

Enhanced Uncompressed HD Media Trans-port System

- Network Display Interface -

School OF INFO. & Mecha., GIST

System Overview

Interfacing uncompressed HD media transport system and networked tiled display◦ A scalable hundreds of megapixels of contiguous display with cost-

effective prices as possible

School OF INFO. & Mecha., GIST

Networked Tiled Display

Uncompressed HD Video Transport

System

Gigabit Ethernetc

Uncompressed HD video streams

SAIL library &(SAGE)

STP library(SMOD)

Display Modules

Display node 1

Display node 2

Display node 3

Display node 4

Interfacing Module(configuration)

Color Space Conversion Module (YUV422 to RGB24)

SAIL library &STP library

.

.

System Architecture

Video source: Uncompressed HD◦ Captured 1920x1080i format HD

signal by using HD-SDI card◦ Covert analog signal to digital sig-

nal, due to SMPTE-292M by using A/D converter

Interfacing Module◦ Put video data to buffer◦ Color-space conversion

Transport◦ Put frames to buffer and receive

frames by using SAIL or STP library

Display modules◦ Display video on all display nodes

School OF INFO. & Mecha., GIST

Performance enhancement for networked display interface◦ Using binary increase congestion control (BIC) is not good for heavy traf-

fic Long delay over high-speed networks Improving TCP-friendliness and RTT-fairness

◦ Using extended version of BIC (CUBIC) – good for heavy traffic (around 1Gbps bandwidth usage) Increasing TCP transport window size for no acknowledge (NACK) Allocating large enough socket buffers Reconfiguring TCP segment sizes

Maximum MTU packet by using a path MTU discovery function

◦ System tuning (After receiving 1000 packets, it cause freezing HD dis-play) All nodes as well as a controller should be set around MTU 9000 byte Increasing buffer size can solve freezing problem Hard sync stream type makes it more stable

System tuning & Configuration

School OF INFO. & Mecha., GIST

Interfacing without internal network

Cost-effective interfacing system◦ Interfacing system is required to set up both sender and re-

ceiver of uncompressed HD media transport system Directly transmit frames sender to networked tiled display (without re-

ceiver) Using sail library instead of RTP in uncompressed HD media transport sys-

tem

School OF INFO. & Mecha., GIST

Comparison of Bandwidth (BIC and CUBIC)◦ BIC

• Input: 921Mbps (29.97 fps)• Output: 239.4Mbps (7 fps)

◦ CUBIC• Input: 921Mbps (29.97 fps)• Output: 682.1Mbps (23 fps)

CUBIC improved and stabilized performance than BIC• Approximately 16fps difference

Experiment Result

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 210

100

200

300

400

500

600

700

800

BICCUBIC

Time (second)

Bandw

idth

(bps)

Time (sec.)

Ban

dw

idth

(M

bp

s)

School OF INFO. & Mecha., GIST

Environment◦ 8bit uncompressed HD video is

transmitted over 1Gbps (MTU: 9180 Bytes)

Top: Artistic dance performance is deliv-ered in live to international destinations by HW-based playout mode

Middle: Interactive video conferencing scenario with preview HD videos by SW-based playout

Bottom: Interfacing with networked tiled display

Ultra-high-resolution support Appropriate control of frame rate

Experimental Snapshots

Hardware-based Playout for Bi-directional Transport

Receiver View

Sender ViewSender View

Receiver View

Node 1 Node 2

Software-based Playout

Network-based Tiled Display Playout

School OF INFO. & Mecha., GIST

Extension of Uncompressed HD Media Transport System

School OF INFO. & Mecha., GIST

◦ Interactive clustered broadcasting with free-viewpoint display Multiple streams for free-viewpoint Views in 3D effects (Interactive views) Ultra-high-resolution support

Overall Architecture

School OF INFO. & Mecha., GIST

Multiple video streams in single machine◦ Required to handle more TWO uncompressed HD video data (over 2Gbps)

Simplifying system handling multiple ways for transport

◦ Improved configuration of network interface card and HD-SDI interface Up to 4 media interfaces

◦ Verified for bi-directional streams and two ways streams

Free-viewpoint video◦ A multi-view display emits more than two views, a viewer can be positioned anywhere◦ One category of 3D video - the viewer shall be given the flexibility to interactively po-

sition itself

Focus on representations and methodologies to capture and process dynamic scenes

Multiple Streams for Free-viewpoint

School OF INFO. & Mecha., GIST

Generalizing 2D video pixels towards 3D◦ Interfacing imCast to visualization toolkit for 3D

effects Instant virtual freeze-and-rotate effects Put interaction mechanism for mouse/key/time

events

Views in 3D Effects

School OF INFO. & Mecha., GIST

Interactive video service for 3D effects◦ Real-time video display with video rotating and rendering◦ Basic procedure

Source -> mapper -> actor -> renderer -> rendering window◦ Applied mapping and rendering process

Views in 3D Effects

School OF INFO. & Mecha., GIST

Environment◦ 8bit uncompressed HD video is transmitted

Over 1Gbps (MTU: 9180 Bytes) Resolution: 1920 x 960

Uncompressed HD video display in 3D effects◦ Real-time video display with interaction◦ Performance problem

While image actor is working, CPU overhead and user memory increase CPU overhead: 20% to 70~90% User memory: 12.5% to 36.7%

Experiment Result

School OF INFO. & Mecha., GIST

Experimental Snapshots

School OF INFO. & Mecha., GIST

Conclusion & Future Works

School OF INFO. & Mecha., GIST

Conclusion & Future works Conclusion

◦ Enhanced uncompressed HD media transport system Preview video support Networked display interface Bi-directional transport system

◦ Extension of uncompressed HD media transport system (ongoing) Multiple streams for free-view-point Views in 3D effects

Future works◦ Interfacing views in 3D effects to networked tiled display (SAGE)◦ Performance problem (CPU overhead, user memory)

School OF INFO. & Mecha., GIST

Thank youAny Questions?

Send inquiry to jcpark@nm.gist.ac.kr