Introduction and Architectures 29/8 - 2005 INF5070 – Media Storage and Distribution Systems:

Introduction and Introduction and ArchitecturesArchitectures

29/8 - 2005

INF5070 – Media Storage and Distribution Systems:

2005 Carsten Griwodz & Pål Halvorsen

INF5070 – media storage and distribution systems

Overview

Intro about the course multimedia applications and challenges

Architectures Media (Video) on Demand Machine internals Video server structures Examples

INF5070:The Course



Lecturers

Carsten Griwodz email: griff @ ifi office: 3343

Pål Halvorsen email: paalh @ ifi office: 3345



Content

Network

Network

Network

Network

architectures

file systems

resource scheduling

media data

protocols

topologies

distribution



Content - lectures System architectures

(server and system designs)

Media and user behavior (wire and file formats, codecs, processing multimedia data, user interactivity)

Resource management (CPU and memory management)

Protocols with and without Quality of Service (QoS) (specific and generic QoS approaches)



Content - lectures Storage systems

(management of multimedia files, retrieval)

Distribution(use of caches and proxy servers)

Peer-to-Peer (various clients, different amount of resources)

Guest lecture 7/11: The FAST searching system (architecture: resource utilization and performance, storage and distribution of data, parallelism, etc.)



Content - student assignment Mandatory student assignment

(will be presented more in-depth later):

write a project plan describing your assignment (3/10) write a report describing the test and its results, and give a

presentation to the class at the end of the course (28/11)

for example (examples from earlier): TCP friendliness variations New TCP implementations Network emulators Comparison of Linux schedulers File system benchmarking (different OSes and file systems) Comparison of open source video servers (Helix vs. Darwin) Comparison of methods for network performance monitoring …



Goals Media servers and distribution system

media characteristics architectures system support protocols distribution mechanisms …

Be able to evaluate any combination of these mechanisms, e.g., data placement on disk and memory caching

algorithm video popularity and proxy caching algorithm …



Exam

Prerequisite: approved presentation of student assignment

Oral exam (??/12-2005): all transparencies from lectures content of your own student assignment

Applications and Challenges



Applications Multimedia enriches the user interface giving new

applications

Broadcast – server is only VCR substitute (e.g., MTV Europe)

Media-on-Demand (MoD) Network-based, user controlled distribution of media content,

e.g.: Video-on-Demand (VoD) (e.g., Bell Atlantic (‘93)~1000 users, 700

choices) News-on-Demand (NoD) (e.g., CNN, BBC web today, BAD quality) Learning-on-Demand (LoD) (e.g., the OMODIS project)

Virtual worlds (typically live, not on-Demand) video conferencing (e.g., USIT’s electronic classroom) games (e.g., Anarchy online, Quake, …)



Requirements Application

QoS – time sensitivity resource capabilities –

support interactive streaming of multimedia content

Business scalability reliability

Architectural topology cost vs. performance



Technical Challenges User end system

real-time processing of streams (1000 MIPS for an MPEG-II decoder)

request/response delay (< 150 ms for videophones) high data rates, e.g., MPEG-II DVD quality:

average video rate of 3.5 Mbps average transport stream of 4 – 8 Mbps (video, audio, headers, error

protection) max. total video data rate of 10.08 Mbps max. user rate of 11.08 Mbps (all included like control signals)

Network real-time transport of continuous media data TCP fairness mobility …



Technical Challenges Servers (and proxy caches)

storage real-time retrieval of contiguous media streams, e.g.:

o 4000 movies * 90 minutes * 15 Mbps (HDTV) = 40.5 TBo 2000 CDs * 74 minutes * 1.4 Mbps = 1.4 TB

I/O many concurrent clients expecting continuous playout of

DVD-quality video (~ 4Mbps)o disk: Seagate X15 - ~400 Mbpso network: Gb Ethernet (1/10 Gbps) o bus(ses): - PCI 64-bit, 133Mhz (8 Gbps)

- PCI-Express (2.5 Gbps each direction/lane)

computing in real-time encryption adaptation transcoding …



Media-on-Demand (MoD) Systems Classification parameters

interaction structure of movies presentation form

Common directions analog digital media distribution interaction media broadcasting media multicasting personalized

media linear movies branched movies variable movies

Evolution broadcast - traditional, no user control near MoD - same media distributed in intervals (pay-per-view) true MoD - full user control, VCR capabilities, bidirectional



Television (Broadcast)

sender

ch

an

nels

time

• analog or digital• traditionally, one program per channel

analog use frequency division multiplexing only digital may additionally use time division multiplexing inside one frequency (several programs per channel)

• linear movies

receiver(s)



Near Video-on-Demand (NVoD)

sender

ch

an

nels

time

• analog or digital broadcasting• one program over multiple channels• time-slotted emission of the program• linear movies

receiver(s)



(True) Video-on-Demand (VoD)

sender

movie

s

time

• digital uni- or multicasting• control channels• linear movies

receiver(s)



Comparison: NVoD vs. TVoD

NVoD TVoD

Response delayed immediate

Servicesprovide some flexibility compared to traditional

TV

improves video rentals:any video, any time

Costscheap:

more clients at lower price compared to TVoD

expensive:existing infrastructure often

not cost-competitive for households

(need to be equal to rentals)

Suiteddigital broadcast(ViaSat, TV1000)

specific (smaller) environments

(hotels, airplanes, …, cities,…)



“Interactive Vision”

sender

movie

time

• digital uni- or multicasting• control channels• fixed non-linear movies

receiver(s)



“Cyber Vision”

sender

time

• digital uni- or multicasting• control channels• variable non-linear “movies”, e.g.,

- games, virtual reality, …

receiver(s)



Application Classification Overview

movie

str

uctu

re

linear

branched

variable

interaction

unidirectional

bidirectional

pre

senta

tion

form

analog

digitalVoD

TVNVoD

Interactive Vision

Cyber Vision

HDTV

VIDEO

Media (Video) on Demand



Challenges VoD goals:

network-based distribution of media content to consumers

bring control to users

VoD in LANs is solved: OVERPROVISIONING works established in studio business established in small area (hotel/hospital/plane/…)

businesses

Network



Challenges VoD in WANs

assumptions: overprovisioning of resources will NOT work no central control of delivery system

programs: need for interoperability – not from a single source need for co-operative distribution systems

amount of data: estimated 65000 movies made in 1995 260 TB MPEG-2

data additionally, data from TV-series, sport clips, news, …

historically: much attention as “interactive TV (ITV)” some years ago many (not successful) field trials now: interest turned to Internet-based systems



ITV Network Architecture Approaches

WAN backbones SONET ATM

Local distribution network ADSL (asymmetric digital subscriber

line) FTTC (fiber to the curb) FTTH (fiber to the home) HFC (hybrid fiber coax) (=cable

modem) E-PON (Ethernet passive optical

network) …

Internet WAN Diffserv over MPLS

(multi-protocol layer switching) point-to-point Gbps ethernet

Internet local IP over the old distribution networks

ATM / SONET

backbonenetwork

wireless

cable

telephone

ADSLInternet based systems



Concerns: Internet-Based VoD Systems

Can technical problems be mastered? broadband communication to every home user-friendly end systems server technology

Market success? what prices will consumers accept?

high equipment (HW & SW) costs data costs

will it be competitive? to existing TV programs to video rentals

what is the consequence of ITV field trials in USA and Europe? no big success only a few private consumers willing to participate trials were cancelled or switched from TV-based to PC-based

platforms



Driving Forces Hardware/software (IT) companies

computer (e.g., IBM, HP, Sun, Microsoft, …) consumer electronics (e.g., Sony, Philips, …)

Network companies telephone (e.g., Telenor, Telia, BT, AT&T, …) cable TV (e.g., UPC, Time-Warner Cable, …)

Content companies media

movies (e.g., Time-Warner, Disney, …) TV programs (e.g., NRK, TV2, TV3, …) hyper media information bases (e.g., Springer, Bertelsmann, …)

home-shopping (e.g., Amazon, Ebay, …) video games (e.g., Nintendo, Sega, Microsoft, …)



VoD Deployment Status – I Digital Video Broadcast (DVB)

no VoD cable, antenna, or satellite broadcast some NVoD scheduling approaches (e.g., TV 1000)

Digital Audio-Visual Council (DAVIC) defines interfaces only no standardization of algorithms for interoperation closed project

Internet Engineering Task Force (IETF) currently no large-scale, wide-area video

distribution sporadic use of cooperative web caching starting AV caching considerations defines protocols and inspires interoperability

testing

Broadcast world suited for Near VoD

Broadcast world suited for True VoD

Internet world suited for True VoD

MPEG 4 suited for

Interactive Vision



VoD Deployment Status – IIDATA CONTROLIE

TF

RTP: Real-Time Protocol package & timing information for

transfer profiles for each encoding format

RTCP: RTP Control Protocol report exchange, allows tuning at the sender

RTSP: Real-Time Streaming Protocol controls: setup, teardown, start, stop

SDP: Session Description Protocol carried by RTSP encoding information, timing, meta info

DA

VIC

MPEG-2 Transport: Moving Pictures Expert Group defines encoding format, packaging, timing, scaling, error correction requires other means of addressing

DSM-CC: Distributed Storage Media- Command & Control MPEG-2 substandard: addressing, setup, teardown, start, stop, … independent standard: complete management standard (usually ignored by industry)



VoD System Architecture

backbonenetwork

local distribution

network

local distribution

network

local distribution

network



VoD Storage Hierarchy Use hierarchies

Popularity of movies:not all are equally popular – most request directed to only a few (Zipf distribution)

Straight forward hierarchy: popular videos replicated

and kept close to clients locality vs.

communication vs. server costs

end-systems

local servers

master servers

regionalservers

completeness of available content



VoD Components Servers

Networks backbone local networks

Intermediate nodes routers proxy cache servers

End-systems

PCs TV sets with set-top boxes

Traditional Server Machine

Internals



General Operating System Structure and Retrieval Data Path

file systemcommunication

system

application

user space

kernel space



Example: Intel Hub Architecture (850 Chipset) – I

CPU socket

RDRAM connectors

PCI connectors

I/O Controller Hub

Memory Controller Hub

Intel D850MD Motherboard:

system bus

RDRAM interface

hub interface

PCIbus



Pentium 4Processor

registers

cache(s)

Example: Intel Hub Architecture (850 Chipset) – II

I/Ocontroller

hub

memorycontroller

hub

RDRAM

RDRAM

RDRAM

RDRAM

PCI slots

PCI slots

PCI slots

system bus(64-bit, 400/533 MHz

~24-32 Gbps)

hub interface(four 8-bit, 66 MHz

2 Gbps)

PCI bus(32-bit, 33 MHz 1 Gbps)

RAM interface(two 64-bit, 200 MHz ~24 Gbps)

network card

disk

file system

communication system

application


system

application

disk network card

Note:these transfers only show data movement between sub-systems and not the commands themselves. Additionally, data touching operations within a sub-system will require that data is moved from memory and to the CPU, e.g.: - checksum calculation - encryption - data encoding - forward error correction



POWER 4 chip

Example: IBM POWER 4

PCI slots

PCI slots

network card

disk

file system

communication system

application


system

application

disk network card

CPU

L1

CPU

L1

core interface switch

L2

fabric controller

GXcontroller L3 controller

remote I/O(RIO)

bridge

PCI host bridge

PCI host bridge

PCI-PCI bridge

PCI-PCI bridge

L3

RAM

RAM

RAM

memorycontroller

GX bus(two 32-bit, 600 MHz ~35 Gbps)

PCI busses(32/64-bit, 33/66 MHz 1-4 Gbps)

(four 6

4-bit, 400 M

Hz ~95 Gbps)

RIO bus(two 8-bit, 500 MHz ~7 Gbps)

(four 6

4-bit, 400 M

Hz ~95 Gbps)

(eight 32-bit,

400 MHz

~95 Gbps)

Note:Again, data touching operations add movement operations



Example: AMD Opteron & Intel Xeon MP 4P servers


system

application

disk network card

Know your hardware – different configuration may have different bottlenecks



Server Internals Data retrieval from disk and push to network

buffer requirements bus transfers CPU usage

concurrent streams (can be merged??) storage (disk) system:

scheduling – ensure that data is available in time block placement – contiguous, interleaving, striping

…

Stable operations: redundant HW multiple nodes

Much more, e.g., caching/prefetching, admission control, …

Video Server Structure



Video Server:Server Components & Switches

Internal content directory: External content directory:

storage device

network attachment

content directory/memory management

file system

storage management

controller

switch

switch

switch

switch

switch

[Tetzlaff & Flynn 94]

switch

switch

switch

switch

switch

IP, …

RPC in application, …

NFS, …

AFS, CODA, …

distributed OS, …

Disk arrays (RAID), …

IBM TigerShark switch

IBM TigerShark

switched network

switch

HP, DEC, Novell, …

HP, DEC, Novell, ….

switched network

switch



Video Server:Simple General Server Architecture

storagesubsystem

networksubsystem

processor subsystem

dataserver

applicationserver

controlserver

clie

nts

Storage subsystem:• stores data• different devices

Network subsystem:• transmit MM data

Processor subsystem:• executing part• management and operations

Application server: • user interface• billing• content database• user database• service gateways

Control server: • administrator• admission control• optimization

Data server: • data delivery• “specialized file system”• buffer manager• data importer/exporter

datacontrol

[Sitaram & Dan 00]

A similar architecture is used in SGI/Kassena Media Base & IBM VideoCharger



Video Server:Directory Access & Data Retrieval

Two-step retrieval: “challenge”:

resource management

Request redirection: “challenge”:

client gets data from another machine

Network

data

content

Network

data

contente.g., Hypertext Transfer Protocol (HTTP), Session Initiation Protocol (SIP),maybe Real-Time Streaming Protocol (RTSP)

givepermission

e.g., Real-time Transport Protocol (RTP),maybe RTSP

e.g., TCP handoveror handoff



Pull model: client sends several requests deliver only small part of data fine-grained client control favors high interactivity suited for editing, searching,

etc.

Push model client sends one request streaming delivery favors capacity planning suited for retrieval,

download, playback, etc.

server client

server client

Video Server:Directory Access & Data Retrieval

e.g., HTTP, File Transfer Protocol (FTP),RTP + RTSP

e.g., Priority Progress Streaming (PPS)



Video Server:Server Topology – I

Single server easy to implement scales poorly

Partitioned server users divided into groups content : assumes equal groups location : store all data on all

servers load imbalance

Network

Network

Network



Video Server:Server Topology – II

Externally switched servers use network to make server pool manages load imbalance

(control server directs requests) still data replication problems (control server doesn’t need to

be a physical box - distributed process)

Fully switched server server pool storage device pool additional hardware costs e.g., Oracle, Intel, IBM

Network

data

data

data

control

Network

data

data

control

I/Oswitch



Video Distribution Server:Typical In the Internet Today

Push systems(pull in video editing/database systems)

Traditional (specialized) file systems – not databases – for data storage

No in-band control (control and data information in separate streams)

External directory services for data location(control server + data pump)

Request redirection for access control

Single stand-alone servers (fully) switched servers

Server Examples



Video Server Components Researchincoming

resolve request

control/application

server

data server

content directory

delivered resolution

storage device

network attachment

memory management

file system

storage management

controller

delivered data

network attachment

incomingdata request

IBM WatsonTetzlaff, Kienzle, …



Video Server Components Research

data server

storage device

network attachment

memory management

file system

storage management

controller

delivered data

incomingdata request

Fall, Druschel, Pai, Silberschatz, Özden, Buddhikot, Miller, …

Shenoy, Nirajan, Silberschatz, Özden, Martin, Haskin, …

Goyal, Vin, Peterson, Tetzlaff, Kienzle, …

Härtig,Sitaram,Dan,Nahrstedt,Steinmetz,Klas,Shulzrinne,Coulson,Seltzer,Rangan,Zhang,Hutchinson,…

Ghandeharidzadeh, Zimmermann, Haskin, …

Jones, Nieh, Chen, Berson, Reddy, Ghandeharidzadeh, Zimmermann, Haskin, …



Video Server “Product Status”1) Real server, VXtreme, Starlight, VDO, Netscape Media Server,

MS Media Server, Apple Darwin1) Real server, VXtreme,VXtreme, Starlight, VDO, Netscape Media Server,VDO, Netscape Media Server,

MS Media Server, Apple Darwinuser level server

standardOS

all standard HW

RTPRTSP

2) IBM Mediastreamer, Oracle Video Cartridge, N-Cube

2) IBM Mediastreamer, IBM Mediastreamer, Oracle Video Cartridge, N-Cubeuser level layer

scalable, RT-aware OS,RT OS, or

OS derivation

custom/special HW

ATM, analogDSM CC, private

3) SGI/Kassena Media Base, SUN Media Center, IBM Video Charger

user level server

RTextensions

selected standard HW

RTPRTSP

standardOS

MMFS

3) SGI/SGI/Kassena Media Base, SUN Media Center,SUN Media Center, IBM Video Charger



userkernel

server

Real Server User space implementation

one control server several protocols several versions of data

in same file adapts to resources

Several formats, e.g., Real’s own MPEG-2 version with

“stream thinning”(dropped with REAL )

Does not support Quality-of-Service load leveling

track

1

track

2

index

request

IP

UDP

RTP/RTCP

Real’sprotoc

ol

TCP

1 23

back

pre

ssure

feedback



VSDwith EDF

IBM Video Charger May consist of one

machine only, or … … several IBM’s

Advanced Interactive eXecutive (AIX) machines

Servers control data

Lightly modified existing components

OS AIX4/5L virtual shared disks

(VSD)(guaranteed disk I/Os)

Special components TigerShark MMFS

(buffers, data rate, prefetching, codec, ...)

stream filters, control server, APIs, ...

control

AIX

SP2

cro

ssbar

swit

ch

specificcontrol server

RTSP

RTPencryptfilter

TigerSharkMMFS

VSD

UDP

IP

distributed computing environment RPC

video stream API

mlib API

DESCRIBE

SETUP

PLAY

TEARDOWN



IBM Mediastreamer Version of Video Charger

failed project to guarantee MPEG-2 over ATM and Cable

similar machine setup as Video Charger (machine cluster)

special HW: SCSI controller MPEG-2 ATM or analog cable out moved to Video Charger

Unlike Video Charger, Mediastreamer runs on old IBM machines due to special HW

Special components TigerShark control server APIs special board

control

AIX

SP2

cro

ssbar

swit

ch

specificcontrol server

RTSP

distributed computing environment RPC

video stream API

mlib API



n4x media hubs:• Intel 860 Chip Set • Intel 1.5 GHz Xeon CPU• Up to 2 GB Rambus Memory• Five 64 bit 66Mhz PCI slots• “Special” PCI slot (HIB board)• nHIO hypercube I/O

request

nCUBE Original research from Cal Tech/Intel (‘83) Bought by C-COR in Jan. 05 (~90M$)

One server scales from 1 to 256 machines, 2n, n [0, 8], using a hypercube architecture

Why a hypercube? video streaming is a switching problem hypercube is a high performance scalable switch no content replication and true linear scalability integrated adaptive routing provides resiliency

Highlights one copy of a data element scales from 5,000 to 500,000 clients exceeds 60,000 simultaneous streams 6,600 simultaneous streams at 2 - 4 Mbps each

(26 streams per machine if n = 8)

Special components boards with integrated components TRANSIT operating system n4 HAVOC (1999)

Hypercube And Vector Operations Controller ASIC-based hypercube technology

n4x nHIO (2002) nCUBE Hypercube I/O controller (8X price/performance)

memory PCI bus

configurableinterface

8 hypercube connectors

vector processorSCSI ports



nCUBE: Naturally Load-balanced

Content striped acrossall disks in the n4x server

Video Video StreamStream

Disks connected to All MediaHubs Each title striped across all MediaHUBs Streaming Hub reads content

from all disks in the video server

Automatic load balancing Immune to content usage pattern Same load if same or different title Each stream’s load spread over all

nodes

RAID Sets distributed across MediaHubs Immune to a MediaHUB failure Increasing reliability

Only 1 copy of each title ever needed Lots of room for expanded content,

network-based PVR, or HDTV content



nCUBE: Sample deployments & Cost

TWC Los Angeles – South Bay HE

Monterey Park CA 2040 streams from 18 n4

MediaHUBs

West Covina CA 840 streams from 8 n4

MediaHUBs

Pasadena CA 840 streams from 8 n4

MediaHUBs

Glendale CA 1320 streams from 12 n4

MediaHUBs

Long Beach CA 1320 streams from 12 n4

MediaHUBs



Small Comparison

Real Video Charger nCUBE

standard HW selected HW special HW

each machine its own storage, or

NFS

shared disk access,

no replication (except for load leveling and fault tolerance)

shared disk access,

no replication

single OS image cluster machines using switch

cluster machines using wired cube

user space server user space server and loadable

kernel modules

server in both kernel and user

space



Summary Multimedia applications and challenges

Media (Video) on Demand

Machine internals

Video server structures

Video server examples

Introduction and Architectures 29/8 - 2005 INF5070 – Media Storage and Distribution Systems:

Documents

Transcript of Introduction and Architectures 29/8 - 2005 INF5070 – Media Storage and Distribution Systems: