I2/09-April-2003 1

Agent-based End to End Video QoS - Assessment and

Prediction

Vinay Chandrasekhar (2), Tyler Johnson(1), Ketan Mayer-Patel (1), Wesley Miaw (1), John Streck

(2), Amin Vahdat (3), Mladen Vouk (2)

North Carolina Networking Initiative

(1) UNC Chapel Hill, (2) North Carolina State University, (3) Duke University

I2/09-April-2003 2

On Cyberinfrastructure(from the Appendix of the Report of the National Science Foundation

Blue-Ribbon Advisory Panel on Cyberinfrastructure, Jan 2003)

“Cyberinfrastructure makes applications dramatically easier to develop and deploy, thus expanding the feasible scope of applications possible within budget and organizational constraints, and shifting the scientist’s and engineer’s (and educator’s) effort away from information technology development (and manipulation) and concentrating it on scientific and engineering research (and education) . Cyberinfrastructure also increases efficiency, quality, and reliability by capturing commonalities among application needs, and facilitates the efficient sharing of equipment and services”.

I2/09-April-2003 3

Motivation: Support of End-User (Work)flows• W-Flow: Order and way in which we do things to

achieve results (e.g., plan and operate video sessions, construct services, do research, solve problems, etc.)

• Network-end devices (video, sensors, network-based appliance and equipment), computers, storage, networks and associated services and software (“glue”) make sense if they support, with success, the workflows of the intended end-users (a broad base of users).

• Case in point “Video on Tap”

I2/09-April-2003 4

What?

I2/09-April-2003 5

Workflow Overlays

Service-Specific Plug and Play, e.g., “video on tap”

Internet

I2/09-April-2003 6

(Pro-active) Resource (QoS) Provisioning and Management Has Many Aspects

• Distributed Computing, Storage, and Communication (capacity, scheduling, availability, persistence, reliability, portability, interoperability, configuration …)

• Heterogeneity, Collaboration, Autonomy, Federation, At-Will, On-Demand, Cooperation, Sharing, Independence, Non-intrusiveness, Promptness, Standards-based, Appliance-like, …

I2/09-April-2003 7

(Pro-active) Resource (QoS) Provisioning and Management Has Many Aspects (2)

• Security, Credentials, Trustworthiness, Authentication, Access, Authorization, Survivability, Policies, …

• Discovery, Life-time Management, “Memory” (state awareness and persistence), Manageability, Reliability, Performance, Scalability, Survivability, Quality of Service, Help, …

• Pro-active workflow support (synchronous and asynchronous) – “standardization/industrialization of this part is still in its infancy … etc

I2/09-April-2003 8

(GRID/P2P)? Middleware & Infrastructure Enhances Workflows of End-Users thru Pro-Active, Adequate

and Transparent Resource Provisioning

Domain Workflows

Applications

Hardware

Communications

OS GRID/P2PResources

(Middle)ware

E/U

Ine

ffic

ien c

y

I2/09-April-2003 9

Overview• Video teleconferencing over IP has been

an area of considerable interest for North Carolina Networking Initiative (large-scale MPEG2 and H.323 video deployments).

• Special interest – “full stack &path” diagnostic and predictive end to end (E2E) QoS management.

• Experiences with ViDeNet tools – very positive, but not Open Source software.

I2/09-April-2003 10

Goals & Progress

• The goal of the project is to build an Open Source service/agent video quality assessment/prediction tool that can be easily replicated at any university, and whose code base can be improved by the development community over time.

• Elements are in place. We are exploring its capabilities and how it may help us construct a “video overlay”.

I2/09-April-2003 11

Where?

I2/09-April-2003

NCNI Network (logical)Research Triangle, NC

Duke NCSU

UNC-CH

MCNC

Abilene

NCREN

Cisco

NC GigaPop

pendingoperational

OtherCompanies

I2/09-April-2003 13

North Carolina Research and Education Network

Greensboro

Charlotte

Pembroke

WinstonSalem

NCSU

NCSUCentennialCampus

NCCUDuke

UNC-CH

Wilmington

ElizabethCity

Asheville

Cullowhee Fayetteville

Greenville

RTP

MCNC

Boone

MoreheadCity

Rocky Mount

Qwest

RTP RPoP

NCREN3• Increased bandwidth• Increased reliability• Increased resiliency

I2/09-April-2003 14

How?

I2/09-April-2003 15

ViDeNet Global mesh of interconnected H.323 zones

connecting campuses via the Internet and Internet2.

Provides a testbed for inter-networked video and voice over IP architectures and related technology.

Goals: interoperability, low cost, a global voice and video network. http://www.cavner.org/videnet/

Video Development Initiative (ViDe)

International consortium of universities promoting the deployment of digital video in higher education.

Working groups in video conferencing and IP telephony, video-on-demand, MPEG4. http://www.vide.net/

Baseline

I2/09-April-2003 16

Other work

• H.323 Beacon project (Ohio ITEC) – I2

• Access Grid beacon (IP multicast) - GRID

• Commercial tools (e.g., NetIQ) –COTS• Vendor-specific (e.g., Polycom,

VCON)• Other

I2/09-April-2003 17

ViDeNet Scout

Internet

Remote user site ARemote user site B

Remote user site C

Are endpoint pairs ready to support this traffic? Prototype uses Chariot.

End-to-end performance results, not just first to last hop.

Readily deployable.

QoS test configurations possible.

Multi-endpoint network configurations possible.

Remote access.

I2/09-April-2003 18

Example of a mixed voice and video traffic load

• Bi-directional.• Multiple streams with heterogeneous requirements.• Possibly asymmetric.

Remote user site A Remote user site B

64 kbps audio

384 kbps video

64 kbps audio

384 kbps video

64 kbps audio

768 kbps video

I2/09-April-2003 19

Scouting Advanced Networks10 minute 384kbs simulated conference

SURFNet (Netherlands)(good connection)

CUDI (Mexico)(unusable connection)

I2/09-April-2003 20

Scouting Out ProblemsPublic Health Outreach Project

• Remote Health Clinic connected back to Internet2 via xDSL was unusable

• Original diagnosis was h.323 problem

• ISP refused problem ownership until presented with Scout results

I2/09-April-2003 21

Tulane - LANet SimulationLouisiana Statewide T1 Network

• Marginal performance due to widespread T1 architecture.

I2/09-April-2003 22

Example (H.263 stream)

No Load Under Load(few % in net losses)

I2/09-April-2003 23

Video Simulation & Analysis Tool (VSAT) 

Experiment

Send Frame Rate (fps)

Bandwidth

at sender (kbps)

Recv. Frame Rate (fps)

Datagram Jitter (micros

ecs)

Number of

Frames lost/total frames

Number of

Datagrams

lost/total frames

Bandwidth

at receiv

er (kbps)

No load 27 276 27 261(0.2 ms)

0 0 276

Under load

6 64 6 233,013(0.2 sec)

4.8 per 158.9

15.8 per 299.8

64

I2/09-April-2003 24

Some Metrics

• Bandwidth, Jitter, Delay, Loss

• Platforms (CPU, memory, OS, stacks, communication protocols, etc.)

• Application level (e.g., Frame rate, frame jitter, losses, recovery algorithm metrics, stream types, etc.)

I2/09-April-2003 25

Resource Monitoring

• NC State RUM (some other metrics, flows, load, packets, “operational profile” shape)

(demo)

I2/09-April-2003 26

More How?

I2/09-April-2003 27

Architecture (1)• Client-based agents, service-based QoS

analysis.

• Client agents collect data about the application, platform, network performance (currently simulated video stream based probing of real paths, developing new QoS MIB) and estimates video network characteristics then passes that information to analysis services.

I2/09-April-2003 28

Architecture (2)

• Data collection and QoS analysis/prediction services – currently only VQM (Video Quality Manager - VQM). In general, any protocol.

• Analysis services provide an estimate of the end-user QoS and possibly recommend remedies (app, platform, network)

I2/09-April-2003 29

Top-Level Global Architecture

e.g., UNCService Registry

UNCCH Services

Duke Services

UNC-A Service

Workflow Composer

Uses registered services to construct new services and/or workflows. Saves product on a server and registers it.

WorkflowAgent

GT Service s

MCNC Services

“Dials” needed services/workflows and executes/runs through the services/workflow, delivers output to user.

NCSU Services

SOAP Service GatewaysandServiceAgents

Registries (e.g, UDDI) and Context Gateways

Agents and Workflow Tools

• • •

XML data descriptionsWSDL process descriptionsXPDL workflow descriptionsURL/URI descriptorsEtc.

Use

rs

I2/09-April-2003 30

Context Mediation

Semantic Mediation

Workflow Support

Registries

Information Wrappers

Services/Objects

GRID/P2P Middleware &Infrastructure & Resources

User (Matt)

Data SourcesComputational

Resourses

A

Data SourcesData Sources

Computational ResoursesComputational

Resourses

GUI + Canonical Workflow Construction & Execution Agentsand Agent-based End-User Support

Domain-Specific and Abstract Problem Solving Support & Data Integration

Advanced Workflow Support Tools IncreaseProductivity of End-Users

Canonical/UniversalData Transforms& Integration

1

2

3

Networks

I2/09-April-2003 31

Workflow

Process (e.g., WSDL)

Information Flow (e.g., XML)

W-Flow: Precedence,Dependencies, Timing,“Memory,” etc.,e.g., XPDL

Semantics andContext awareness??

I2/09-April-2003 32

Context• McCarthy (87, etc.) – formalization of the context

idea using objects (operations, values, variables, relationships), e.g., ist(NCSU, Professor(mav))

• Formally represented knowledge is based on a conceptualization: the objects, concepts, and other entities assumed to exist in some area of interest and the relationships that hold among them. Every knowledge base, knowledge-based system, or knowledge-level agent is committed to some conceptualization, explicitly or implicitly.

I2/09-April-2003 33

Ontology• An ontology is an explicit specification of a

conceptualization (Gruber93)• An agent commits to an ontology if its

observable actions are consistent with the definitions in the ontology

• Vocabulary with which queries and assertions are exchanged among agents.

• Use of shared vocabulary does not imply sharing of knowledge.

I2/09-April-2003 34

Some Details?

I2/09-April-2003 35

1. Client sends request with characteristics2. rtp-recast resends video from the library3. vicdump dumps reconstructed video frames4. VQM compares original and reconstructed frames5. Oracle returns a quality value

VQM Oracle

Video & NetworkCharacteristics Quality

Library

rtp-recast

VQM

vicdump

Oracle SystemClient Client

Wesley Miaw <wesley@cs.unc.edu> April 21, 2023Computer Science DepartmentUniversity of North Carolina at Chapel Hill

I2/09-April-2003 36

Video QoS MIB - Motivation

H.341 MIB

Defines various control parameters – H.323, H.320, H.245 and Gateway information

Also contains System capabilities, line rates etc.

Does NOT contain actual multimedia data information

RTP MIB Defines RTP Sessions and session entries.

Sender/Receiver tables contain num. of packets and bytes

Does NOT contain frame level information

Does NOT differentiate between audio and video datagrams

Video QoS MIB

All missing information, along with other neede information maintained in one place. Extensible design.

Designed for use with VSAT.

Does not concern itself with control information (at present)

Could use H.341/RTP MIBS in conjunction for control info.

I2/09-April-2003 37

VSAT Block Diagram

java

Java Reporter / Monitor

(SNMP MIB Query and

Display, Oracle Interface)

Monitor Computer

Internet

Pseudo-Video Source

Client Endpoint A

Pseudo-Video Sink

Server Endpoint B

SNMP MIB SNMP MIB

N.B. Use of IPERF(vbr, cbr)

I2/09-April-2003 38

Video QoS MIB Design

C o n ta ct

S e n d er T a b le R e ce ive r T a b le

Q oS L o gs

R o ot

Sender Table: Session ID, Test Number, Video Data Bytes Sent, Video Frames Sent, Number of Video Datagrams Sent, Video Frames per Second, Max Encoding Rate, Activity Level, Video Picture Type, Test Time duration Receiver Table: Session ID, Test Number, Video Data Bytes Received, Video Frames Received, Number of Video Datagrams Received, Effective Video Bandwidth, Video Datagram Jitter, Number of Lost Datagrams, Number of Lost Frames, Datagrams Transferred

I2/09-April-2003 39

Screen SnapshotsVSAT GUI

I2/09-April-2003 40

VSAT GUI

I2/09-April-2003 41

Monitor GUI

I2/09-April-2003 42

Bigger Picture - Overlays• Large-scale utility for network services

– Web services, multimedia distribution, event notification, application blades, overlays

• Challenges– Scalable to 10k’s sites

– Adaptive to factor of 1000 spike in load

– Fault tolerant: failures are the common case

• Dynamically adapt to changes in utility members, client access patterns, network conditions– Under resource constraint: provision for target levels of

performance, availability, and data quality

I2/09-April-2003 43

OPUS: An Overlay Peer Utility Service

• Dynamically allocate resources to competing services– Based on changing application and network characteristics, SLAs

• Create topology based app requirements– Bandwidth, latency, loss rate, cost ($) sensitivity

App demand

Overlay node

Peering

I2/09-April-2003 44

Adaptive Cost, Delay, Resource Constrained Overlays

• Model– Nodes self-organize to build efficient data

dissemination structure• Pre-specified root of overlay tree

– Independent metrics assigned to links: e.g., cost and delay, other resources

• For scalability, cannot:– Broadcast

– Use centralized information

– Perform global locking, global probing

I2/09-April-2003 45

Scalability and Adaptivity • Adaptivity

– Must constantly adapt to changing network conditions– Constant probing of peers to determine current best

parent– Worst case: requires O(n) state and O(n) probing

• E.g., Narada, RON

• Scalability– Limit state and probing overhead to O(log n)

• E.g., Distributed hash lookup schemes

• Is it possible to be both adaptive and scalable?– In large-scale distributed environment

I2/09-April-2003 46

Flexibly Trading Cost for Delay

11.11.21.31.41.51.61.71.81.9

22.12.22.32.42.52.62.72.8

1 1.2 1.4 1.6 1.8 2 2.2 2.4 2.611.11.21.31.41.51.61.71.81.922.12.22.32.42.52.62.72.8

Cos

t/MST

Cos

t

Ach

ieve

d D

elay

/SPT

Del

ay

Delay Bound/SPT Delay

costdelay

I2/09-April-2003 47

Concentric Layers of Resources

Local

Immed.

Metro

State/National Global

Delay (ns to ms)CouplingResponseCapacityAvailabilityReliability...

CoverageCostAccessUsability…

(resources “come” to users, not vice versa)

I2/09-April-2003 48