1

User-perceived QoS in networked multimedia services

Harald Øverby, PhD

28/10-2008

2

Topics

• Quality of Service (QoS)– User-preceived QoS

• Networked multimedia services– Voice-over-IP (VoIP) (1 hour)

• Shared packet redundancy in VoIP

– Networked online games (1 hour)• Real Time Strategy (RTS) game Warcraft III

How does QoS influence user experience in networked multimedia services?

3

Resources

• Y-W Leung, “Shared packet loss recovery for internet telephony”, IEEE communications letters vol. 9, no. 1, january 2005

• P. O. Osland et al., “Perceived VoIP quality under varying traffic conditions”, Proceedings of the Nordic Teletraffic Seminar (NTS), 2004

• N. Sheldon et. al., ”The effect of latency on user performance in Warcraft III”, Proceedings of the 2nd workshop on Network and system support for games

• Wikipedia: VoIP: http://en.wikipedia.org/wiki/Voice_over_IP

4

What is Voice over IP (VoIP)

• Voice over IP, IP Telephony, Internet Telephony, …

– Packet switched telephony service

– Independent packets routed over a packet network

• Alternative to circuit switched telephony service

– End to end circuits established for each call

– ”Stupid” network vs Intelligent network

5

VoIP functionality

• Voice is encoded in packets, and transmitted when necessary• Ability to transmit more than one telephone call down the same

broadband-connected telephone line. • Many VoIP packages include PSTN features that most telcos

normally charge extra for, or may be unavailable from your local telco, such as 3-way calling, call forwarding, automatic redial, and caller ID.

• VoIP can be secured with existing off-the-shelf protocols such as Secure Real-time Transport Protocol.

• VoIP is location independent. • VoIP phones can integrate with other services available over

the Internet.

6

QoS in VoIP

• QoS definition (one of many…)– Degree of compliance of a service to the agreement that exists

between the user and the provider of this service.

• QoS (in broad sense) encompasses– Performance

– Dependability

– Security

• In this talk we focus on Performance aspects of VoIP

7

Performance aspects of VoIP

• Performance is characterized by a set of paramters, which can be measured quantitatively.

• Important performance related QoS parameters in VoIP includes– Available bandwidth

– Packet loss

– End-to-end delay

– Delay jitter

8

User-perceived QoS

• How the user perceives the service quality• Measured using e.g. questionaires• E.g.: Will the user see difference between 0 end-to-

end delay and 0.1 ms end-to-end delay?• Measured using Mean Opinion score (MOS)

– 5: Max, 1: Minimum

• Influenced by QoS parameters

9

1. Available bandwidth

• Measured in bps• Depends on the codec

– How the voice signals are decoded– Trade-off between complexity, bandwidth savings and other

performance related QoS parameters

• Bandwidth requirement dependent on the codec– H.323 and SIP are popular protocols

• Overhead may constitute a significant part of total bandwidth consumption– E.g. data=5.6 kbps, overhead=12.4 kbps, total=18 kbps

10

2. Packet loss

• Measured in ratio of lost packets to all packets sent• Caused by

– Bit errors

– Queue overflow in routers

– End-to-end delay for packet too high

• Acceptable levels of packet loss ratio is ~2 % for VoIP

• Dependent on the codec

11

3. End-to-end delay

• Average delay between a user say a word until the received hears the same word

• The end-to-end delay is comprised of:– Local computer delay– Compression delay– Packetizing delay– Transmission delay

• Propagation delay

• Queueing delay

– …

• End-to-end delay below 150 ms is acceptable

12

4. Delay jitter

• Variations in the end-to-end delay– Difference in the end-to-end delay between independent packets

• Caused by– Variations in the queueing delay

– Packetizing delay

– Computer processing delay

• Delay jitter below 50 ms is acceptable for VoIP

13

Shared packet redundancy in VoIP• Novel mechanism to reduce the packet loss ratio

(PLR)• Adding redundancy packets to a set of data packets• Requires additional bandwidth• Influences the end-to-end delay and delay jitter?• Complexity issues?

14

Architecture• Each gateway serves N users• Packets arrive to the gateway in slots

– n packets in a certain time slot

• The gateway has B channels in each time-slot• Packets may be lost in the Internet

15

Architecture

User1

User2

User3

N=3

Gate-Way

slot5 slot4 slot3 slot2 slot1

n=2 n=0 n=3 n=2 n=1

Data1_3 Data1_2

Data1_5 Data1_3 Data1_1

Data1_5 Data1_3 Data1_2

B=2

16

Shared packet redundancy

1. A set of n data packets, add r redundancy packets

2. Transmit all n+r packets

3. If received at least n packets, reconstruction of lost data packets is possible.

• Example:

1. From 3 data packets we create and add 2 redundancy packets

2. All 5 packets are transmitted

3. If we receive at least 3 packets (among 5 packets), reconstruction is possible -> no data packets are lost

17

Example

• 2 data packets, 1 redundancy packet– Redundancy packet generated using bitwise XOR

in data packets– Red=Data1Data2

• If Data1 is lost, it can be reconstructed as Data1=Data2Red

• If Data2 is lost, it can be reconstructed as Data2=Data1Red

• If two or more packets are lost, reconstruction is not possible

• With more redundancy packets, reed solomon codes must be used

Data1 Data2 Red

1 1 0

1 0 1

0 1 1

0 0 0

18

Shared packet redundancy in VoIP

data1

data2

data3

red1

red2

data1

data2

data3

red1

red2

Router Sent

data3

red1

red2

Received

data3

Reconstruction

data1

data2

19

Parameters• N: number of customers served• n: outgoing voice streams• B: available capacity (slots)• e: Packet loss ratio : probability that a session is active

– The probability that a user has a packet to send

• A: activity probability (probability of n active voice streams)

20

Erasure coding

• n=B: No modifications• n>B: Drop packets• n<B: Add B-n redundancy

packetsn=3 withprob. An

B=5

N=4

Packet set

21

Erasure conding in VoIP

• Always transmit B packets in a packet set– Case 1: n data packets and B-n redundancy packets

• Packets are lost if less than n packets are received• Else, lost data packets may be reconstructed

– Case 2: n=B data packets and 0 redundancy packets• Lost packets cannot be reconstructed

• Packet sets are independent

22

Analysis

23

Analysis: Case 1: n<B

• n voice (data) packets are transmitted• B-n redundancy packets are added to the data

packet set.• A packet is lost when

– It is dropped in the network AND

– Less than n of the remaining B-1 packets are dropped

24

Analysis: Case 2: n≥B

• No redundancy packets are added• Packets are lost due to

– Overflow in the server

– In the network with probability e

• Example– B=80, n=100, e=0.1

– P(E|An)=(100-80)/100+80*0.1/100=0.28

– Of 100 packets: 20 are lost at server, and 8 in the network

25

Analysis

• Overall loss formula• Summing over all n

26

Results

B=100=0.5

27

Qualitative evaluation of the shared packet redundancy scheme for VoIP

Parameter w/o shared packet redundancy

w/ packet redundancy

Available bandwidth

Moderate Increased

Packet loss Moderate Reduction

End-to-end delay Moderate Increased

Delay jitter Moderate Moderate

Complexity Moderate Increased

28

Topics

• Quality of Service (QoS)– User-preceived QoS

• Networked multimedia services– Voice-over-IP (VoIP) (1 hour)

• Shared packet redundancy in VoIP

– Networked online games (1 hour)• Real Time Strategy (RTS) game Warcraft III

How does QoS influence user experience in networked multimedia services?

29

Motivation

• Computer/video games have become a multi-billion industry and is a driving force for technology development: Hardware, Software, Networks, etc.

• Games are played by people of all ages - female and male.

• Games are used in serious applications areas: Education, Work training, Physical exercise, Social training, Brain training, etc.

30

What make games intriguing?

• Characteristics of enjoyable activities (playing games) can be divided into three categories:

– Challenge: • Mastering a challenge can give a self-esteem boost.• Ideally the game should adjust to players abilities.

– Fantasy: • Fantasies and abstractions enhance and make the experience more interesting.

– Curiosity:• Curiosity is the motivation to learn and investigate.• Desire to bring completeness, consistency to their knowledge

31

Game genres• Action games - shooting games, platform games• Adventure games - logic puzzles in a virtual world• Fighting games - games involving combat• Puzzle games - e.g. Tetris• Role playing game - players assume the role of

imaginative creature or person• Simulations - the player is put in control of

simplified process. E.g., flight simulators, SimCity, driving simulation etc.

• Sport games - games involving sports• Strategy games - typically commanding

armies and warfare planning.

32

Online Gaming in General

• Most new-released games for PCs and game consoles (PS3, Xbox 360 and Wii) provide online game functionality.

– For some games, online-features are main parts of the game (Battlefield)– For other games online-features are add-on functionality (Burnout)– The number of player supported varies from game to game (genre)

• For consoles, Xbox Live and Playstation Network (PSN) are driving forces for online game functionality/support.

• Xbox Live and PSN have initiated series of simpler online games XBOX Live Arcade, and PSN games (often retro games).

• XNA from Microsoft enables independent developers to create own online games distributed on Xbox Live.

33

Main Challenges for Online Gaming in General• Latency (main challenge!)

– Ensure smooth gameplay and that other players move realistically (not in steps, but in one motion).

– Ensure that variation in latency does not affect players performance/scorein the game.

• Bandwidth– Ensure that variation in bandwidth does not affect players

performance/score in the game.

• Heterogeneity:– Support various types of hardware (PC, consoles) and services

(Xbox Live, Playstation Network, etc)

34

Mobile Online Gaming

• Mobile online game functionality is becoming more and more common on Playstation Portable and Nintendo DS (add-on to original game): E.g., Killzone: Liberation, Test Drive Unlimited

• Few online games for mobile phones:– Online highscore lists– Turn-based and slow-paced games– MMOG on mobile phones in Japan: Samurai Romanesque

• Mobile Online Gaming has high potential as there are plenty of users and the users are “always” online.

35

MMOG/MMO

• Massively multiplayer online game (MMOG/MMO): Computer game which is capable of supporting hundreds or thousands of players simultaneously.

• Games played over the Internet• Feature at least one persistent world.• Enable players to cooperate and compete on a grand

scale.• Most MMOGs require a monthly subscription fee.• Huge success last decade

36

Types of MMOGs (Massive Multiplayer Online Games)• MMORPG (Role-playing Game): Guild Wars, FF XI, WoW,

EverQuest.• FPS (First-Person Shooter): Counterstrike, Call of Duty.• RTS (Real-Time Strategy): Shattered Galaxy, Boundless Planet,

Dreamlords, Ballerium.• MMODG (Dance/Rhythm Games): DANCE! Online, O2 Jam,

Audition Online.• MMOR (Racing): Test Drive Unlimited, RaceLands, KartRider.• MMOSG (Social/Strategy/Sports Game): Second Life, Home,

Empire.• MMO Real-words simulations: flight-simulations, traffic control,

military, etc.

37

Main Challenges for Massive Multiplayer Online Games• Scalability/bandwidth to servers! (Main challenge)

– Allow several players to play simultaneously in the same game world.– Divide the world into zones served by separate servers.– Run same game worlds/zones in parallel.

• Consistency– Ensure that changes to the game world is consistently distributed among all

players.

• Latency– Ensure that the players have a responsive gameplay and that game objects

move around smoothly.

• Bandwidth to clients– Used to be a challenge when players used modem. With broadband

connections this is no longer a big challenge.

38

The Effects of Latencyon User Performance in Warcraft III

Nathan Sheldon, Eric Gerard, Seth Borg, Mark Claypool, Emmanuel Agu

Computer Science DepartmentWorcester Polytechnic Institute

Worcester, MA, USA

http://www.cs.wpi.edu/~claypool/papers/war3/

39

Network Games and Latency• Latency degrades performance of interactive

applications– Web-browsing – seconds

– Audioconference – 100’s of milliseconds

– First Person Shooters (FPS) – 100’s of milliseconds• Real-Time Strategy (RTS)?

• Knowing effects of latency useful for– Building better network games

– Building better networks to support games (QoS)

Effects of Latency on Warcraft III (RTS)

40

Warcraft III Overview

RTS User Interaction Components:• Exploration• Building• Combat

41

Exploration Map

Performance?• Time(to reach end)

42

Building Map

Performance?• Time(to build tech-nology tree)

43

Combat Map

Performance?• Games Won• Unit Scores

44

Controlling Latency

• Warcraft III uses client-server– Set computer B as server (also a client)

– Set computer C or D as client

• NIST Net on computer A – Induce latency [0 ms to 3500 ms]

45

Building and Latency

Build Time vs. Latency

R2 = 0.0516

7:40

7:55

8:09

8:24

8:38

8:52

9:07

9:21

9:36

0 500 1000 1500 2000 2500 3000 3500 4000

Latency (ms)

Bu

ild

Tim

e (

m:s

)

Time To Build Linear (Time To Build)

46

Exploration and LatencyExplore Time vs Latency

R2 = 0.6334

3:50

4:04

4:19

4:33

4:48

5:02

5:16

5:31

0 200 400 600 800 1000 1200

Latency (ms)

Ex

plo

re T

ime

(m

:s)

Time To CompleteLinear (Time To Complete)

47

Combat and Latency (1)Unit Score Difference vs. Latency

R2 = 0.0138

-3000

-2000

-1000

0

1000

2000

3000

0 200 400 600 800 1000 1200 1400 1600

Latency (ms)

Un

it S

co

re D

iffe

ren

ce

48

Combat and Latency (2)

49

Bandwidth LAN vs. Internet

3.8 Kbps

4.0 Kbps

6.8 Kbps

50

User-Level Analysis• 0-500 ms latency, users could easily adjust• 800+ ms, game appeared erratic

– Degradation in gaming experience

• 500-800 ms degradation depended upon – User

• More skilled were more sensitive

– Strategy• Micro managers were more sensitive

• Combat managers were more sensitive

51

Summary of the Effect of Latency on RTS Games• Typical Internet latencies do not significantly affect user

performance in Warcraft III– Some effect on exploration– No statistical effect on building or combat

• RTS game play emphasizes “strategy” (which takes 10s of seconds or minutes), not “real-time”

• RTS games less sensitive to latency than are FPS– RTS in QoS class similar to that of Web browsing

• At the network level:– Small packets with low bandwidth– Command aggregation at higher latencies