Understanding the Performance of Understanding the Performance of Thin-Client GamingThin-Client Gaming

12011/5/11 CQR 2011 / Yu-Chun Chang

Yu-Chun Chang1, Po-Han Tseng2, Kuan-Ta Chen2, and Chin-Laung Lei1

1Department of Electrical Engineering, National Taiwan University2Institute of Information Science, Academia Sinica

OutlineOutline

• Introduction• Experiment methodology– Experiment setup– Performance metric extraction

• Performance evaluation• Conclusion & future work

2011/5/11 2CQR 2011 / Yu-Chun Chang

Introduction (1/2)Introduction (1/2)

3

Client Server

User’s inputs

Display updates

• Thin-client system

2011/5/11 CQR 2011 / Yu-Chun Chang

Introduction (2/2)Introduction (2/2)

• Motivation– To understand which performance metric is more

sufficient for thin-client gaming• Frame rate, frame delay, frame loss, and etc

• Challenges– Most thin-client products are proprietary

• Image compression, data-transmission protocol and display update mechanism

42011/5/11 CQR 2011 / Yu-Chun Chang

Our focusOur focus

52011/5/11

QoEPerf.Metric

NetworkCondition

Server

Client

Thin-client program

User

NetworkCondition

Server

CQR 2011 / Yu-Chun Chang

QoEPerf.Metric

OutlinesOutlines

• Introduction• Experiment methodology– Experiment setup– Performance metric extraction

• Performance evaluation• Conclusion & future work

2011/5/11 6CQR 2011 / Yu-Chun Chang

Experiment MethodologyExperiment Methodology

72011/5/11

Why Use Ms. Pac-Man?Why Use Ms. Pac-Man?

• Move Pac-Man to eat pills and get the score

• Control through thin-client applications and move Pac-Man in the game of server– Good network condition: score↑– Bad network condition: score↓

• Score Quality of Experience

2011/5/11 CQR 2011 / Yu-Chun Chang 8

Ms. Pac-Man & BotMs. Pac-Man & Bot

9

• Ms. Pac-Man– Save score after the pacman ran out of 3 lives

• Bot: ICE Pambush3 (published in IEEE CIG 2009)– Java-based controller to move the pacman– Capture the screen of the game and determine the position of the

pacman, ghosts, and pills

Number Score

Pill 220 10

Power pill 4 50

Ghost 4 200 (after eating power pills)

2011/5/11 CQR 2011 / Yu-Chun Chang

• Three thin-client systems– LogMeIn– UltraVNC– TeamViewer

• Network conditions

10

Network condition Settings

Network delay 0 ms, 100 ms, 200 ms

Network loss rate 0%, 2.5%, 5%

Bandwidth Unlimited, 600 kbps, 300 kbps

2011/5/11 CQR 2011 / Yu-Chun Chang

• Performance metric– Display frame rate– Frame distortion (MSE: Mean Square Error)

• Record game play as video files in 200 FPS

112011/5/11 CQR 2011 / Yu-Chun Chang

OutlinesOutlines

• Introduction• Experiment methodology– Experiment setup– Performance metric extraction

• Performance evaluation• Conclusion & future work

2011/5/11 12CQR 2011 / Yu-Chun Chang

132011/5/11

Thin Clients are Different!Thin Clients are Different!

CQR 2011 / Yu-Chun Chang

Visual Difference Really Matters!Visual Difference Really Matters!

142011/5/11 CQR 2011 / Yu-Chun Chang

Statistical RegressionStatistical Regression

15

RegressionModel

QoE(score)

Independent factors

Display frame rateFrame distortion

2011/5/11 CQR 2011 / Yu-Chun Chang

Frame-Based QoE ModelFrame-Based QoE Model

• Linear model

• QoE =

16

Adjusted R-squared: 0.72

2011/5/11 CQR 2011 / Yu-Chun Chang

Frame-Based QoE ModelFrame-Based QoE Model

17

Which Performance Metric is More Which Performance Metric is More Sufficient?Sufficient?

• QoE degradation– Optimal user’s QoE – user’s QoE predicted by model

• Frame rate is more sufficient!

2011/5/11 18

Frame Rate and Network ConditionsFrame Rate and Network Conditions

19

NetworkCondition

2011/5/11

QoEPerf.Metric

Server

Client

Thin-client program

User

CQR 2011 / Yu-Chun Chang

The Frame Rate Prediction ModelThe Frame Rate Prediction Model

• Frame rate =

• app1, app2: dummy variables– LogMeIn : app1 = 1, app2 = 0– TeamViewer : app1 = 0, app2 = 1– UltraVNC : app1 = 0, app2 = 0

• d: delay, l: loss rate, b: bandwidth

• dl, dt, du : delay of LogMeIn, delay of TeamViewer, delay of UltraVNC

2011/5/11 20CQR 2011 / Yu-Chun Chang

The Frame Rate Prediction ModelThe Frame Rate Prediction Model

2011/5/11 21

Adjusted R-squared: 0.85

CQR 2011 / Yu-Chun Chang

Delay of LogMeIn

Delay of UltraVNC

Bandwidth of LogMeIn

Bandwidth of UltraVNC

Predicted Frame RatePredicted Frame Rate

2011/5/11 22CQR 2011 / Yu-Chun Chang

Network delay Bandwidth

Which Thin-Client is Better?Which Thin-Client is Better?

23

NetworkConditions

2011/5/11

QoEPerf.Metric

Server

Client

Thin-client program

User

CQR 2011 / Yu-Chun Chang

Network-Based QoE ModelNetwork-Based QoE Model

• QoE =

2011/5/11Adjusted R-squared: 0.81

The Thin-Client with Best PerformanceThe Thin-Client with Best Performance

• o symbol: empirical network condition– 300 records collected by PingER project

2011/5/11 25CQR 2011 / Yu-Chun Chang

Conclusions & Future WorkConclusions & Future Work

• Display frame rate and frame distortion are both critical to gaming performance on thin-clients

• LogMeIn performs the best among the three implementations we studied

• Future work– Add more thin-clients to see comparisons of performance– Design a generalizable experiment methodology for thin-

client gaming with different game genres

2011/5/11 26CQR 2011 / Yu-Chun Chang

Thank you for your attention!

2011/5/11 CQR 2011 / Yu-Chun Chang 27