Understanding the Performance of Understanding the Performance of Thin-Client GamingThin-Client Gaming
12011/5/11 CQR 2011 / Yu-Chun Chang
Yu-Chun Chang1, Po-Han Tseng2, Kuan-Ta Chen2, and Chin-Laung Lei1
1Department of Electrical Engineering, National Taiwan University2Institute of Information Science, Academia Sinica
OutlineOutline
• Introduction• Experiment methodology– Experiment setup– Performance metric extraction
• Performance evaluation• Conclusion & future work
2011/5/11 2CQR 2011 / Yu-Chun Chang
Introduction (1/2)Introduction (1/2)
3
Client Server
User’s inputs
Display updates
• Thin-client system
2011/5/11 CQR 2011 / Yu-Chun Chang
Introduction (2/2)Introduction (2/2)
• Motivation– To understand which performance metric is more
sufficient for thin-client gaming• Frame rate, frame delay, frame loss, and etc
• Challenges– Most thin-client products are proprietary
• Image compression, data-transmission protocol and display update mechanism
42011/5/11 CQR 2011 / Yu-Chun Chang
Our focusOur focus
52011/5/11
QoEPerf.Metric
NetworkCondition
Server
Client
Thin-client program
User
NetworkCondition
Server
CQR 2011 / Yu-Chun Chang
QoEPerf.Metric
OutlinesOutlines
• Introduction• Experiment methodology– Experiment setup– Performance metric extraction
• Performance evaluation• Conclusion & future work
2011/5/11 6CQR 2011 / Yu-Chun Chang
Experiment MethodologyExperiment Methodology
72011/5/11
Why Use Ms. Pac-Man?Why Use Ms. Pac-Man?
• Move Pac-Man to eat pills and get the score
• Control through thin-client applications and move Pac-Man in the game of server– Good network condition: score↑– Bad network condition: score↓
• Score Quality of Experience
2011/5/11 CQR 2011 / Yu-Chun Chang 8
Ms. Pac-Man & BotMs. Pac-Man & Bot
9
• Ms. Pac-Man– Save score after the pacman ran out of 3 lives
• Bot: ICE Pambush3 (published in IEEE CIG 2009)– Java-based controller to move the pacman– Capture the screen of the game and determine the position of the
pacman, ghosts, and pills
Number Score
Pill 220 10
Power pill 4 50
Ghost 4 200 (after eating power pills)
2011/5/11 CQR 2011 / Yu-Chun Chang
• Three thin-client systems– LogMeIn– UltraVNC– TeamViewer
• Network conditions
10
Network condition Settings
Network delay 0 ms, 100 ms, 200 ms
Network loss rate 0%, 2.5%, 5%
Bandwidth Unlimited, 600 kbps, 300 kbps
2011/5/11 CQR 2011 / Yu-Chun Chang
• Performance metric– Display frame rate– Frame distortion (MSE: Mean Square Error)
• Record game play as video files in 200 FPS
112011/5/11 CQR 2011 / Yu-Chun Chang
OutlinesOutlines
• Introduction• Experiment methodology– Experiment setup– Performance metric extraction
• Performance evaluation• Conclusion & future work
2011/5/11 12CQR 2011 / Yu-Chun Chang
132011/5/11
Thin Clients are Different!Thin Clients are Different!
CQR 2011 / Yu-Chun Chang
Visual Difference Really Matters!Visual Difference Really Matters!
142011/5/11 CQR 2011 / Yu-Chun Chang
Statistical RegressionStatistical Regression
15
RegressionModel
QoE(score)
Independent factors
Display frame rateFrame distortion
2011/5/11 CQR 2011 / Yu-Chun Chang
Frame-Based QoE ModelFrame-Based QoE Model
• Linear model
• QoE =
16
Adjusted R-squared: 0.72
2011/5/11 CQR 2011 / Yu-Chun Chang
Frame-Based QoE ModelFrame-Based QoE Model
17
Which Performance Metric is More Which Performance Metric is More Sufficient?Sufficient?
• QoE degradation– Optimal user’s QoE – user’s QoE predicted by model
• Frame rate is more sufficient!
2011/5/11 18
Frame Rate and Network ConditionsFrame Rate and Network Conditions
19
NetworkCondition
2011/5/11
QoEPerf.Metric
Server
Client
Thin-client program
User
CQR 2011 / Yu-Chun Chang
The Frame Rate Prediction ModelThe Frame Rate Prediction Model
• Frame rate =
• app1, app2: dummy variables– LogMeIn : app1 = 1, app2 = 0– TeamViewer : app1 = 0, app2 = 1– UltraVNC : app1 = 0, app2 = 0
• d: delay, l: loss rate, b: bandwidth
• dl, dt, du : delay of LogMeIn, delay of TeamViewer, delay of UltraVNC
2011/5/11 20CQR 2011 / Yu-Chun Chang
The Frame Rate Prediction ModelThe Frame Rate Prediction Model
2011/5/11 21
Adjusted R-squared: 0.85
CQR 2011 / Yu-Chun Chang
Delay of LogMeIn
Delay of UltraVNC
Bandwidth of LogMeIn
Bandwidth of UltraVNC
Predicted Frame RatePredicted Frame Rate
2011/5/11 22CQR 2011 / Yu-Chun Chang
Network delay Bandwidth
Which Thin-Client is Better?Which Thin-Client is Better?
23
NetworkConditions
2011/5/11
QoEPerf.Metric
Server
Client
Thin-client program
User
CQR 2011 / Yu-Chun Chang
Network-Based QoE ModelNetwork-Based QoE Model
• QoE =
2011/5/11Adjusted R-squared: 0.81
The Thin-Client with Best PerformanceThe Thin-Client with Best Performance
• o symbol: empirical network condition– 300 records collected by PingER project
2011/5/11 25CQR 2011 / Yu-Chun Chang
Conclusions & Future WorkConclusions & Future Work
• Display frame rate and frame distortion are both critical to gaming performance on thin-clients
• LogMeIn performs the best among the three implementations we studied
• Future work– Add more thin-clients to see comparisons of performance– Design a generalizable experiment methodology for thin-
client gaming with different game genres
2011/5/11 26CQR 2011 / Yu-Chun Chang
Thank you for your attention!
2011/5/11 CQR 2011 / Yu-Chun Chang 27
Top Related