GamingAnywhere: An Open Cloud Gaming System · OnLive demands for 5 Mbps for reasonable quality •...
Transcript of GamingAnywhere: An Open Cloud Gaming System · OnLive demands for 5 Mbps for reasonable quality •...
![Page 1: GamingAnywhere: An Open Cloud Gaming System · OnLive demands for 5 Mbps for reasonable quality • OnLive dictates a backbone latency of 22 ms, and partially copes with it by setting](https://reader033.fdocuments.in/reader033/viewer/2022042320/5f0a43937e708231d42ace8e/html5/thumbnails/1.jpg)
GamingAnywhere: An Open-Source Cloud Gaming Testbed Chun-Ying Huang, De-Yu Chen, Cheng-Hsin Hsu, and Kuan-Ta Chen
ACM Multimedia 2013 OSS Competition, Barcelona, Spain
1
![Page 2: GamingAnywhere: An Open Cloud Gaming System · OnLive demands for 5 Mbps for reasonable quality • OnLive dictates a backbone latency of 22 ms, and partially copes with it by setting](https://reader033.fdocuments.in/reader033/viewer/2022042320/5f0a43937e708231d42ace8e/html5/thumbnails/2.jpg)
Cloud Gaming is Hot • Cloud gaming is expected to lead the future growth of
computer games: 9 times in 6 years [CGR]
[CGR] http://www.cgconfusa.com/report/documents/Content-5minCloudGamingReportHighlights.pdf
T5-Labs
2
![Page 3: GamingAnywhere: An Open Cloud Gaming System · OnLive demands for 5 Mbps for reasonable quality • OnLive dictates a backbone latency of 22 ms, and partially copes with it by setting](https://reader033.fdocuments.in/reader033/viewer/2022042320/5f0a43937e708231d42ace8e/html5/thumbnails/3.jpg)
What is Cloud Gaming
3
Real-time game playing using light-weight clients
![Page 4: GamingAnywhere: An Open Cloud Gaming System · OnLive demands for 5 Mbps for reasonable quality • OnLive dictates a backbone latency of 22 ms, and partially copes with it by setting](https://reader033.fdocuments.in/reader033/viewer/2022042320/5f0a43937e708231d42ace8e/html5/thumbnails/4.jpg)
Selling Points of Cloud Gaming • Gamers’ perspectives: • Frees gamers from indefinitely upgrading their computers
• Eliminates setup overhead and compatibility issues for trying out a game, as everything needed will be settled in data centers by game operators
• Enables gamers to play games anywhere, anytime
• Game manufacturers’ perspectives: • Allows developers to support more platforms
• Reduces the production cost
• Prevents pirating
4
![Page 5: GamingAnywhere: An Open Cloud Gaming System · OnLive demands for 5 Mbps for reasonable quality • OnLive dictates a backbone latency of 22 ms, and partially copes with it by setting](https://reader033.fdocuments.in/reader033/viewer/2022042320/5f0a43937e708231d42ace8e/html5/thumbnails/5.jpg)
Limitations of Existing Services • OnLive demands for 5 Mbps for reasonable quality • OnLive dictates a backbone latency of 22 ms, and partially
copes with it by setting up 3 data centers (CA, VA, TX) • Only people who live in 1000 mile radius from a data center can play
the game • and more…
• We, researchers, have tons of ideas to improve cloud gaming services, but all cloud gaming systems are proprietary and closed
5
![Page 6: GamingAnywhere: An Open Cloud Gaming System · OnLive demands for 5 Mbps for reasonable quality • OnLive dictates a backbone latency of 22 ms, and partially copes with it by setting](https://reader033.fdocuments.in/reader033/viewer/2022042320/5f0a43937e708231d42ace8e/html5/thumbnails/6.jpg)
Solution
• GamingAnywhere is the first cloud gaming platform for researchers, service providers, and users
6
![Page 7: GamingAnywhere: An Open Cloud Gaming System · OnLive demands for 5 Mbps for reasonable quality • OnLive dictates a backbone latency of 22 ms, and partially copes with it by setting](https://reader033.fdocuments.in/reader033/viewer/2022042320/5f0a43937e708231d42ace8e/html5/thumbnails/7.jpg)
Design Objectives • Extensibility
• Components can be implemented as modules • Features can be easily expanded
• Portability
• Cross-platform ready
• Configurability • Exposes as many configurations as possible
• Openness • It has been released to the public since April 2013
7
![Page 8: GamingAnywhere: An Open Cloud Gaming System · OnLive demands for 5 Mbps for reasonable quality • OnLive dictates a backbone latency of 22 ms, and partially copes with it by setting](https://reader033.fdocuments.in/reader033/viewer/2022042320/5f0a43937e708231d42ace8e/html5/thumbnails/8.jpg)
Cloud Gaming Scenario
8
![Page 9: GamingAnywhere: An Open Cloud Gaming System · OnLive demands for 5 Mbps for reasonable quality • OnLive dictates a backbone latency of 22 ms, and partially copes with it by setting](https://reader033.fdocuments.in/reader033/viewer/2022042320/5f0a43937e708231d42ace8e/html5/thumbnails/9.jpg)
System Architecture • The client and the server, with many components • Implemented by leveraging open-source packages
Game console
Data Flow
Control Flow
Running the selected game
Age
nt
Proc
ess/
Thre
ad
Game Server Game Client
Internet
Audio / VideoEncoder
RTSP / RTP / RTCP
Audio / VideoCapturerReplay User Inputs
(Keyboard, Mouse, ...)
Decode Input Events(Customized Protocol)
RTSP / RTP / RTCP
Audio / VideoDecoder
Encode Input Events(Customized Protocol)
Audio / VideoPlayer User Inputs
(Keyboard, Mouse, ...)
9
![Page 10: GamingAnywhere: An Open Cloud Gaming System · OnLive demands for 5 Mbps for reasonable quality • OnLive dictates a backbone latency of 22 ms, and partially copes with it by setting](https://reader033.fdocuments.in/reader033/viewer/2022042320/5f0a43937e708231d42ace8e/html5/thumbnails/10.jpg)
Implementation Overview • GamingAnaywhere core is implemented as a library
• With several modules and executable • Server components
• Video source (modularized, platform dependent) • Audio source (modularized, platform dependent) • Encoders (modularized, currently w/ ffmpeg) • Replayer (modularized, platform dependent) • RTSP/RTP server (in library, w/ ffmpeg)
• Client components • RTSP/RTP client (using live555) • Decoders (in library, w/ ffmpeg) • Controller (using SDL) • Renderer (using SDL)
10
![Page 11: GamingAnywhere: An Open Cloud Gaming System · OnLive demands for 5 Mbps for reasonable quality • OnLive dictates a backbone latency of 22 ms, and partially copes with it by setting](https://reader033.fdocuments.in/reader033/viewer/2022042320/5f0a43937e708231d42ace8e/html5/thumbnails/11.jpg)
License • GamingAnywhere adopts 3-clause BSD license
• Less restrictive than GPL
• Libraries used by GamingAnywhere
11
Library License Library License ffmpeg LGPL zlib zlib glibc LGPL lame LGPL libasound2 LGPL libvpx 3-clause BSD libSDL zlib opus 3-clause BSD libX11 LGPL x264 GPLv2 live555 LGPL libva MIT
![Page 12: GamingAnywhere: An Open Cloud Gaming System · OnLive demands for 5 Mbps for reasonable quality • OnLive dictates a backbone latency of 22 ms, and partially copes with it by setting](https://reader033.fdocuments.in/reader033/viewer/2022042320/5f0a43937e708231d42ace8e/html5/thumbnails/12.jpg)
GamingAnywhere Screenshots
12
Server
Client #1
Client #2
Client #3
![Page 13: GamingAnywhere: An Open Cloud Gaming System · OnLive demands for 5 Mbps for reasonable quality • OnLive dictates a backbone latency of 22 ms, and partially copes with it by setting](https://reader033.fdocuments.in/reader033/viewer/2022042320/5f0a43937e708231d42ace8e/html5/thumbnails/13.jpg)
Performance Evaluation • Compare against OnLive and StreamMyGame • Environment setup • Sample results are presented
14
![Page 14: GamingAnywhere: An Open Cloud Gaming System · OnLive demands for 5 Mbps for reasonable quality • OnLive dictates a backbone latency of 22 ms, and partially copes with it by setting](https://reader033.fdocuments.in/reader033/viewer/2022042320/5f0a43937e708231d42ace8e/html5/thumbnails/14.jpg)
Response Delay Measurement – For Closed Systems (OnLive and SMG)
0.1 sec 0.2 sec 0.3 sec
Screen Response delay
Response delay (RD) = Network delay (ND) + Processing delay (PD) + Playout delay (OD)
• Kuan-Ta Chen, Yu-Chun Chang, Po-Han Tseng, Chun-Ying Huang, and Chin-Laung Lei, “Measuring the Latency of Cloud Gaming Systems,” ACM Multimedia 2011
15
![Page 15: GamingAnywhere: An Open Cloud Gaming System · OnLive demands for 5 Mbps for reasonable quality • OnLive dictates a backbone latency of 22 ms, and partially copes with it by setting](https://reader033.fdocuments.in/reader033/viewer/2022042320/5f0a43937e708231d42ace8e/html5/thumbnails/15.jpg)
t0 (Key event sent) t1 (Key event received)
t2 (Frame sent) t3 (Frame received)
Server Client
Menu frame Menu screen shown
t4 (Frame displayed)
• Network delay (ND) : network RTT • Processing delay (PD) : t2 – t1 t3 – t0 – ND
• Playout delay (OD) : t4 – t3
16
![Page 16: GamingAnywhere: An Open Cloud Gaming System · OnLive demands for 5 Mbps for reasonable quality • OnLive dictates a backbone latency of 22 ms, and partially copes with it by setting](https://reader033.fdocuments.in/reader033/viewer/2022042320/5f0a43937e708231d42ace8e/html5/thumbnails/16.jpg)
GA Has The Lowest Response Delay
17
With GA: For a strict 100 ms RD requirement, ND can be as long as 52 ms
From Barcelona to Greenland (~4000 km)
![Page 17: GamingAnywhere: An Open Cloud Gaming System · OnLive demands for 5 Mbps for reasonable quality • OnLive dictates a backbone latency of 22 ms, and partially copes with it by setting](https://reader033.fdocuments.in/reader033/viewer/2022042320/5f0a43937e708231d42ace8e/html5/thumbnails/17.jpg)
18
http://gaminganywhere.org/
• In 6 months • Web: 16,103 visits, 9,689 unique visitors
• Forum: 53 topics, 178 posts
![Page 18: GamingAnywhere: An Open Cloud Gaming System · OnLive demands for 5 Mbps for reasonable quality • OnLive dictates a backbone latency of 22 ms, and partially copes with it by setting](https://reader033.fdocuments.in/reader033/viewer/2022042320/5f0a43937e708231d42ace8e/html5/thumbnails/18.jpg)
Visitor Distribution
19
![Page 19: GamingAnywhere: An Open Cloud Gaming System · OnLive demands for 5 Mbps for reasonable quality • OnLive dictates a backbone latency of 22 ms, and partially copes with it by setting](https://reader033.fdocuments.in/reader033/viewer/2022042320/5f0a43937e708231d42ace8e/html5/thumbnails/19.jpg)
Conclusion • GamingAnywhere is the first open cloud gaming platform
• May be used by researchers, engineers, and gamers
• We hope it will stimulate more studies on cloud gaming,
• Codecs for cloud gaming • Usability study for cloud gaming interfaces • Cloud gaming in heterogeneous networks • 3D/Stereoscopic cloud gaming • … and many more!
• We need your participation!
20
![Page 20: GamingAnywhere: An Open Cloud Gaming System · OnLive demands for 5 Mbps for reasonable quality • OnLive dictates a backbone latency of 22 ms, and partially copes with it by setting](https://reader033.fdocuments.in/reader033/viewer/2022042320/5f0a43937e708231d42ace8e/html5/thumbnails/20.jpg)
QUESTIONS? Join us at http://gaminganwhere.org
21
![Page 21: GamingAnywhere: An Open Cloud Gaming System · OnLive demands for 5 Mbps for reasonable quality • OnLive dictates a backbone latency of 22 ms, and partially copes with it by setting](https://reader033.fdocuments.in/reader033/viewer/2022042320/5f0a43937e708231d42ace8e/html5/thumbnails/21.jpg)
BACKUP
22
![Page 22: GamingAnywhere: An Open Cloud Gaming System · OnLive demands for 5 Mbps for reasonable quality • OnLive dictates a backbone latency of 22 ms, and partially copes with it by setting](https://reader033.fdocuments.in/reader033/viewer/2022042320/5f0a43937e708231d42ace8e/html5/thumbnails/22.jpg)
Server Overview
23
Audiosource
Desktop/Game
Videosource
Videoencoder
Audioencoder
Audiobuffer
Videobuffer
(1a)audio capture
(1v)video
capture
Threads
Shared buffers
(2a)write audio
frames
(2v)write a video frame
(3a)wake upencoder
(3v)wake upencoder
(4a)read audio
frames
(4v)read a video frame
(5a)encode andsend
(5v)encode and
send
Object owner
RTSP server thread
Data Flow Connections (RTSP/RTP/RTCP)
(1n)handleclients
Inputreplayer
Control Flow Connections
(2i)replayinputevents
(1i)receiveinputevents
![Page 23: GamingAnywhere: An Open Cloud Gaming System · OnLive demands for 5 Mbps for reasonable quality • OnLive dictates a backbone latency of 22 ms, and partially copes with it by setting](https://reader033.fdocuments.in/reader033/viewer/2022042320/5f0a43937e708231d42ace8e/html5/thumbnails/23.jpg)
Process Video Frames w/ a Shared Buffer • A shared buffer for a single screenshot
• The writer could be blocked by readers
24
![Page 24: GamingAnywhere: An Open Cloud Gaming System · OnLive demands for 5 Mbps for reasonable quality • OnLive dictates a backbone latency of 22 ms, and partially copes with it by setting](https://reader033.fdocuments.in/reader033/viewer/2022042320/5f0a43937e708231d42ace8e/html5/thumbnails/24.jpg)
Process Video Frames • Process a video frame
1. Video source: Capture a frame 2. Converter: Convert from RGB format to YUV format 3. Encoder: encoding and packetization
25
![Page 25: GamingAnywhere: An Open Cloud Gaming System · OnLive demands for 5 Mbps for reasonable quality • OnLive dictates a backbone latency of 22 ms, and partially copes with it by setting](https://reader033.fdocuments.in/reader033/viewer/2022042320/5f0a43937e708231d42ace8e/html5/thumbnails/25.jpg)
Process Video Frames in Parallel • Suppose the targeted inter-frame delay is ∆t • The response delay may greater than ∆t
• frame capture + color space conversion + encoding
• It could degrade encoding bitrate • Process in parallel
26
F1 F2
F1
F1
F2
F2
F3
F3
F3
Video source(frame capture)
t
Video encoder
F4
Color space converter(RGB to YUV)
t+∆ t t+2∆ t t+3∆ t
![Page 26: GamingAnywhere: An Open Cloud Gaming System · OnLive demands for 5 Mbps for reasonable quality • OnLive dictates a backbone latency of 22 ms, and partially copes with it by setting](https://reader033.fdocuments.in/reader033/viewer/2022042320/5f0a43937e708231d42ace8e/html5/thumbnails/26.jpg)
Process Audio Frames • Each audio encoder has an audio source buffer
• A queue for captured audio frames
27
![Page 27: GamingAnywhere: An Open Cloud Gaming System · OnLive demands for 5 Mbps for reasonable quality • OnLive dictates a backbone latency of 22 ms, and partially copes with it by setting](https://reader033.fdocuments.in/reader033/viewer/2022042320/5f0a43937e708231d42ace8e/html5/thumbnails/27.jpg)
Client Overview
28
Game Interaction
Mainthread
Control Flow Connections
(1i)receiveinputevents
(2i)sendinputevents
SDL Rendering Input Events
Data Flow Connections
RTSP
clie
nt th
read
Videobuffer
Audiobuffer
Threads
Buffers
Object owner
(1r)receiveencodedA/Vframes
(2rv)buffer
an encoded
audio frames
(2ra)bufferencodedaudio frames
(3rv)decode and render
video frames
(3ra)decode and renderaudio frames(callback)
![Page 28: GamingAnywhere: An Open Cloud Gaming System · OnLive demands for 5 Mbps for reasonable quality • OnLive dictates a backbone latency of 22 ms, and partially copes with it by setting](https://reader033.fdocuments.in/reader033/viewer/2022042320/5f0a43937e708231d42ace8e/html5/thumbnails/28.jpg)
Video Playout Buffering • The 1-frame buffering strategy
• Based on the RTP marker bit • An H.264 frame can be split into different numbers of packets • The marker bit (with a value of 1) indicates the last packet of a
frame • An example: 30KB frame transmitted in a 50Mbps network would have
a buffering time of 30KB*8-bit/50Mbps =~ 5ms
29
M M M M M
RTP Packets
Frame #1 Frame #2 Frame #3 Frame #4 Frame #5
time
![Page 29: GamingAnywhere: An Open Cloud Gaming System · OnLive demands for 5 Mbps for reasonable quality • OnLive dictates a backbone latency of 22 ms, and partially copes with it by setting](https://reader033.fdocuments.in/reader033/viewer/2022042320/5f0a43937e708231d42ace8e/html5/thumbnails/29.jpg)
Performance Evaluation (Cont’d) • Audio: The LAME encoder • Video: The x264 encoder
• One of the VideoLan projects
• With the below default parameters • $r is the bitrate variable, default 3Mbps
--profile main --preset faster --tune zerolatency --bitrate $r --ref 1 --me dia --merange 16 --intra-refresh --keyint 48 --sliced-threads --slices 4 --threads 4 --input-res 1280x720
30
![Page 30: GamingAnywhere: An Open Cloud Gaming System · OnLive demands for 5 Mbps for reasonable quality • OnLive dictates a backbone latency of 22 ms, and partially copes with it by setting](https://reader033.fdocuments.in/reader033/viewer/2022042320/5f0a43937e708231d42ace8e/html5/thumbnails/30.jpg)
Delay Decomposition of GA • Processing delay
• Memory lock + copy • RGB to YUV • x264 encoder • Packetization (RTP packets)
• Playout delay • 1-frame buffering • S/W decoder • SDL rendering
31
![Page 31: GamingAnywhere: An Open Cloud Gaming System · OnLive demands for 5 Mbps for reasonable quality • OnLive dictates a backbone latency of 22 ms, and partially copes with it by setting](https://reader033.fdocuments.in/reader033/viewer/2022042320/5f0a43937e708231d42ace8e/html5/thumbnails/31.jpg)
GA Incurs Low to Moderate Network Loads
• Configured to capture at 50 fps with a bitrate of 3 Mbps
32
![Page 32: GamingAnywhere: An Open Cloud Gaming System · OnLive demands for 5 Mbps for reasonable quality • OnLive dictates a backbone latency of 22 ms, and partially copes with it by setting](https://reader033.fdocuments.in/reader033/viewer/2022042320/5f0a43937e708231d42ace8e/html5/thumbnails/32.jpg)
GA Provides Superior Video Quality
33