Open Standards for Gaming
-
Upload
the-khronos-group-inc -
Category
Technology
-
view
516 -
download
1
description
Transcript of Open Standards for Gaming
© Copyright Khronos Group 2013 - Page 1
Open Standards for Today’s Gaming Industry
Erik Noreke VP of Business Development, Khronos Group
© Copyright Khronos Group 2013 - Page 3
Gaming Evolution
New high performance games
continue to evolve on high-end platforms
Mobile is the new platform for game innovation.
Mobile APIs unlock hardware acceleration and
conserve battery life, making portable gaming
possible
Diverse platforms – mobile, TV, embedded
– means HTML5 will become
increasingly important as a universal
gaming platform
DESKTOP MOBILE
© Copyright Khronos Group 2013 - Page 4
Courtesy of PC Gaming Alliance
www.pcgamingalliance.org
© Copyright Khronos Group 2013 - Page 5
Courtesy of PC Gaming Alliance
www.pcgamingalliance.org
© Copyright Khronos Group 2013 - Page 6
Courtesy of PC Gaming Alliance
www.pcgamingalliance.org
US$28.7B gaming revenues generated on platforms supporting
Khronos native graphics and compute APIs
© Copyright Khronos Group 2013 - Page 7
OpenGL 3D API Family Tree
OpenGL ES 1.0
OpenGL ES 1.1 OpenGL ES 2.0 OpenGL ES 3.0
2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013
OpenGL 1.5 OpenGL 2.0 OpenGL 4.3 OpenGL 2.1
OpenGL 3.0
OpenGL 3.1
OpenGL 3.2
OpenGL 3.3
OpenGL 4.0
OpenGL 4.1
OpenGL 4.2
2002
OpenGL 1.3
ES-Next
GL-Next
OpenGL ES 2.0
Content OpenGL ES 1.1
Content
OpenGL ES 3.0
Content
ES3 is backward compatible
so new features can be
added incrementally Fixed function
3D Pipeline
Programmable vertex
and fragment shaders
WebGL 1.0
OpenGL 4.4 is a
superset of DX11
WebGL-Next
Desktop 3D
Mobile 3D
OpenGL 4.4
© Copyright Khronos Group 2013 - Page 8
Mobile OpenCL Shipping • Android ICD extension released in latest extension specification
- OpenCL implementations can be discovered and loaded as a shared object
• Multiple implementations shipping in Android NDK
- ARM, Imagination, Vivante, Qualcomm, Samsung …
© Copyright Khronos Group 2013 - Page 9
Active Gamers by Platform
Courtesy of PC Gaming Alliance
www.pcgamingalliance.org
CAMERA
© Copyright Khronos Group 2013 - Page 10
Sensors & Cameras • Single sensor RGB cameras are just the start of the mobile visual revolution
- IR sensors – LEAP Motion, eye-trackers
• Multi-sensors: Stereo pairs -> Plenoptic array -> Depth cameras
- Stereo pair can enable object scaling and enhanced depth extraction
• Hybrid visual sensing solutions
- Different sensors mixed for different distances and lighting conditions
Dual Camera LG Electronics
Plenoptic Array Pelican imaging
Capri Structured Light 3D Camera PrimeSense
© Copyright Khronos Group 2013 - Page 11
Typical Imaging Pipeline • Pre- and Post-processing can be done on CPU, GPU, DSP…
• ISP controls camera via 3A algorithms
Auto Exposure (AE), Auto White Balance (AWB), Auto Focus (AF)
• ISP may be a separate chip or within Application Processor
Pre-processing Image Signal Processor
(ISP)
Post-
processing
CMOS sensor
Color Filter Array
Lens
Bayer RGB/YUV
App
Lens, sensor, aperture control 3A
Need for advanced
camera control API!
CAMERA
© Copyright Khronos Group 2013 - Page 12
Low Power Environment Scanning • Many sensor use cases would consume too much power to be running 24/7
- Environment aware use cases have to be very low power
• ‘Scanners’ - very low power, always on, detect things in the environment
- Trigger the next level of processing capability
ARM 7 1 MIP and accelerometers can
detect someone in the vicinity
DSP Low power activation of camera
to detect someone in field of view
GPU GPU acceleration for precision
gesture processing
© Copyright Khronos Group 2013 - Page 13
StreamInput Sensor Fusion Stack
OS Sensor OS APIs (E.g. Android SensorManager or
iOS CoreMotion)
Low-level native API defines access to
fused sensor data stream and context-awareness
…
Applications
Sensor Sensor
Sensor
Hub Sensor
Hub
StreamInput implementations
compete on sensor stream quality,
reduced power consumption,
environment triggering and context
detection – enabling sensor
subsystem vendors to increased
ADDED VALUE
Middleware (E.g. Augmented Reality engines,
gaming engines)
Platforms can provide
increased access to
improved sensor data stream
– driving faster, deeper
sensor usage by applications
Middleware engines need platform-
portable access to native, low-level
sensor data stream
Mobile or embedded
platforms without sensor
fusion APIs can provide
direct application access
to StreamInput
Hardware transport
interfaces are defined
by each system, e.g.
IIO or HID sensor
© Copyright Khronos Group 2013 - Page 14
Worldwide Game Users – 2012 (Active and Casual)
0
100
200
300
400
500
N.America Europe Asia Other
WORLDWIDE GAME USERS BY REGION AND PLATFORM (IN MILLIONS)
Console PC Dedicated Handheld Mobile
Courtesy of PC Gaming Alliance
www.pcgamingalliance.org
© Copyright Khronos Group 2013 - Page 15
C/C++
SDK Dalvik (Java)
Objective C C#
DirectX
HTML/CSS HTML/CSS HTML/CSS
Cross-OS, Cross Platform Portability
HTML5 provides cross
platform portability. GPU
accessibility through
WebGL available soon on
~90% mobile systems
Preferred development
environments not
designed for portability
Native code is portable-
but apps must cope with
different available APIs
and libraries
© Copyright Khronos Group 2013 - Page 16
WebGL Availability in Browsers
- Microsoft – “where you have IE11, you have WebGL – turned on by default and working all the time” - Microsoft - WebGL also enabled for Windows applications - web app framework and web view - Apple - WebGL must be explicitly turned on MAC Safari and only exposed on iOS for iAds - Chrome OS - WebGL is the only cross-platform API to program the GPU - Google IO announcement - Chrome on Android will soon launch with WebGL
Much WebGL content uses three.js library:
http://threejs.org/
© Copyright Khronos Group 2013 - Page 17
WebCL – Parallel Computing for the Web • JavaScript bindings to hardware acceleration
- Giving performance gaming in a web browser
http://www.youtube.com/user/SamsungSISA#p/a/u/1/9Ttux1A-Nuc
© Copyright Khronos Group 2013 - Page 18 Remember Me image courtesy of CapCom
© Copyright Khronos Group 2013 - Page 19
3D Needs a Transmission Format! • Compression and streaming of 3D assets becoming essential
- Mobile and connected devices need access to increasingly large asset databases
• 3D is the last media type to define a compressed format
- 3D is more complex – diverse asset types and use cases
• Needs to be royalty-free
- Avoid an ‘internet video codec war’ scenario
• Eventually enable hardware implementations of successful codecs
- High-performance and low power – but pragmatic adoption strategy is key
Audio Video Images 3D
MP3 H.264 JPEG ? !
An effective and widely adopted codec ignites previously
unimagined opportunities for a media type
© Copyright Khronos Group 2013 - Page 20
glTF Goals • Binary file format for efficient transmission for 3D assets
- Reduce network bandwidth and minimize client processing overhead
• Run-time neutral - DO NOT IMPLY OR MANDATE ANY RUN-TIME BEHAVIOR
- Can be used by any app or run-time – usually WebGL accelerated
• Scalable to handle compression and streaming
- Though baseline format does not include compression
• ‘Direct load efficiency’ for WebGL
- Little or NO processing to drop glTF data into WebGL client
• Carry conditioned data from any authoring format
- Prototyping and optimizing efficient handling of COLLADA assets
A standards-based
content pipeline for
rich native and Web 3D
applications Playback Authoring
© Copyright Khronos Group 2013 - Page 21
Game On! • Gaming and Animation is a complex and expanding application domain and
coherent multiple standards across multiple domains are needed to enable the
market
• Khronos family of standards allow you to focus your efforts on game
development, not porting between platforms
• Now is a great time to get involved with the standards initiatives
that effect your business and game development
• www.khronos.org
© Copyright Khronos Group 2013 - Page 22
© Copyright Khronos Group 2013 - Page 23
WebGL First Wave Application Categories • Maps and Navigation
• Modeling Tools and Repositories
• Games
• 3D Printing
• Visualization
• Music Videos and Promotion
• Education
• Photo Editors
• Music Visualizers
• Vision/Video Processing
© Copyright Khronos Group 2013 - Page 24
COLLADA and glTF Open Source Ecosystem
Tool Interop
Three.js glTF Importer. Rest3D initiative
COLLADA2GLTF
Translator
OpenCOLLADA
Importer/Exporter
and COLLADA
Conformance Tests
On GitHUB
Pervasive WebGL deployment
Other
authoring
formats
Web-based Tools
https://github.com/KhronosGroup/glTF
https://github.com/KhronosGroup/OpenCOLLADA
https://github.com/KhronosGroup/COLLADA-CTS
© Copyright Khronos Group 2013 - Page 25
Google Maps • All rendering (2D and 3D) in Google Maps uses WebGL
© Copyright Khronos Group 2013 - Page 26
WebCL – Parallel Computing for the Web • JavaScript bindings to OpenCL APIs
- Enables initiation of Kernels written in OpenCL C within the browser
• Bindings stay close to the OpenCL standard
- Maximum flexibility to provide a foundation for higher-level middleware
- Minimal language modifications for 100% security and app portability - E.g. Mapping of CL memory objects into host memory space is not supported
• API definition underway – public draft released - https://cvs.khronos.org/svn/repos/registry/trunk/public/webcl/spec/latest/index.html
• Compelling use cases
- Physics engines for WebGL games
- Image and video editing in browser River Trail
JavaScript binding to OpenCL APIs to enable initiation of OpenCL C kernels
Data-parallel language extensions to JavaScript
for offload at run-time
© Copyright Khronos Group 2013 - Page 27
Leveraging Proven Native APIs into HTML5 • Khronos and W3C liaison
- Leverage proven native API investments into the Web
- Fast API development and deployment
- Designed by the hardware community
- Familiar foundation reduces developer learning curve
Native APIs shipping
or Khronos working group
JavaScript API shipping,
acceleration being developed
or work underway
WebVX? Vision
Processing
WebCAM(!) Camera
control and
video
processing
Possible future
JavaScript APIs or
acceleration
WebStream? Sensor Fusion
Native
JavaScript Canvas
Path Rendering
Camera
Control
HTML
© Copyright Khronos Group 2013 - Page 32
OpenGL ES 3.0 Highlights • Better looking, faster performing games and apps – at lower power
- Incorporates proven features from OpenGL 3.3 / 4.x
- 32-bit integers and floats in shader programs
- NPOT, 3D textures, depth textures, texture arrays
- Multiple Render Targets for deferred rendering, Occlusion Queries
- Instanced Rendering, Transform Feedback …
• Make life better for the programmer
- Tighter requirements for supported features to reduce implementation variability
• Backward compatible with OpenGL ES 2.0
- OpenGL ES 2.0 apps continue to run unmodified
• Standardized Texture Compression
- #1 developer request!
© Copyright Khronos Group 2013 - Page 33
DirectX 11.1
2004 2006 2008 2009 2010 2005 2007 2011
Accelerating OpenGL Innovation
DirectX 10.1
OpenGL 2.0 OpenGL 2.1 OpenGL 3.0
OpenGL 3.1
DirectX 9.0c DirectX 10.0 DirectX 11
OpenGL 3.2
OpenGL 3.3/4.0
OpenGL 4.1
Bringing state-of-the-art functionality to cross-platform graphics
2012
OpenGL 4.2
OpenGL 4.4
2013
OpenGL 4.3
© Copyright Khronos Group 2013 - Page 34
OpenGL 4.3 Compute Shaders • Execute algorithmically general-purpose GLSL shaders
- Can operate on uniforms, images and textures
• Process graphics data in the context of the graphics pipeline
- Easier than interoperating with a compute API IF processing ‘close to the pixel’
• Standard part of all OpenGL 4.3 implementations
- Matches DX11 DirectCompute functionality
Physics AI Simulation Ray Tracing Imaging Global Illumination
© Copyright Khronos Group 2013 - Page 35
OpenGL Conformance Test Suite released!
Conformance submissions are required for GL 4.4 implementations encouraged for earlier driver versions
Shared codebase with OpenGL ES 3.0 CTS additional desktop-specific tests
Core profile functionality
Enhancements underway to add more coverage
© Copyright Khronos Group 2013 - Page 36
OpenCL and OpenGL Compute Shaders • OpenGL compute shaders and OpenCL support distinctly different use cases
- OpenCL provides a significantly more powerful and complete compute solution
Enhanced 3D
Graphics apps
“Shaders++”
Pure compute
apps touching
no pixels
Compute Shaders
1. Full ANSI C programming of
heterogeneous CPUs and GPUs
2. Utilize multiple processors
3. Precisely defined IEEE accuracy
1. Fine grain compute operations
inside OpenGL
2. GLSL Shading Language
3. Execute on single GPU only
Imaging
Video
Physics
AI
© Copyright Khronos Group 2013 - Page 37
Gaming Evolution
New high performance games still evolve on high-
end platforms
Mobile is the new platform for game innovation.
Mobile APIs unlock hardware acceleration and
conserve battery life, making portable gaming
possible
Diverse platforms – mobile, TV, embedded
– means HTML5 will become
increasingly important as a universal
gaming platform
DESKTOP MOBILE