Programming of Graphics

GPU overviewGraphics and Game Engines

Peter Mileff PhD

University of MiskolcDepartment of Information Technology

2

Overview of the GPU...

GPU Overview⦿ Graphics Processing Unit (GPU) is the central unit

of your graphics card⦿ Its objective:

● Performing complex graphical operations● Directly accelerate the visualization● Offload the CPU:

○ taking high-level visualization tasks from the CPU○ therefore CPU can be used to do other things

⦿ The reason of the spread of the GPUs:● Hardware manufacturers quickly recognized the business

opportunities. Creating:○ Multimedia applications (e.g. Photoshop)○ Engineering systems (e.g. CAD systems)○ Games

3

⦿ In 1996, 3dfx company released Voodoo I⦿ Voodoo I characteristics:

● The first 3D accelerator card (4MB RAM, 50 Mhz)● Huge success● Support only the 3D visualization

○ It required an additional 2D video card

⦿ The idea: ● The 2D transformations are performed by a fast 2D video card

○ E.g. the popular Matrox video card● The 3D transformations are performed by the Voodoo card

○ its hardware were able to make faster calculations than software rendering.

First Achievements

4

Other important events

⦿ In the same year: ● NVIDIA and the ATI started their own GPU series● Nvidia: NV1, RIVA 128, Geforce 256● ATI: 3D Rage, Rage Pro, Rage 128

⦿ The video cards immediately became very popular⦿ The reasons of this are:

● Reasonable price● These cards could be buy in every computer shop● Cards were supported by games and operating systems (mainly

by windows)

5

6

Today main (GPU) trades

7

Architecture of the GPU…

CPU vs GPU⦿ The GPU architecture is very different from the CPU

(already from the very beginning!)⦿ Reason 1:

● They are designed for specific purposes: typically to speed up graphical calculations

● Graphical calculations have different requirements than the needs against the CPU

● The CPU is for general purposes⦿ Reason 2:

● Graphical calculations and the process of the rasterization can be heavily parallelized⦿ The development of the GPUs started to this direction

8

CPU vs GPU⦿ CPU: implements a single-threaded computing

architecture● allows to run multiple processes on a single threaded pipeline● application data can be reached through one memory interface

⦿ GPU: the architecture follows the stream processing technology● This is much more efficient approach to process large amount

of data● A GPU can contain even thousands of stream processors● There are no conflicts and the wait like at the CPU

○ Stream processors form a pipeline

9

CPU vs GPU

10

Geforce 8800

11

Geforce GTX 280

12

CPU vs GPU⦿ CPU: uses a lot of resources to

● the control of the programs, ● to switch between instructions and tasks

⦿ GPU: is totally unsuitable for this● GPU contains a lot of arithmetic logic units (ALU),

○ has the ability to calculate faster with order of magnitude● Limitations:

○ every processing unit should run the same command – Data parallelism!

⦿ CPU also supports data parallelism! ● with extended instruction sets (e.g.. SSE, SSE2, SSE3, SSE4,

AVX, ALTIVEC, stb), ● with multicore CPUs

13

The problem of data transfer⦿ There is a distance between the GPU and CPU

● they are connected through the system bus⦿ The data transfer problem appeared soon!

● Transfer data from main memory to the GPU memory is time consulting

14

The problem of data transfer

⦿ For this reason, numerous bus types were developed● Former standards: ISA, MCA, VLB, PCI● In 1997, the AGP (Accelerated Graphics Port)

standard was developed⦿ Very fast data transfer between CPu and GPU⦿ Today is still present in the AGP standard

⦿ Today, the dominant solution is the PCI Express standard● a high-speed serial computer expansion bus standard

15

PCIe 1.0 PCIe 2.0 PCIe 3.0 PCIe 4.0250 MB/s 500 MB/s 984,6 MB/s 1969.2 MB/s

Tendency of evolution⦿ The GPUs evolution far exceeds the development of CPUs

⦿ Moore's law (1965):

● is the observation that the number of transistors in a dense integrated circuit doubles approximately every two years.

⦿ Today:● CPU: the speed slowed to 18 months● GPU: doubling rate reduced to 6 months

16

Tendency of evolution⦿ Example:

ATI Radeon HD 3800 GPU family:

● 320 stream processor● 666 million transistors● Performance > 1 terraFLOPS

Intel Core 2 Quad CPU ● 582 million transistors ● Performance ~ 9.8 gigaFLOPS

17

Tendency

18

Tendency

19

20

Programming the GPU…

Programming APIs⦿ In parallel with the development of video cards

numerous low-level programming interfaces (API) were developed● Under strong influence of hardware vendors

⦿ First well known API: Glide API● Developed by 3dfx for their own Voodoo cards● OpenGL like interface● Targeted games in terms of performance and functionality● It was dominant in game industry until mid-1990s● In 2000, Nvidia acquired 3dfx

21

Direct3D vs OpenGL⦿ Direct3D

● Part of the Microsoft’s DirectX graphical API● Available only for Windows platforms

○ Desktops, XBox, Windows Phone● The most popular graphical APIs for game developers

⦿ The reason of its popularity:● development is perfectly follows the evolution of graphics

hardwares● Provides also built in higher level solutions:

○ Optimized mathematical solutions. E.g. matrices, vectors, collision detection, etc

○ Own 3D bone animation based model format called X● Other additional higher-level APIs: DirectDraw, DirectInput,

DirectSound, etc22

Direct3D vs OpenGL⦿ OpenGL (Open Graphics Library):

● Specification standard for platform independent 2D és 3D visualization

● Introduced in 1992 by Silicon Graphics Inc● The ARB (Architecture Review Board) consortium was responsible for

its development○ Members are the major software and hardware manufacturers:○ ATI, NVIDIA, Intel, Microsoft, etc.)

● In 2006 Khronos Group consortium took over its development○ https://www.khronos.org/

● Slower development: the development of the specification is a slow process, which significantly hinders the graphics-intensive applications developers.

23

Direct3D vs OpenGL⦿ Real competitors in the field of game development⦿ Both API has its own advantages and drawbacks

● Mainly there are only structural differences, the two APIs are almost identical in functionality

⦿ Advantage of the OpenGL (the future):● Platform independence: opengl has the opportunity to run on almost all

devices● OpenGL can also be used for embedded systems and mobile devices.

○ This version is called OpenGL ES● Popular operating systems are using OpenGL

○ iOS - OpenGL ES○ Linux, Unix, BSD - OpenGL○ Playstation - OpenGL○ AmigaOS, MorphOS, Haiku OS○ etc

24

25

Game and Graphics Engines…

Game engines⦿ Objective 1: to provide a toolkit for the developers team

(developer, designer, tester),● E.g.: editors,runtime environment, network, audio

⦿ Efficient, convenient and fast game development becomes possible

⦿ It is a layer between the Operating System and the game logic.

⦿ It simplifies the routine programming tasks:● Otherwise these should be performed for all games● E.g: creating a window, audio, play video, loading assets, collision

detection, etc.⦿ Objective 2: representing an appropriate technical quality

● in terms graphics quality and performance

26

Structure of a Game Engine

⦿ The process of game development requires a complex IT knowledge!● The game engine supports these process and therefore it’s

functionality should be also complex

They are organized into well-defined subsystems:

⦿ Core subsystem: core functions, controls the modules and other subsystems. Provides platform independency, forwards events to other engine parts.

⦿ Graphics subsystem: responsible for visualization. It is typically built upon an API (OpenGL, DirectX) ● Display models, lights, effects, post-processing, particle systems, etc.

27

Structure of a Game Engine

⦿ Audio and Music subsystem: playing audio effects and music

⦿ Artificial intelligence subsystem⦿ Network subsystem: support for network connections and

data transfer⦿ Input and Event subsystem: handle input devices and event

management⦿ Scripting subsystem: support script based development⦿ Resource subsystem: functions to access to resources⦿ Physics subsystem: make physical based simulations

possible. (E.g. racing games)⦿ Other subsystems: for math calculations, video playing, etc.

28

Structure of a Game Engine

⦿ Subsystems should be a replaceable unit⦿ Sometimes a subsystem is not developed by in-house

● the companies may decide to buy an existing and well-functioning technology.

● If the development of the new subsystem will cost more than licensing an existing○ Typical example is integrating a physical subsystem

⦿ Examining today's major game engines modularity can be seen● Main components are written using a low level language (e.g.

C/C++)○ Because of performance

● Game logic is written using a higher level language○ fewer errors○ Cheaper developers 29

Today’s major Engines

⦿ Thanks to technology, the graphics and game engines can offer sumptuous visuals

⦿ Games become increasingly complex● They contain even more cinematic parts, and functionality

⦿ A modern game engine can be very expensive● In return: developers will receive multiple years of experience

in the form of implemented algorithms

30

Today’s major Engines

⦿ Unreal Engine 4 - Epic Games⦿ Engine is free, but 5% royalty should be payed after the first

$3,000 of revenue per product per quarter⦿ ID Tech 5 – ID Software⦿ Frostbite 3 - EA Digital Illusions CE⦿ Cryengine 3 - Crytek⦿ Source Engine – Valve⦿ Unity Engine - Unity⦿ ShiVa 3D - Stonetrip⦿ C4 Engine - Terathon Software

31

Game Engines database⦿ http://devmaster.net/devdb/engines⦿ Options for detailed searching:

● Licence, programming language, API, functionality, platform, etc

32

33

Starting development with APIs...

Other APIs

⦿ Why helper API-s are needed?● The application requires a window provided by the

operating system⦿ Where the visualization will be performed⦿ It is operating system dependent!

● Event management depends also from OS: mouse, keyboard, thread management, etc

⦿ These parts of the program code are unique and platform dependent● Shipping the game to multiple platforms requires to make an

operating system layer for each platform⦿ Developers need to know exactly the operating system API⦿ Introducing a new platform requires quite a lot of work

34

Cross Platform Low Level APIs

⦿ SDL (Simple DirectMedia Layer): free cross-platform development library● Designed to provide low level access to audio, keyboard, mouse,

joystick, and graphics hardware via OpenGL and Direct3D● Platforms: Linux, Windows, Windows CE, BeOS, MacOS, Mac OS X,

FreeBSD, NetBSD, OpenBSD, BSD/OS, Solaris, IRIX, QNX, Android, Amiga

● Especially popular among game developers. Example games:○ Unreal Tournament○ Doom3○ Quake4○ Quake Wars○ Civilization: Call to Power○ etc

35

http://www.libsdl.org/

Cross Platform Low Level APIs

⦿ SFML (Simple and Fast Multimedia Library): ● Free C++ based multimedia library● It’s aim is to be like SDL● Provides rich functionality● Its programming level is higher than SDL

○ Platforms: Windows, Linux, Mac OS X, Android, iOS

⦿ GLUT (The OpenGL Utility Toolkit): ● OpenGL Utility Toolkit, a window system independent toolkit for

writing OpenGL programs● It is designed for constructing small to medium sized OpenGL

programs● It is not a full-featured toolkit. Mainly for demonstration purposes

36

http://www.sfml-dev.org/

http://www.opengl.org/resources/libraries/glut/

Cross Platform Low Level APIs

⦿ GLFW: ● Open Source, multi-platform library for creating windows with

OpenGL contexts and receiving input and events● Easy to integrate into existing applications● It is written in C

⦿ has native support for Windows, OS X and many Unix-like systems (Linux, FreeBSD)

37

http://www.glfw.org/

Cross Platform Low Level APIs

⦿ JOGL: ● Java Binding for the OpenGL API● It is designed to provide hardware-supported 3D graphics to

applications written in Java● JOGL integrates with the AWT, Swing and SWT widget sets,● It native windowing toolkit

⦿ NativeWindow API.JOGL also provides its own native windowing toolkit, NEWT.

38

http://jogamp.org/jogl/www/

High Level Frameworks

⦿ Unity: ● a cross-platform game engine● Well featured editor● Unity is notable for its ability to target games to multiple

platforms.⦿ Within a project, developers have control over delivery to

mobile devices, web browsers, desktops, and consoles● Most popular engine

39

https://unity3d.com

High Level Frameworks

⦿ Unreal Engine: C/C++ based framework. Rich editor. ● For professional purposes.

⦿ Construct 2: HTML5 / JavaScript game engine● Well-featured editor

⦿ Game Make: no programming required. Mainly for 2D Games

⦿ Source Engine: for professional purposes

40

There are a lot of other engines :)

41