Akademia Górniczo-Hutnicza im. Stanisława Staszica w ...dydaktyka:pawel_rosner... ·...

Demonstration of 3D audio library for video games 1

Akademia Górniczo-Hutnicza im. Stanisława Staszica w

Krakowie

Wydział Informatyki, Elektroniki i Telekomunikacji

KATEDRA ELEKTRONIKI

Praca dyplomowa inżynierska

Demonstration of 3D audio library for video games

Autor: Paweł Rosner

Kierunek studiów: Elektronika i Telekomunikacja

Opiekun pracy: dr inż. Bartosz Ziółko

Kraków, rok akademicki 2015/2016

Paweł Rosner - AGH 2015/2016


Oświadczam, świadom odpowiedzialności karnej za poświadczenie nieprawdy, że

niniejszą pracę dyplomową wykonałem osobiście i samodzielnie i nie korzystałem

ze źródeł innych niż wymienione w pracy.



Chciałbym w tym miejscu złożyć

serdeczne podziękowania dla

Pana Doktora Bartosza Ziółko

oraz Pana mgr inż. Szymona

Pałki z Zespołu Przetwarzania

Sygnałów AGH za nieocenioną

pomoc w trakcie realizacji tej

pracy dyplomowej.



Contents

1. Introduction

1.1 Goals

1.2 Content

2. Sound in video games

2.1 Current approach in game development

3. RAYAV audio engine

3.1 Raytracing

3.2 Soundtracing

4. Unity

4.1 Overview

4.2 Unity Asset Store

5. Implementation

5.1 Workflow

5.2 Game description

5.3 Mechanics

5.4 Models and graphics

6. Summary

6.1 Future plans

7. Bibliography



1. IntroductionThis project was chosen as a diploma thesis because of great interest in game

development and virtual entertainment. Although I never had any possibility to

gain practical experience in this matter I decided to face this challenge. The topic

of dynamic generation of sound in video games is something new, not only for

me but I think for most people nowadays who are either related to game industry

or not. RAYAV can have huge impact for future commercial productions and I

really wanted to extend my knowledge and connect love to video games and

academic work.

1.1 Goals

The most important part of this thesis was to provide demonstration

environment for RAYAV audio engine. To achieve that the simple video game

needed to be created. The choice fell on First Person Shooter designed in Unity,

as this game engine seemed to be best for beginners.

1.2 Content

Following document contains all information gathered in order to create this

game and all workflow and steps that was necessary to completion. Starting from

general description of sounds in video games, next through ray- and

soundtracing, description of capabilities of Unity, implementation workflow and

ending on summary of whole project.



2. Sound in video gamesOne of the most important feature of every video game since the beginning was

always sound. Music can create tension in thrillers, pathos in war games,

sublimity in fantasy. But aside of soundtrack in-game sound means also sound

effects (sound fx). Even the best graphics can not really let the player feel

surrounding world. Lifelike looking forrest would still be phony without lifelike

singing bird. Sound design for video games has definitely different approach than

sound design for movies. While making films we already know the outcome and

in case of video games such thing is unpredictable. We know that the player has

to perform some actions in order to complete objectives but there is no possible

way of knowing how long does it take him or which paths he use. Most important

thing to keep in mind is that, in games we are unable to control exact sequence

of events that player may experience. So that is why designing and creating

sound effects for this kind of media needs to, on some level, relay more on

probability of events than on some linear script. Vast majority of game

developers simply prepare sound in easy usable chunks that can be use in

runtime.[1]

2.1 Current approach in game development

Gaming industry is currently focused on two ways to achieve realistic sound

effects.[2] First of them concerns producing large numbers of samples and

processing them in order to obtain satisfying result. For example single foot step

should be available in different environments. There is a huge dissimilarity

between walking on the wooden floor or stone or metal stairs. Moreover we also

have to remember about sound propagation. Narrow corridors, dense jungles,

open spaces and many other cases which have to be taken into consideration.

These all locations have huge impact on sound propagation. So this static way of

preparing sound samples requires a lot of work and is very time and resource



consuming and in the end we still do not can be sure that we provided enough

sound samples which will match every in-game situation and every location

where player might be.

Second way of preparing sound effects rely on dynamic sound processing in real

time. This solution makes single sound sample possible to play in whichever

location in game and every time it will be propagated in a way corresponding to

actual environment. The point is to fully automatize part responsible for working

with sound. [3]

Figure 2.1[4]



3. RAYAV audio engine

The project of developing tool to simulate realistic behavior of sound in closed

and open spaces was challenged by Digital Signal Processing team from AGH

University of Science and Technology in Cracow. The goal was to create hybrid

image renderer by using raytracing and similar soundtracer which can be used in

commercial project in game industry.[5]

3.1 Raytracing

Raytracing is a technology of generating 3D scenes which allows to model physics

of light propagation like reflections, diffractions, refractions and other physical

phenomenons. The algorithm is following:

1. The ray starts from the point where the observer currently.

2. The nearest cross point with objects on the scene in searched.

3. For every light source placed on the scene brightness in that point is marked

according to defined lightning model.

Figure 3.1.1

The ray tracing algorithm [6]



Also in recursive version of the algorithm there is one additional step. If cross

point is part of reflecting or transparent object, derivative rays are produced and

sent and the algorithm recursively repeat itself starting from second step. Color

of the point is generated when results of computing of all derivative rays are

known.

Figure 3.1.2

Image generated with use of ray tracing [7]



3.2 Soundtracing

Soundtracing is a method based on raytracing to generate and track sound waves

in the scene. In most cases with have more than one sound source, single

receiver and every object that is part of the environment and is able to affect

propagation of sound, geometry, materials and so on. Based on this data we can

calculate how sound should be propagated from sources to the receiver. For

example some locations can suppress lower frequencies, some can produce

reverberations or interferences. All physics of sound is very important for realism

and can be simulated by use of soundtracing.

The following image helps to explain how it works.

Figure 3.1



Z – source P1 , P2 - obstaclesFp – original beam FO1 ,FO2 – reflections

Fd – diffraction [5]

As we can clearly see in Figure 3.1 sound source „Z” is generating sound wave as

a ray. This ray facing obstacle „P1”. In the result new rays are produce. FO1 and FO2

are representation of reflections from obstacle „P1” and ray „Fd” is an effect of

diffraction on edge of obstacle. If original ray does not meet any object on it's

way it can travel unchanged forward. All these rays are create in real time by

RAYAV.

Figure 3.2 [5]



Figure 3.3

Figures 3.2 and 3.3 shows how raytracing works in closed locations. Figure 3.3

was obtained by using Raya Viewer. The tool is also provided with RAYAV and

helps to visualize how sound behave in our scene. Pink cube on this scene

represents sound source and red one in our receiver. We can see how sound rays

are reflected.



4. Unity

4.1 Overview

Unity is a cross-platform game engine created by Unity Technologies. It can be

used in game development for PC, consoles, mobile devices and websites.[4]

Unity is free to personal and commercial usage, although there are some legal

requirements when commercial product starts generating profit.

Unity provides dedicated editor for creating 3D or 2D scenes.

Figure 4.1.1

Unity editor view [9]



It can be used to create every type of scene, either close or open areas. Beside of

graphical part of game engine support also various programming languages to

write scripts like C# or JavaScript. These script allow us to control every aspect of

game behavior. Every script can be attached to game object and starts to be

integral part of our scene.

4.2 Unity Asset Store

The Asset Store is a repository of free and paid-for assets including editor

extensions, scripts, 3D models and more. It provides everything that could be

needed in order to complete project.[10] The repository itself can be accessed

via browser and directly from editor which makes using it even easier and user-

friendly. Almost all models and graphics that are part of my project came from

Asset store and it proves that everyone, even inexperienced in 3D modeling and

graphics can create games.

Figure 4.2.1Asset Store as seen in editor [11]



5. ImplementationThe main goal of this project was to create a game that can show full capabilities

of RAYAV. In order to achieve that it was needed to provide variety of locations.

Thanks to use of narrow corridors, spacious rooms or open spaces in the scene,

player can easily feel aspects of three dimensional sound effects. Moreover it was

necessary to create game in first person perspective. In this perspective it is

possible to most accurately observe environment and to receive sound effects.

The genre of this game is commonly known First Person Shooter. This kind of

video games was chosen because they are very user-friendly. As this project aim

to be a demonstration of capabilities of RAYAV it simply cannot be

overcomplicated or too time consuming for regular person to finish the level. Too

high amount of possible failures, too complex controls or objectives could lead to

premature given up. Furthermore too much pressure on surviving in game and

finishing the level could prevent proper reception of sound effects and finally the

whole presentation would lost it's point.

Figure 5.1

DOOM95 as an example of classic first person shooter [12]



5.1 Workflow

Creation of project needed several points to complete.

First of them was to build a scene in Unity editor. It was a graphical part of the

project. All geometry of terrain and environment had to be provided, then

enemies had to placed on positions and player had to be given control over his

character. It was also necessary to write considerable amount of C# scripts. The

purpose of these scripts was to control every aspect of game, starting from

controlling character and weapons, through artificial intelligence of enemies and

ending on RAYAV itself.

Figure 5.1.1

Unity editor view



Next step was to design and produce acoustic map using Blender program. This

kind of geometry is mandatory for RAYAV since it has no ability to read or "see"

graphical models from Unity level. Based on provided acoustic map RAYAV can

perform calculations and generate sound beams. It is essential that these two

maps would have exactly the same dimensions, in other case sound could simply

reflect from walls that we can not see and it would have very negative impact on

reception and realism.

Figure 5.1.2

Blender editor view



And in the end it was needed to provide set of configuration files which would

load proper acoustic model, set up RAYAV parameters and allow to use it in

game. From the side of Unity the only requirement was to use few simple scripts,

attach them in proper place in the project and inject them into the scripts

controlling for example firing weapon or behavior of enemies.

5.2 Game description

The scenario place player as the only survived soldier inside occupied by

unidentified monsters military facility. His objective is to find the way out of the

infested building and look for the rescue on the other side of the valley. On his

way he will have to face countless enemies and use environment to safely pass

through whole level.

Figure 5.2.1

Main menu



The designed location is composed of two large buildings separated by sizable

woody valley. First building, where the game starts, has two floors and a secret

tunnel leading to underground garage end then to the exit. After that player

enters to the guarded by monsters valley.

Figure 5.2.2

Designed level



This location is designed as forsaken remote village deep into the forest. Several

abandoned and ramshackled houses, rusty cars and overall dimness establish

unique nature of this place which brings to mind classic horror movies. On the

way out of the valley the player enters the second building where after he defeat

his opponents the final doors are available to use and he can leave this horrifying

location and successfully finish the game.

Figure 5.2.3

The Valley



5.3 Mechanics

The project is first person shooter game. To the player's disposal was added one

rifle which is powerful enough to kill an enemy with only few shots. But in order

for the gameplay to not be too easy the gun can only fire in semi-automatic

mode, so in the end player have to wait for certain amount of time until the

weapon is reloaded and available to shoot another time.

The movements are typical for first person shooters and we use keys W, S, A, D

which corresponds to move forward, backward, left and right. Control of the

character is fully supported by special Unity game object - FPS Controller. This

component allows to set up parameters of camera, movement and any special

actions. Additional options for movement was also provided by use of FPS

Controller. Jump - space bar, and run - movement key and shift brings in more

dynamic and can be used in various situations to avoid danger. However these

abilities are limited by statistics of character.



The player's Head-Up Display, commonly known as HUD, is composed of several

important features. First of them is weapon view and crosshair. The gun was

centered in the way to make an impression that player actually hold it in his

hands. To make aiming possible the crosshair was also provided, with use of it

player can accurately fire to the enemies. Another element of HUD are status

bars. There are three of them and each one represents current status of

character's state. First of them - red - represents our health points. This bar

shrinks when player takes damage. If it attribute health reaches zero the

characters dies and level is restarted. The next is yellow armor bar. It is

unreachable and allow us to take some amount of hits from enemy until the

health starts to be reduce. Last of them is stamina bar. This green bar represents

ability to jump and run. When player make any special movement action this bar

shrinks, and when stamina drops to zero player should stop for a moment a wait

for it to regenerate, in other case he will only have ability to slow walk.



5.4 Models and graphics

The vast majority of models and textures originate from Unity Asset Store or was

a standard asset in Unity editor. Asset store turned out to be almost totally

sufficient source of models and there was no need to create them by self.

Downloaded models were used in the project for cars, weapon, small building in

the valley, monsters, doors, containers, barrels and boxes. Only the two larger

buildings, mentioned before, terrain and acoustic map were modeled by use of

Unity editor and Blender. These two editors were enough to achieve all goals and

even huge lack of experience in 3D modeling and graphics was not an obstacle.



6. Summary

Even though the project turned out to be very challenging for someone

inexperienced in game development, the main goal, which was to develop fully

functional demonstration of RAYAV, was accomplished thanks to many

facilitations offered by Unity. The result of all of this is completely playable level

with several minutes of gameplay. The overall framing of relatively simple and

user-friendly game allow user to focus on main aspect of the project - sound

effects. Fulfillment of all the requirements took many months of hard work and

especially lot of learning. Even with huge user friendliness of Unity and general

intuitiveness of this environment it was still very time consuming to get used to

all of the capabilities.

6.1 Future plans

At the moment project has a lot of different paths for development. In the future

it should be firstly extended by adding more levels with various locations which

could better show RAYAV's features. It is also worth to consider adding more

features to the game itself like enlarged user equipment, new weapons and

enemies and improvement of enemy AI. Project is still open and beside of

accomplishment of all main goals there are many ways to make it even better.



7. Bibliography

[1] http://www.gamesounddesign.com/GettingYourSoundIntoTheGame.html

[2] Karen Collins, An Introduction To The History, Theory

And Practice Of Video Game Music And Sound Design, 2008 MIT

[3] http://www.agh.edu.pl/blog-naukowy/info/article/dzwiek-w-grach-komputerowych/

[4] https://www.youtube.com/watch?v=EWatzCC7rk0#t=32

[5] www.dsp.agh.edu.pl

[6] https://upload.wikimedia.org/wikipedia/commons/thumb/8/83/Ray_trace_diagram.svg/300px-Ray_trace_diagram.svg.png

[7] http://techreport.com/blog/16771/fun-with-raytracing

[8] www.unity3d.com/unity/

[9] http://forum.unity3d.com/threads/how-to-make-unity-

editor-display-in-higher-resolution.355954/

[10] http://unity3d.com/asset-store/help

[11] https://docs.unity3d.com/351/Documentation/Manual/AssetStor

e.html

[12] http://www.techspot.com/article/140-21-first-person-

shooters/


Akademia Górniczo-Hutnicza im. Stanisława Staszica w ...dydaktyka:pawel_rosner... ·...

Documents

Transcript of Akademia Górniczo-Hutnicza im. Stanisława Staszica w ...dydaktyka:pawel_rosner... ·...