Simulation Modeling of a Stylized Game Asset Pipeline...

Author: Benedict Delannor

Student ID: BD87160

Exam Number: 402541

Supervisor: Hartanto Wijaya Wong

Simulation Modeling of a Stylized Game Asset Pipeline: Using Simulation as a Support-Tool for Setting Milestone Deadlines

Submitted in partial fulfillment of the requirements for the degree of

B.Sc. in Economics and Business Administration

At

Department for Business Studies

Aarhus School of Business and Social Sciences

Aarhus University

May 1, 2012

BSc(B) – Bachelor Thesis Simulation Modeling of a Stylized Game Asset Pipeline: Using Simulation as a Support-Tool for Setting Milestone Deadlines

Author: Benedict Delannor of 49

Abstract

The creation of computer games is a complex process that can take years to finish. Many risk

factors influence this process which makes schedule durations difficult to predict. Since

commitments to deadlines can be costly if schedules are too tight, game companies can

benefit from having a reliable tool for evaluation of schedule duration and milestone

deadlines.

A stylized game asset pipeline model and simulation program for sampling simulated

schedule durations under uncertainty was developed for this thesis in order to investigate its

potential applicability as a support-tool for setting milestone deadlines through selection of

percentile ranks from simulated distributions of schedule durations.

The thesis offers a comprehensive visual presentation of the developed model while

briefly describing the simulation program that has been developed in relation to this thesis

featuring dynamic rescheduling and preemption of tasks. A baseline is established and

analyzed before the model is simulated with four different risk factors in isolation and in

combination and compared against the baseline.

The simulation results show that on average each risk factor in the model adds time to

the schedule duration. However, risk factors in combination do not add time to the baseline

duration equivalent to the sum of the individual factors because of overlapping effects.

Simulation modeling and programming is an intricate and time consuming task and

models can easily get overly complicated in the effort to create a good fit with the system it is

modeling. Still, it might be possible that a simulation model can be developed with enough

detail and accuracy to reflect a real game asset pipeline and evaluate the robustness of

deadlines for the benefit of game companies. Even if the level of detail needed to achieve this

is yet unknown.



Table of Contents

1 INTRODUCTION ...................................................................................................................................... 7

1.1 PROBLEM STATEMENT .................................................................................................................................... 8

1.2 LITERATURE REVIEW (SHORT FORM) ................................................................................................................. 8

1.3 APPROACH ................................................................................................................................................... 9

1.4 GENERAL ASSUMPTIONS ................................................................................................................................. 9

1.5 DELIMITATION ............................................................................................................................................ 10

1.6 DEFINITION OF TERMS .................................................................................................................................. 10

2 GAMES DEVELOPMENT AND THE ASSET PIPELINE ................................................................................ 12

3 SIMULATION PROGRAM ....................................................................................................................... 13

3.1 CHOICE OF PLATFORM .................................................................................................................................. 13

3.1.1 Object-oriented & Event-driven Program Design .......................................................................... 14

3.1.2 Pseudo-Random Number Generator ............................................................................................. 14

3.1.3 Probability Distribution Functions ................................................................................................. 15

3.1.4 Memory Structures ....................................................................................................................... 15

4 SIMULATION MODEL ............................................................................................................................ 15

4.1 MODEL ..................................................................................................................................................... 15

4.1.1 Workers ......................................................................................................................................... 17

4.1.2 Workflows ..................................................................................................................................... 18

4.1.3 Work-tools – Computers ............................................................................................................... 20

4.2 SIMULATION LOGIC ...................................................................................................................................... 21

4.2.1 Overview of Events ........................................................................................................................ 21

4.2.2 Worker Arrival Event ..................................................................................................................... 22

4.2.3 Worker End Task Event ................................................................................................................. 23

4.2.4 Worker Leave Sick & Recover Events ............................................................................................ 24

4.2.5 Machine Breakdown & Repair Events ........................................................................................... 25

4.3 DISPATCHING RULES & SCHEDULING POLICIES .................................................................................................. 26

4.4 STOCHASTIC VARIABLES AND DISTRIBUTIONS .................................................................................................... 27

4.4.1 Task Duration ................................................................................................................................ 27

4.4.2 Machine Breakdowns.................................................................................................................... 28

4.4.3 Worker Absenteeism ..................................................................................................................... 28

4.4.4 Embedded QA ............................................................................................................................... 28

4.5 OBJECTIVE ................................................................................................................................................. 29

4.6 SHORT-COMINGS OF THE MODEL .................................................................................................................... 29



5 EXPERIMENTAL DESIGN ........................................................................................................................ 30

5.1 INPUT VARIABLES ........................................................................................................................................ 31

5.1.1 Resources ...................................................................................................................................... 31

5.1.2 Asset Requirements ...................................................................................................................... 31

5.1.3 Standard Task Durations ............................................................................................................... 32

6 RESULTS ................................................................................................................................................ 32

6.1 DEFINITIONS ............................................................................................................................................... 33

6.2 BASELINE ANALYSIS ..................................................................................................................................... 33

6.2.1 Baseline – Level 1 .......................................................................................................................... 35

6.2.2 Baseline – Level 2 .......................................................................................................................... 36

6.3 SCENARIO I – DEADLINE ROBUSTNESS WITH TASK DURATION VARIABILITY.............................................................. 39

6.4 SCENARIO II – DURATION ROBUSTNESS WITH MACHINE BREAKDOWNS ................................................................. 41

6.5 SCENARIO III – DEADLINE ROBUSTNESS WITH WORKER ABSENCE ......................................................................... 42

6.6 SCENARIO IV – DEADLINE ROBUSTNESS WITH EMBEDDED QA ............................................................................. 43

6.7 SCENARIO V – DEADLINE ROBUSTNESS WITH ALL FACTORS COMBINED .................................................................. 45

7 VALIDATION ......................................................................................................................................... 47

8 CONCLUSION ........................................................................................................................................ 47

9 FURTHER STUDIES................................................................................................................................. 48

10 REFERENCES ......................................................................................................................................... 49

List of Tables

TABLE 1 – DEFINITION OF TERMS ................................................................................................................................. 10

TABLE 2 - WORKER SPECIALIZATIONS ............................................................................................................................ 18

TABLE 3 - PRODUCTION TEAM ..................................................................................................................................... 31

TABLE 4 - GAME ASSET REQUIREMENTS ......................................................................................................................... 32

TABLE 5 - STANDARD TASK DURATIONS ......................................................................................................................... 32

TABLE 6 – BASELINE –WORKER AVERAGE (IN STANDARD DAYS) .......................................................................................... 36

TABLE 7 – BASELINE - CHARACTER ARTIST AVERAGE (IN STANDARD DAYS) ............................................................................ 37

TABLE 8 – BASELINE - ENVIRONMENT ARTIST AVERAGE (IN STANDARD DAYS) ........................................................................ 37

TABLE 9 – BASELINE – ANIMATOR AVERAGE (IN STANDARD DAYS) ...................................................................................... 37

TABLE 10 – BASELINE - VFX ARTISTS (IN STANDARD DAYS PER WORKER) .............................................................................. 38

TABLE 11 – BASELINE – SCRIPTERS (IN STANDARD DAYS PER WORKER) ................................................................................. 38

TABLE 12 – BASELINE - LEVEL DESIGNERS (IN STANDARD DAYS PER WORKER) ........................................................................ 38



TABLE 13 – BASELINE - TECH ARTISTS (IN STANDARD DAYS PER WORKER) ............................................................................. 39

TABLE 14 – DURATION, SLACK AND RANK – TASK DURATION VARIABILITY (DURATION IN STANDARD DAYS) ................................ 40

TABLE 15 – DURATION, SLACK AND RANK – MACHINE BREAKDOWNS (DURATION IN STANDARD DAYS) ...................................... 42

TABLE 16 – DURATION, SLACK AND RANK – WORKER ABSENCE (DURATION IN STANDARD DAYS) .............................................. 43

TABLE 17 – DURATION, SLACK AND RANK – EMBEDDED QA ............................................................................................. 44

TABLE 18 – DURATION, SLACK AND RANK – ALL UNCERTAINTY FACTORS (DURATION IN STANDARD DAYS) .................................. 45

List of Figures

FIGURE 1 - A CRUDE SKETCH OF THE OBJECT-ORIENTED EVENT-DRIVEN PROGRAM DESIGN ........................................................ 14

FIGURE 2 - CONCEPTUAL MODEL OF GAME ASSET PIPELINE............................................................................................... 16

FIGURE 3 - EXPANDED MODEL USING CHARACTER ASSET WORKFLOW AS EXAMPLE ............................................................... 17

FIGURE 4 - CHARACTER ASSET WORKFLOW .................................................................................................................... 18

FIGURE 5 - SIMPLE ENVIRONMENT ASSET WORKFLOW ..................................................................................................... 19

FIGURE 6 - SIMPLE ANIMATED ENVIRONMENT ASSET WORKFLOW ...................................................................................... 19

FIGURE 7 - COMPLEX ENVIRONMENT ASSET WORKFLOW .................................................................................................. 19

FIGURE 8 - COMPLEX ANIMATED ENVIRONMENT ASSET WORKFLOW ................................................................................... 20

FIGURE 9 - LEVEL ASSET WORKFLOW ............................................................................................................................ 20

FIGURE 10 - EVENT DIAGRAM ...................................................................................................................................... 22

FIGURE 11 - WORKER ARRIVAL FLOWCHART................................................................................................................... 23

FIGURE 12 - WORKER END TASK FLOWCHART ................................................................................................................ 24

FIGURE 13 - WORKER LEAVE SICK AND RECOVER FLOWCHART ........................................................................................... 25

FIGURE 14 - MACHINE BREAKDOWN AND REPAIR FLOWCHART .......................................................................................... 26

FIGURE 15 –INDIVIDUAL DURATION FOR 8 NON-OVERLAPPING CONSECUTIVE PIPELINES ........................................................ 34

FIGURE 16 – AVERAGE PIPELINE DURATION ................................................................................................................... 34

FIGURE 17 –INDIVIDUAL DURATION FOR 8 OVERLAPPING CONSECUTIVE PIPELINES ................................................................ 34

FIGURE 18 – AVERAGE PIPELINE DURATION ................................................................................................................... 34

FIGURE 19 –TOTAL DURATION DECOMPOSITION ............................................................................................................. 35

FIGURE 20 – WORK RATIO .......................................................................................................................................... 35

FIGURE 21 –BASELINE (TOTAL DURATION SPLIT) ............................................................................................................. 36

FIGURE 22 –DISTRIBUTION OF TOTAL DURATION (TP1) ................................................................................................... 40

FIGURE 23 – DISTRIBUTION OF TOTAL DURATION (TP2) ................................................................................................... 40

FIGURE 24 –DISTRIBUTION OF TOTAL DURATION (MP1) .................................................................................................. 41

FIGURE 25 – DISTRIBUTION OF TOTAL DURATION (MP2) ................................................................................................. 41

FIGURE 26 –DISTRIBUTION OF TOTAL DURATION (WP1) .................................................................................................. 42

FIGURE 27 – DISTRIBUTION OF TOTAL DURATION (WP2) ................................................................................................. 42



FIGURE 28 –DISTRIBUTION OF TOTAL DURATION (QP1) ................................................................................................... 44

FIGURE 29 – DISTRIBUTION OF TOTAL DURATION (QP2) .................................................................................................. 44

FIGURE 30 –DISTRIBUTION OF TOTAL DURATION (AP1) ................................................................................................... 45

FIGURE 31 – DISTRIBUTION OF TOTAL DURATION (AP2) .................................................................................................. 45

FIGURE 32 –PLOT OF PERCENTILE RANK VS. SLACK (AP1) ................................................................................................. 46

FIGURE 33 – PLOT OF PERCENTILE RANK VS. SLACK (AP2) ................................................................................................ 46



1 Introduction

The computer games industry is a multi-billion dollar industry with a few large companies

and a host of small to medium-sized companies all fiercely competing on technology,

innovation, and product differentiation in a hit-driven market. Only a few game companies

make really big profits while the vast majority of them are struggling.

Most game companies are reliant on contracting with publishers for financing of the

production and marketing of their games in full or in part. Usually these contracts ensure

payment releases to the game company dependent on timely milestone deliveries and often

penalties are installed for not meeting milestone deadlines. Such penalties may force

developers to cut corners and lower quality to meet milestone deadlines which may prove

even costlier for the game company in the long run. It puts the company in a precarious

situation where they will incur costs no matter what they do if their initial estimates have

turned out to be too tight.

Game production setups are as diverse as games are different. Games exist in many

different themes and genres, and on many different platforms. In fact, no two game

productions are exactly the same. Even if game code and assets to some extent can be reused

the drive to push technology and be innovative makes for an ever changing production

environment. Estimating schedule durations and establishing robust deadlines is a feat which

many game companies fail to deliver on. Not only is the production of games inherently

complex it also faces many sources of risk and uncertainty which makes for a huge challenge

in planning and scheduling. Many game companies use the critical path method (CPM) and

simple tools like Microsoft Project for scheduling, and by adding slack (allowance) to their

schedules they try to accommodate the impact of uncertainty. This is seemingly an effective

approach. However, it has led to many a headache among managers as projects often do not

meet deadlines, even with high levels of slack in the schedule. Causes for this can be many.

Managers and producers have different perspectives, knowledge and agendas. Managers tend

to want shorter development times and producers typically want as robust a schedule as

possible. Negotiating schedules with regards to milestone deadlines easily becomes a game of

give and take. Producers add extra slack to their schedules because they know managers will

demand shorter development time regardless, and managers anticipate that producers will add



extra slack so they demand shorter development time regardless, without knowing the exact

details and impact on the schedule.

Game companies would benefit from having a tool that could provide robust and

realistic deadlines that both managers and producers could accept and believe in; chances are

that such a tool would be adopted by game companies and that more game development

projects would finish on time in the future.

1.1 Problem Statement

The aim of this thesis is to develop a simulation model of a stylized game asset

pipeline that can be used to demonstrate how schedule durations can be simulated and

deadlines chosen from associated percentile ranks from the simulated distribution output. For

this purpose, a simulation model and program is developed featuring dynamic rescheduling

and preemption of tasks. The simulation program is used to establish a deterministic baseline

(a.k.a. predictive schedule) before simulating scenarios incorporating stochastic variation.

Firstly, the baseline is analyzed by decomposition of resources and time usage before

comparison. Secondly, the impact of four risk factors on schedule duration, in isolation and in

combination, is compared against the baseline, with focus on the slack needed for the

baseline to meet desired percentile ranks. Finally, the plausibility that a simulation model

could be developed with enough detail to estimate schedule durations for setting realistic

milestone deadlines for actual game asset pipelines is considered.

1.2 Literature Review (Short Form)

Carter (2004) gives a practical introduction to the game asset pipeline.

Chandler (2010) describes aspects of game production from a producer’s perspective.

Law (2007) gives a comprehensive walkthrough of the process of making a discrete-event

system.

Hulett (2009) offers a practical approach to analyzing schedule risk.



Herroelen and Leus (2004) review possible procedures for generation of robust schedules and

offer a framework for identifying proper scheduling methods

Herroelen and Leus (2005) review the fundamental approaches for scheduling under

uncertainty

Rubinstein and Kroese (2008) give a rigorous introduction to simulation and the Monte Carlo

Method.

Vieira et al. (2003) describes a framework for understanding rescheduling strategies, policies,

and methods.

Note that the above literature list is not a comprehensive list. Only the most essential sources

are reviewed here. References for all sources can be found in the References section on the

last page of the thesis.

1.3 Approach

The process of acquiring data for this thesis was iteratively exercised in four stages. Firstly,

the model was conceived. Secondly, the model was developed and tested. Thirdly, data was

created using the developed simulation program. Finally, data was validated and analyzed.

1.4 General Assumptions

The model is stylized and is assumed to be a closed system, i.e. no interference from outside

the modeled system.

Independence is assumed between all stochastic variables in the model.

Modeling Programming Simulation Data Analysis



1.5 Delimitation

The thesis strictly focuses on the game asset pipeline. Any other organizational aspect is

ignored. No optimization will be performed on schedules or resources or any other aspect of

the simulation for that matter. Stochastic variables and distributions have been simplified as

the correctness of distributions is not a concern for this thesis.

1.6 Definition of Terms

Table 1 – Definition of Terms

Term Description

Deadline Robustness Deadline robustness is defined here as the percentile rank of the simulated distribution of schedule

durations. E.g. a deadline robustness of 99% means the 99th percentile of the associated simulated

distribution of schedule durations. This approximates to a 99% certainty that the simulated schedule

duration will not be longer than the duration at the 99th percentile, given a high number of

simulated observations.

Realistic Deadline A deadline is softly defined as realistic if there is a “good chance” that the deadline can be met

under real circumstances. The “good chance” is up to the manager or producer to define and is done

by choosing a suitable percentile rank for calculating the slack percentage and deadline (schedule

duration).

Artist Artist is the term for a worker working with creating art assets for the game.

- Character Artist An artist who specializes in creating models, maps and textures for character assets

- Environment Artist Environment artists create environment assets.

- Animator Create rigs on models and animations

- VFX Artist Creates visual effects for various game assets

- Scripter Scripters create game logic (scripts). Even though scripters technically do programming they are

part of the asset production team.

- Tech Artist The tech artist bridges the gap between programmers and artists and in model developed in this

thesis tech artist also handle the technical conversion of art assets to file formats usable in the game

build

- Level Designer The level designer designs levels in a level editor which requires game assets and relies on dummy

assets for those assets that are not finished yet.

Asset or Game Asset An asset is a final object creation (game part) going into the game. There are six types of assets:

Character, Simple Environment, Complex Environment, Simple Animated Environment, Complex

Animated Environment, Level

Game Build A build is the current version of a game. The latest build will usually be compiled with all the

newest assets.

Job A job is the requirement definition of a specific asset. A job infers the use of a specific workflow.



Level Asset A level asset is a part of the game world where players have virtual existence. Levels are built from

many different game assets and employ a high degree of asset reuse.

Task A task is the basic unit scheduled for work in a workflow. Abstractly, tasks are handled at

workstations by workers with the right qualifications.

- Modeling Modeling is the task of creating model sub-assets for character and environment assets

- Mapping Mapping is the task of creating map sub-assets for characters and environment assets

- Texturing Texturing is the task of creating texture sub-assets for characters and environment assets

- Rigging Rigging is the task of creating a rig (skeleton) sub-assets for characters and complex animated

environment assets

- Animation Animation is the task of creating animation sub-assets for characters and animated environment

assets.

- VFX VFX is the task of creating visual effects

- Scripting Scripting is the task of creating game logic

- Tech Art Is the task of transforming sub-assets into a format that can be used

- Level Design Level design is the task of creating a level (or virtual world) in a level editor which draws on all

relevant sub-assets for that level

Sub-asset Is an asset that is required for the fulfillment of another asset

Workflow The flow of sub-assets to form a specific asset type.

Workstation In a workstation workers can perform exactly one type of task. The workstation is an abstract

construct.

Schedule Duration Duration or schedule duration is defined as the time between the start of the first task until the end

time of the last task. This is often referred to as lead time.

Work Time Work time is defined as the aggregated time that elapses from the start of a task until the finishing

of that task. This is often is referred to as processing time (or sometimes lead time).

Idle Time Idle time is the workers’ non-productive time that may indicate excess of resources or a poor layout

or synchronization of workflows. Idle time is split into three parts: Idle Lead, Idle Between, and

Idle Lag.

Idle Lead Idle lead is defined as the aggregated time that elapses from the start of the first pipeline production

and to the time when a worker starts working on his first task. This is normally part of idle time.

Idle Between Idle Between is defined as the aggregated time that elapses between the end of a task and the start

of another.

Idle Lag Idle Lag is defined as the workers’ non-productive time from the time they have finished their last

task until the time when the production has ended.



2 Games Development and the Asset Pipeline

Computer games development is generally considered a complex process that relies

extensively on the use of human resources with highly specialized skillsets. While the

number of people involved in a large-scale games production can be counted in the hundreds,

the number of distinct assets and sub-assets going into a game is thousand-fold. Computer

games development is an iterative process that often is divided into 4 main phases:

Conceptualization, pre-production, production and post-production. Sometimes a prototyping

phase is inserted between conceptualization and pre-production. (Chandler, 2010).

During the pre-production phase, the production pipeline will be established and

resource requirements as well as standard task durations will be verified. The production will

be planned and executed accordingly. The importance of the pre-production phase cannot be

understated. Every task of the production pipeline should ideally be tried and proven useful in

this phase and in turn the derived knowledge should be used for planning the production

phase. There is always the risk that time estimates are too optimistic for sustained

production. Therefore planners usually add time slack to these estimates based on previous

experience and some underlying perception of risk. Complexity of the task measured as

standard hours divided by a worker ability factor can be used as a guideline for estimating the

time necessary to complete a task. (Hulett, 2009).

Worker absenteeism poses a genuine risk to the production schedule which must be

accounted for by planners. The magnitude of this risk can be assessed from worker absence in

previous productions or estimated from industry standards. Machine breakdowns do not

occur as frequently as worker absenteeism and with proper backup solutions in place the

impact of a breakdown is generally less than the absence of a worker. Miscommunication and

misspecifications often are the cause of faulty game parts that need to be reworked. Quality

assurance (QA) embedded in the pipeline catches most of these errors. While the impact on

the schedule can be huge, the frequency of faulty game parts is best estimated from past

productions using the same production team. All of these risk factors along with need for

robust schedules in games development projects.



The games production pipeline comprises of three structural dimensions: Workers,

workflows, and work-tools. These dimensions form an elaborate mesh of interdependency,

which in and by itself easily causes bottlenecks, and together with inherent sources of risk

and uncertainty it makes for a big challenge in the planning and maintaining of an efficient

production pipeline. Even small changes in one end of the pipeline can have sizeable impact

in the other end, which can translate into massive waste of time and resources, and requires

planners to either constantly revise their production plans or plan with excessive time slack.

The typical asset pipeline involves artists of various specializations to create a variety

of assets and sub-assets. Typical artist specializations are: Character artists, environment

artists, animators, VFX artists, scripters, tech artists and level designers. (Chandler, 2010).

3 Simulation Program

The simulation program is based on a discrete-event system (DES) with a time unit of one

minute. It applies dynamic rescheduling and allows preemption of tasks. The program

implements the model presented in section 4 and it produces output of accumulated work

time and idle time for aggregates of workers, which can easily be copied to Excel and SPSS

for validation and analysis.

3.1 Choice of Platform

In comparison with special-purpose simulation languages, such as Simula and others or

packages such as Simul8, Arena, etc., a general-purpose language offers higher degree of

freedom in modeling, albeit at the cost of extensive development time. The choice of

platform for implementing the simulation program fell on Microsoft Visual C#, a simple

general-purpose object-oriented programming language. The simple structure of C#, strong

typing, plenty of standard libraries and automatic memory management makes C# a practical

language for programming a simulation model. C++ was another apparent contender but

automatic memory management tipped the scale towards C#. (MSDN, 2010).



3.1.1 Object-oriented & Event-driven Program Design

The simulator is event-driven and the program is object-oriented, utilizing features such as

inheritance, polymorphism and encapsulation of data structures (Wikipedia, 2012). In this

way the programming becomes more intuitive and it is easier to maintain than traditional

programs.

Figure 1 - A crude sketch of the object-oriented event-driven program design

3.1.2 Pseudo-Random Number Generator

The simulator uses the Mersenne-Twister algorithm for pseudo-random number generation.

Mersenne-Twister has a very long period (219937

-1) compared to other pseudo-random

number generators which is practical for creating independent stochastic variables

(Matsumoto, 1998). The simulator assumes independence by using unique seeds and having

the simulation stay within this period. For the purpose of this thesis, the proposed finite

model uses only a fraction of the random number period available which eliminates the need

for constructing independent streams spanning several seeds.

Job

Event

MachineWorkerWorkstation

PriorityEventQueue

Simulator

Assets Specialized Workers

1

Model

Workflow

1

SimElement



3.1.3 Probability Distribution Functions

The program only employs three types of distribution: The uniform and triangular

distributions as well as the very versatile inverse Weibull distribution (Gusmão et al., 2011).

3.1.4 Memory Structures

The main memory structure component in the simulation program is the priority queue.

Priority queues are used with the simulator’s event queue and job task queues at

workstations. The priority queue implementation utilizes sorted lists to achieve its purpose.

Also general linked lists and dictionaries are among the many memory structures applied in

the program.

4 Simulation Model

The simulation model presented here is stylized and therefore does not approximate an actual

pipeline. The stylized game asset pipeline model is loosely based on the pipeline found at

Autodesk’s website (Autodesk, 2012).

4.1 Model

In the model, workers create game assets on their computers in accordance with job

descriptions. Work is done through asset-specific workflows. Final assets comprise a number

of sub-assets created throughout a dedicated workflow for each asset type. When enough

game assets are created work on levels can commence.



Figure 2 - Conceptual Model of Game Asset Pipeline

Note that circles and arrows with dashed lines in the figure above are not part of the

simulation model. Game Design is the process of creating the game design document which

defines the job requirements for game assets. Note also that Level Specs is not to be confused

with the process of creating level assets. Rather, it is the process of creating the level design

document which defines the job requirements for level assets. Game build is the current

compilation of the game with all currently finished assets.

Figure 3 shows a more detailed model using the character asset workflow as an

example to visually demonstrate the full conceptual model. Jobs are queued at the first

workstation in the workflow and processed by workers assigned to the workstation. Each

worker will handle one task at a time. When the task is finished the job is moved forward to

the next succeeding workstations in the workflow (black arrows) where idle workers will

commence work and do embedded QA testing on required sub-assets (the result of preceding

tasks). In case sub-assets do not pass embedded QA they are set for rework (purple dashed

arrows). If there are no idle workers to commence work the tasks are queued for later

processing (see section 4.2 and 4.3 for detailed simulation logic).

Jobs AssetsWorkflows

Workers

Game Assets

Workflow Levels

Levels

JobsLevel

Specs

Game

Design

Game

Build



Figure 3 - Expanded Model using Character Asset Workflow as Example

4.1.1 Workers

Workers are assigned with one computer on which they create assets. Each worker is

assigned to a number of different workstations according to their skillset and can work in

exactly one workstation on one job (asset/sub-asset) at a time. In the model there are six

different worker types, each with a specialized skillset (see Table 2).

Each worker has a designated computer with all necessary work-tools installed on it. All

workers a specialized to some degree and can at one or more workstations, however, only at

one workstation at a time.

When a worker is ready to work he/she starts working on the first task where required sub-

assets are available in accordance with the dispatching rules.

Modeling

Mapping Texturing

Animation Tech Art

Workflow

(Character Asset)

VFX

Rigging Scripting

Job Queue

(FIFO)

Legend

Workers’ Task Priorities:

1. Own Preempted Task

2. Closest to Finished Asset

1. Rework Tasks (LWR/FIFO)

2. New Tasks (LWR/FIFO)

3. Other Preempted Tasks (LWR/FIFO)

Finished

AssetsJob Asset

Embedded QA

Character Artists Animators Scripters VFX Artists Tech Artists

Computers

Workstation

New Task Preempted Task

Workflow

Embedded QA

Rework Task

Workers

Specilization



Table 2 - Worker Specializations

Worker Type Skillset

Character Artist Modeling, Mapping and Texturing on Character Assets

Environment Artist Modeling, Mapping and Texturing on Environment Assets

Animator Rigging, Animation

Level Designer Level design

Tech Artist Tech art

Scripter Scripting

VFX Artist Visual effects

4.1.2 Workflows

The workflow is represented by a simple bi-directional graph of workstations. Workers

assigned to a workstation can create sub-assets which are required at the next workstation in

the workflow. Final result from a workflow is an asset. The model introduces six different

types of assets, each with its own workflow.

The task at the workstations in the workflows is denoted a standard task duration

which is estimated from the work done in the pre-production phase where every task has tried

and tested. Embedded QA at each workstation evaluates predecessors in the graph and may

result in rework having to be done. Without the embedded QA backflow the workflows are

basically stochastic PERT networks. The program evaluation and review technique (PERT)

is often used in project management (Rubinstein and Kroese, 2008).

Note that legends for the following workflow diagrams are the same as the one presented in

Figure 3.

Figure 4 - Character Asset Workflow

Modeling

Mapping Texturing

Animation Tech Art

Workflow

(Character Asset)

VFX

Rigging Scripting

Embedded QA



Figure 5 - Simple Environment Asset Workflow

Figure 6 - Simple Animated Environment Asset Workflow

Figure 7 - Complex Environment Asset Workflow

The workflow of complex animated environment assets (CAEA) resembles the workflow of

character assets. The only difference is that while the modeling, mapping and texturing of

character assets are done by character artists, the same tasks on CAEAs are done by

environment artists.

Modeling

Mapping Texturing

Tech Art

Workflow

(Simple Environment Asset)

Scripting

Embedded QA

Modeling

Mapping Texturing

Animation Tech Art

Workflow

(Simple Animated Environment Asset)

Scripting

Embedded QA

Modeling

Mapping Texturing

Tech Art

Workflow

(Complex Environment Asset)

VFX

Scripting

Embedded QA



Figure 8 - Complex Animated Environment Asset Workflow

Asset requirements for each level are kept very simple: All levels are released into the

pipeline when 50% of the game assets are done. Then they are processed first in first out.

Scripting, VFX (visual effects) and level design are done in parallel.

Figure 9 - Level Asset Workflow

4.1.3 Work-tools – Computers

All computers are assumed to have been installed with all needed software tools. In principle,

this allows workers to use any computer that is available at any given time. But in the model

there is no changing of computers.

Modeling

Mapping Texturing

Animation Tech Art

Workflow

(Complex Animated Environment Asset)

VFX

Rigging Scripting

Embedded QA

Tech Art

Workflow

(Level Asset)

VFX

Scripting

Embedded QA

Level Design

Finished

Asset

Package

Scheduled according to

the level’s asset

requirements

Level Job Queue

(FIFO)



4.2 Simulation Logic

In order to fully appreciate the workings of the simulation one must understand the logic that

it follows, which is presented in this section.

4.2.1 Overview of Events

The main simulation logic revolves around five main procedures as depicted in the event

diagram below. The three procedures, Worker Arrival, Machine Arrival, and Leave Sick, are

scheduled once from the main program to kick off the simulation. Thereafter the simulation is

‘keeping itself alive’ through events (dashed lines) and direct calls (solid lines) between the

procedures.

The focal point is Worker Arrival where the scheduling of tasks is happening.

Machine breakdowns are handled in a closed system between Machine Arrival and

Departure. Worker Absenteeism is handled in the same way between the procedures, Recover

and Leave Sick.

In theory the event-driven simulation could have been made with just two main

procedures which would simplify the model on paper, however, at the same time the logic

becomes less readable and less maintainable in code. The logic from the event diagram is

detailed in flowcharts on the following pages.

Arrows with dashed lines denote an event that is processed through the event queue.

Arrows with solid lines denote a direct call, circumventing the event queue.



Figure 10 - Event Diagram

4.2.2 Worker Arrival Event

The worker arrival event triggers the start of a task in accordance with the given scheduling

policies assuming that the worker is not sick and that the worker’s computer is not down.

The starting of a new task immediately triggers embedded quality assurance on the

required sub-tasks as per policy (the flowchart assumes that embedded QA is always

happening while in fact this can be turned on and off by the user of the program). If the sub-

asset does not pass QA, rework will be needed and if there is an idle worker this worker will

immediately start rework on that sub-asset, otherwise the task is queued. Note that sub-assets

will be reworked only once, as per policy.

Worker Arrival

Worker End Task

Machine Arrival Machine Departure

Recover

Leave Sick



Figure 11 - Worker Arrival Flowchart

4.2.3 Worker End Task Event

When the worker end task event is triggered the next tasks in the workflow are readied and

started if there are idle workers. Note that if either a Machine Down Event or a Leave Sick

Event is due on the same time as the Worker End Task Event is processed, the worker cannot

start a new task. Again the starting of a task causes embedded quality assurance on preceding

sub-assets.

Worker Arrival

Worker Sick?

Yes

Machine

Down?

No

Own

Preempted

Task?

No

New Task in

Queue?

Preempted

Task in

Queue?

No

Yes

Yes

Yes Start New Task

Resume Own

Preempted Task

Resume Other

Worker’s

PreemptedTask

End

Calculate Task

Duration

Schedule End

Task

Worker End Task

No

Check Each

Workstation in

Priority Order

Yes

Embedded QA

Sub-Asset

Pass?

No

Idle Worker? YesIdle Worker

Start Task ReworkNo

Enqueue Task for

Rework Lster

Yes

Check Each

Workstation in

Priority Order

No

Check Sub-AssetMore Sub-

Assets?Yes

No



Figure 12 - Worker End Task Flowchart

4.2.4 Worker Leave Sick & Recover Events

When the simulation is started, all workers’ next time to be sick (TTS) is calculated and

Leave Sick Events are scheduled accordingly.

When a Leave Sick Event is triggered it first checks if the worker is busy. If so, the

current task is preempted. Then the time to recover (TTM) is calculated and a Recover Event

is scheduled accordingly.

Worker End Task

Finish Work

Rework?

NoWorker Arrival

End

Get Next Due

Task in WorkflowNo

Can Start

New Task?Yes Start Due Task

No

More Tasks

Up?

No

Yes Idle Workers?

Enqueue Task

Let Idle Worker

Start Due TaskYesNo

Update List of Due

Tasks in Workflow

Busy?

Yes

Yes

Calculate Task

Duration

Schedule End

Task

Worker End Task

Embedded QA

Sub-Asset

Pass?

No

Idle Worker? YesIdle Worker

Start Task ReworkNo

Enqueue Task for

Rework Lster

Yes

Check Sub-AssetMore Sub-

Assets?Yes

No



When a Recover Event is triggered it calculates the time to next sickness period (TTS)

and schedules the Leave Sick Event. If the worker’s computer is working a Worker Arrival

Event is triggered directly.

Figure 13 - Worker Leave Sick and Recover Flowchart

4.2.5 Machine Breakdown & Repair Events

The Machine Arrival Event simulates the event when the computer has been repaired and

returns into production. The Machine Departure Event simulates the event when a machine

breaks down and can no longer be used in production. There is a simple relationship between

the two.

When a Machine Arrival Event is triggered, the time to next breakdown (TTB) is

calculated and a Machine Departure Event is put at the event queue for that time. If the

worker is neither sick nor busy then a Worker Arrival Event is triggered directly.

When a Machine Departure Event is triggered, the time to repair (TTR) is calculated

and a Machine Arrival Event is scheduled for that time. If the worker is busy the current task

is preempted and returned to the queue.

Leave Sick

Schedule

Recovery

Worker Busy?

Yes

No

Cancel Pending

End Task Event

Premptive Scheduling

Calculate Time To

Mend (TTM)

Recover

End

Calculate Time To

Sickness (TTS)

Schedule

Sickness

Machine

Down?

Yes

NoWorker Arrival



Figure 14 - Machine Breakdown and Repair Flowchart

4.3 Dispatching Rules & Scheduling Policies

Scheduling policies (in no particular order):

Workers can only work on one task at a time.

Preemption of tasks is allowed in case of worker sickness or computer breakdowns.

Dispatching rules (in no particular order):

Workers always choose the task that has least work remaining.

Workers may commence work on tasks from subsequent pipelines in ascending order

if no tasks are pending for the current pipeline. This results in early start on pipelines.

Tasks due for rework because it did not pass embedded QA are commenced

immediately if possible, otherwise they are queued as normal tasks.

Environment asset jobs are shuffled and dispatched in pseudo-random order

(randomized with the same seed every time it is simulated).

Machine Arrival

Schedule Machine

Departure

Machine

Departure

End

Calculate TTB

(Time To

Breakdown)

Calculate TTR

(Time To Repair)

Schedule Machine

Arrival

Worker Busy?

Preempt Task

YesNo

Worker Busy

or Sick?

YesNo

Worker Arrival



Work on level assets commences only after 50% of all other assets in the pipeline are

created.

4.4 Stochastic Variables and Distributions

Four different stochastic variables have been introduced in the model and can be turned on

and off in the simulation program to be studied individually or in any combination.

Identifier Description

α Scale parameter for Weibull distribution

β Shape parameter for Weibull distribution

γ Location parameter for Weibull distribution

Γ The gamma function

τ The standard task duration

TTB Time to computer breakdown

TTR Time to repair

TTS Time to sickness

TTM Time to mend

TTEQA Time to embedded quality assurance

4.4.1 Task Duration

A three-parameter inverse Weibull(α,β,γ) distribution is applied to simulate task durations.

The gamma parameter (γ) denotes the minimum duration and is set to half the standard task

duration. The shape parameter is set to β=4.5 while the scale parameter, α, is calculated by

use of the equation:

( ) ( )

(Gusmão et al., 2011)

and rearranging it to get α corresponding with the expected value of the Weibull distribution

for the standard task duration, τ:



Equation 1 – Calculation of Weibull α so it corresponds with the expected value of the standard task duration

( )

( )

4.4.2 Machine Breakdowns

The time before a breakdown occurs (TTB) is modeled using a uniform distribution set at any

time between 0 and 400 standard days: Uniform(0,400). The time it takes to repair or replace

the machine (TTR) is modeled with a triangular distribution with Min=1 std. day, Max=5 std.

days, and Mode=2 std. days: Triangular(1,5,2).

4.4.3 Worker Absenteeism

Worker absenteeism is modeled with triangular distributions for the time before they become

sick and absent (TTS) and the time they need to mend and arrive at work again (TTM).

TTS=Triangular(1,30,20) and TTR=Triangular(1,10,5). These numbers are not entirely

realistic and seem to be in the high end when it comes to sick days. Barmby et al. (1991)

describes worker absenteeism empirically.

4.4.4 Embedded QA

Embedded QA is simulated by evaluating each of the immediate required sub-assets at

TTEQA. There is a 95% probability that the sub-asset is passing embedded QA and 5%

probability that the sub-asset is found to be flawed in some way and needs rework done.

Subsequently, a uniform distribution is used to determine the revised completion level at

[20%; 90%] for each non-passing sub-asset. The non-passing sub-asset is queued with high

priority, and based on the revised completion level the task duration is recalculated as rework

is commenced.



4.5 Objective

The objective of the simulation model is to record the total schedule duration and accumulate

work time and idle time for aggregates of workers in single and multi-pipeline settings using

given input variables and risk factors while following the defined scheduling policies and

dispatching rules of the model.


n Is the number of workers

w Is the individual worker

Tw Is the accumulated work time of a worker, w

Iw Is the accumulated idle time of a worker, w

∑ ( )

4.6 Short-comings of the model

If this model was meant to approximate a real-world pipeline it would come up short.

However, since the model is made only to conclude broadly on the combined impact of

certain risk factors on schedule duration, the model is adequate. Even if real-world game

asset pipelines were miles away from the stylized model it would still serve its purpose.

Granted, many aspects could be done more accurately which would result in more precise

numbers, although it is doubtful that added precision could have any effect on the

conclusions made in this thesis. Still conclusion based on the results from simulating this

model should be made with caution.

A shortlist of things that could be done to improve the model is presented below:

The model does not take into account the fact that there are many other activities in

the production games than just the asset pipeline. Activities such as game design,

engine and tool programming can have severe impact on the pipeline but is not

modeled. Mini-projects like sound and voice-over productions, and are essentially

also assets but are often outsourced and are not modeled. Concept art and game



design are not modeled. The model only covers the physical asset creation. A large

portion of the workers’ time is spent on meetings and information exchange, but this

is not modeled. Basically a huge part of the game production process, not necessarily

part of the pipeline but definitely affecting it, is not modeled.

The embedded QA assumes that the worker can continue work on the current task

while rework is done on the sub-assets. This is not always the case in reality where

workers might have to wait for rework to be done for the sub-asset to be usable in the

current task.

Days are not modeled, so the model does not take into account that days vary; that

workers in the real world do not show up on time every day and that they do not leave

work exactly at office closing hours. The simulation results must be translated into

standard measures of hours, days or months which are not true to real world.

In the real world, if a worker has preempted a task from the day before, it usually

takes some time to get started again. This is not modeled but assumed to be covered in

the standard task duration.

Naturally the output is highly impacted by the number of resources assigned. The

model does not optimize resources.

The model does not portray multiple pipelines realistically since the relationship with

game designers and developers are not incorporated into the model.

In the real world it may not be possible to start pipeline productions as early as the

model allows because of other organizational dependencies such as game designers

and developers, or external dependencies, such as publishing or outsourcing contracts.

5 Experimental Design

The impact of the risk factors, task duration variability, worker absence, machine

breakdowns, and failure to meet asset quality requirements, will be studied in single and dual-

pipeline production settings focusing on evaluation of schedule duration robustness.

Baselines for single and dual-pipeline settings are simulated deterministically using common

scheduling policies involving no risk factors.



Five different scenarios will be simulated by use of dynamic rescheduling and

compared against the baseline. In each scenario two consecutive and overlapping pipelines

will be simulated a thousand times and resulting durations for one and two pipelines will be

recorded for a comparison of robustness. Robustness of pipeline durations (or deadlines) will

be measured as percentiles of the resulting simulated distributions. In the first four scenarios

the impact of just one of the four risk factors each will be studied. The last scenario will

simulate all risk factors at once.

5.1 Input Variables

The input variables, resources, asset requirements and task durations, will be the same for the

baseline and all scenarios.

5.1.1 Resources

The resources that go into the model represent a fairly large pipeline production team. Most

pipeline production teams in the real world are smaller than this, but one can also find teams

twice this size or larger.

Table 3 - Production Team

Workers Count

Character Artists 4

Environment Artists 15

Animators 5

Level Designers 22

Technical Artists 4

Scripters 5

VFX Artists 5

Total Workers 60

5.1.2 Asset Requirements

The number of assets going into the model is chosen somewhat arbitrarily to represent

enough work for the production team to last more than a year.



Table 4 - Game Asset Requirements

Assets Count

Character Assets 50

Simple Environment Assets 250

Complex Environment Assets 50

Simple Animated Environment Assets 125

Complex Animated Environment Assets 25

Levels 25

Total Assets 525

5.1.3 Standard Task Durations

There is no formula to precisely come up with standard task durations and in the real world

these would be estimated from the work done in the pre-production phase.

Table 5 - Standard Task Durations

Task Character SEA CEA SAEA CAEA Level

Modeling 3 1 2 1 2

Mapping 2 1 2 1 2

Texturing 3 1 2 1 2

Rigging 1 1

Animation 10 1 2

VFX 5 1 2 5

Scripting 5 2 2 2 2 30

Tech Art 1 1 1 1 1 1

Level Design 60

6 Results

In the first subsection the baseline will be established and analyzed. In each of the following

scenario subsections the impact of risk factors will be isolated by subtracting the baseline

from the simulated mean of a thousand runs. Moreover, robustness metrics will be calculated

as percentile ranks of the simulated distributions. In this way the manager can evaluate the

robustness of a chosen slack percentage but can also pick a robustness measure (percentile



rank) to see what slack percentage would be needed. This robustness metric is chosen

because it is simple and intuitive as opposed to other more elaborate metrics. The scenario

results will focus on comparing the resulting robustness measure of 25% slack and the slack

percentage needed to meet a robustness of 99%. Other results are reported in the tables but

not commented on.

6.1 Definitions


BP1 and BP2 Is the baseline for the first and second dependent pipelines

TP1 and TP2 Is the first and second dependent pipelines dynamically rescheduled with task duration variability in

1000 runs.

WP1 and WP2 The first and second dependent pipelines dynamically rescheduled with worker absence in 1000 runs.

MP1 and MP2 The first and second dependent pipelines dynamically rescheduled with machine breakdowns in 1000

runs.

QP1 and QP2 The first and second dependent pipelines dynamically rescheduled with embedded QA in 1000 runs.

AP1 and AP2 The first and second dependent pipelines dynamically rescheduled with all risk factors enabled in 1000

runs.

6.2 Baseline Analysis

The baseline (predictive schedule) is established by running the simulation model

deterministically with fixed standard task durations, no machine breakdowns, no worker

absence and no embedded QA. Baseline figures are presented for the aggregate of all workers

and for each specialized worker group. The baseline is simulated without slack. Note,

however, that slack is present within some worker groups in the form of lag time due to the

modeled workflow interdependencies and non-optimized resource configurations.

Before establishing the baseline to compare against, it is worth having a look at the

development of individual pipeline durations when executed consecutively - with and without

overlaps. The non-overlapping execution of pipelines simply starts a new production pipeline

when the former is finished and the duration is the same for every individual which is

expressed as constant average pipeline duration. From the graph below it is evident that

individual pipeline durations with overlap become longer and longer and with increasing

overlap. This is because workers are allowed to start jobs from the next pipeline if jobs are

not available for the current. In other words they are allowed to start as early as possible.



Individual pipeline durations are increased dramatically by this, but at the same time total

duration is decreased. If instead workers were forced to a late start, individual pipeline

durations would be lower while total duration would stay the same provided that the late start

time were chosen carefully. However, the average pipeline duration based on the total

duration is reduced, and since the focus of this thesis is on deadlines not on individual

schedules, no effort is made to change the model.

Figure 15 –Individual Duration for 8 Non-Overlapping

Consecutive Pipelines

Figure 16 – Average Pipeline Duration

Figure 17 –Individual Duration for 8 Overlapping

Consecutive Pipelines

Figure 18 – Average Pipeline Duration

By splitting the total duration into Work, Idle Lead, Idle Between and Idle Lag times

it becomes obvious that there is a tremendous amount of time where workers do not work.

This can also be seen from the work ratio graph. Only slightly above a third of the total time

is actual Work time. In a real project, planners would have to pay attention to this and

optimize where possible. However, that is not done for this thesis.



The low Idle Lead means that workers spend little time idling before starting their

first task. Note, however, that Idle Lead is only calculated for the first pipeline executed

which makes the proportion of the whole duration smaller when more pipelines are executed

consecutively. High levels of Idle Lag, means that workers spend much time idling after

finishing their last task up until end of total duration. Idle Between is high, partly because of

the workflow and resource assignments and partly because of the early start on subsequent

pipelines policy. Of course, late start on subsequent pipelines would only reduce Idle

Between if at the same time Idle Lead was calculated for every subsequent pipeline as it

would subtract from the Idle Between.

Figure 19 –Total Duration Decomposition

Figure 20 – Work Ratio

6.2.1 Baseline – Level 1

Going forward, the total duration will be used as baseline for comparison. The focus will now

be on pipeline executions of single and dual pipelines respectively, as this is where the largest

difference is found.



Figure 21 –Baseline (Total Duration Split)

When comparing BP1 with BP2, there should of course be twice as much work to be

done when two identical pipelines are processed as opposed to only one. And the Idle Lead

should not change since Idle Lead is calculated for the whole duration, i.e. Idle Lead is not

calculated separately for successive pipelines.

The schedule duration ratio is less than 2:1 which is expected since overlapping

pipelines are allowed, and there is idle time to be replaced with work time in the first

pipeline. It is worth noticing the massive ratio for idle time which will be discussed at a later

point.

Table 6 – Baseline –Worker Average (in standard days)

6.2.2 Baseline – Level 2

Going one level deeper, to see how each of the worker groups compare, differences among

the groups are revealed. The group-based baselines show consistent numbers but also

individual differences from the overall baseline.

Workers (60) BP1 BP2 Difference Ratio

Work 133.75 267.50 133.75 2.0000

Idle Lead 48.82 48.82 0.00 1.0000

Idle Between 23.62 175.22 151.60 7.4192

Idle Lag 191.82 274.47 82.65 1.4309

Duration 398.00 766.00 368.00 1.9246



Large amounts of Idle Lead, as can be seen with level designers, offer the manager

the opportunity for late hiring of resources to save costs. Character and environment artists as

well as animators essentially display the same pattern. Idle Lead and Idle Between times are

short or zero, but all have large chunks of Idle Lag time, which should prompt the manager to

either reallocate or get rid of the resources when their work is done. In fact, large amounts of

Idle Lag time in general, as can be seen with all groups but scripters and tech artists, can be

interpreted as an opportunity for reallocating or getting rid of the resources after they have

ended their work. Otherwise it will just be an unnecessary cost.

Table 7 – Baseline - Character Artist Average (in standard days)

Table 8 – Baseline - Environment Artist Average (in standard days)

Table 9 – Baseline – Animator Average (in standard days)

Character Artists (4) BP1 BP2 Difference Ratio

Work 100.00 200.00 100.00 2.0000

Idle Lead 0.00 0.00 0.00 1.0000

Idle Between 0.00 0.00 0.00 1.0000

Idle Lag 298.00 566.00 268.00 1.8993

Duration 398.00 766.00 368.00 1.9246

Env. Artists (15) BP1 BP2 Difference Ratio

Work 150.00 300.00 150.00 2.0000

Idle Lead 0.00 0.00 0.00 1.0000

Idle Between 0.00 0.00 0.00 1.0000

Idle Lag 248.00 466.00 218.00 1.8790

Duration 398.00 766.00 368.00 1.9246

Animators (5) BP1 BP2 Difference Ratio

Work 145.00 290.00 145.00 2.0000

Idle Lead 2.60 2.60 0.00 1.0000

Idle Between 4.00 8.80 4.80 2.2000

Idle Lag 246.40 464.60 218.20 1.8856

Duration 398.00 766.00 368.00 1.9246



Large amounts of idle time may indicate excess resources or poorly designed or

synchronized workflows. Most notably there is an extreme increase in Idle Between time for

level artists as well as VFX artists. In BP2 the some of the Idle Lag time from the first

pipeline execution carries over as Idle Between for the second.

Table 10 – Baseline - VFX Artists (in standard days per worker)

The level of utilization of scripters is very high which should prompt the manager to

consider whether to hire more scripters.

Table 11 – Baseline – Scripters (in standard days per worker)

Table 12 – Baseline - Level Designers (in standard days per worker)

VFX Artists (4) BP1 BP2 Difference Ratio

Work 181.25 362.50 181.25 2.0000

Idle Lead 6.00 6.00 0.00 1.0000

Idle Between 5.75 198.50 192.75 34.5217

Idle Lag 205.00 199.00 -6.00 0.9707

Duration 398.00 766.00 368.00 1.9246

Scripters (5) BP1 BP2 Difference Ratio

Work 380.00 760.00 380.00 2.0000

Idle Lead 4.40 4.40 0.00 1.0000

Idle Between 0.40 0.40 0.00 1.0000

Idle Lag 13.20 1.20 -12.00 0.0909

Duration 398.00 766.00 368.00 1.9246

Level Designers (22) BP1 BP2 Difference Ratio

Work 68.18 136.36 68.18 2.0000

Idle Lead 129.00 129.00 0.00 1.0000

Idle Between 0.00 314.82 314.82 -

Idle Lag 200.82 185.82 -15.00 0.9253

Duration 398.00 766.00 368.00 1.9246



Tech Artists have high Idle Between time in BP1 clearly indicating excess resources.

Table 13 – Baseline - Tech Artists (in standard days per worker)

Naturally it would be possible to go one level deeper and extract numbers for each

individual worker which would offer perfectly detailed information for optimization of

resource allocation. However, as this thesis is not trying to optimize the schedule this kind of

detail would produce an excessive amount of data and would not add anything to the

conclusions and therefore is left out.

6.3 Scenario I – Deadline Robustness with Task Duration Variability

Task duration variability has direct impact on work time and indirectly impacts idle time and

the total schedule duration.

Histograms are presented to make the distributions more comprehensible. By visual

inspection of the histograms for TP1 and TP2 it seems clear that both distributions are

skewed to the right with rather high spread and a few outliers.

Tech Artists (5) BP1 BP2 Difference Ratio

Work 105.00 210.00 105.00 2.0000

Idle Lead 6.40 6.40 0.00 1.0000

Idle Between 274.40 549.40 275.00 2.0022

Idle Lag 12.20 0.20 -12.00 0.0164

Duration 398.00 766.00 368.00 1.9246



Figure 22 –Distribution of Total Duration (TP1)

Figure 23 – Distribution of Total Duration (TP2)

The robustness metric for TP1 shows that with 25% slack achieves robustness

(percentile rank) of 99.6%. A robustness of 99% requires only 17.99% slack. The robustness

metric for TP2 reveals that the same 25% slack achieves robustness of 99.9%, while a 99%

target for robustness requires only 11.02% slack.

Table 14 – Duration, Slack and Rank – Task Duration Variability (duration in standard days)

Duration Slack Rank Duration Slack Rank

Baseline 398.00 0.00% 35% 766.00 0.00% 20%

Baseline +25% Slack 497.50 25.00% 99.60% 957.50 25.00% 99.90%

Baseline +35% Slack 537.30 35.00% #N/A 1034.10 35.00% #N/A

Baseline +50% Slack 597 50.00% #N/A 1149 50.00% #N/A

90th percentile 427.60 7.44% 90.00% 810.29 5.78% 90.00%

95th percentile 440.42 10.66% 95.00% 822.94 7.43% 95.00%

99th percentile 469.59 17.99% 99.00% 850.42 11.02% 99.00%

Min 364.09 -8.52% 0.00% 737.46 -3.73% 0.00%

Max 517.83 30.11% 100.00% 1016.81 32.74% 100.00%

Mean 406.28 2.08% 58.20% 783.80 2.32% 54.30%

Mode 405.42 1.86% 55.90% 778.89 1.68% 45.60%

Range 153.74 38.63% 279.35 36.47%

Std. dev. 18.43 22.72

Kurtosis 5.3430 12.1438

Skewness 1.6252 1.8736

Task Duration Variability (Std. Days)

TP1 TP2



6.4 Scenario II – Duration Robustness with Machine Breakdowns

Machine breakdowns have a direct impact on non-work time. The distributions look quite

similar; slightly skewed to the right and with very little spread.

Figure 24 –Distribution of Total Duration (MP1)

Figure 25 – Distribution of Total Duration (MP2)

The robustness metric for MP1 shows that 25% slack is beyond the percentile ranks,

i.e. higher than the maximum. This is clearly a drawback of using this metric. A robustness of

99% requires only 2.36% slack. The robustness metric for MP2 again reveals that 25% slack

is beyond the metric. A target for robustness of 99% requires only 2.40% slack.

Clearly, machine breakdowns have very little impact on schedule duration.



Table 15 – Duration, Slack and Rank – Machine Breakdowns (duration in standard days)

6.5 Scenario III – Deadline Robustness with Worker Absence

By visual inspection the distribution for WP1 seems fairly normal distributed. WP2 seems

slightly skewed to the left. Both seem to have reasonably low spread.

Figure 26 –Distribution of Total Duration (WP1)

Figure 27 – Distribution of Total Duration (WP2)


Baseline 398.00 0.00% #N/A 766.00 0.00% #N/A




90th percentile 405.26 1.82% 90.00% 780.80 1.93% 90.00%

95th percentile 406.13 2.04% 95.00% 782.18 2.11% 95.00%

99th percentile 407.40 2.36% 99.00% 784.36 2.40% 99.00%

Min 399.35 0.34% 0.00% 771.72 0.75% 0.00%

Max 410.61 3.17% 100.00% 789.13 3.02% 100.00%

Mean 403.01 1.26% 54.70% 777.74 1.53% 54.10%

Mode 403.76 1.45% 69.70% 776.79 1.41% 38.20%

Range 11.26 2.83% 17.40 2.27%

Std. dev. 1.67 2.38

Kurtosis 0.3464 0.85501

Skewness 0.5900 0.6910

Machine Breakdowns (Std. Days)

MP1 MP2



The robustness metric for WP1 shows that 25% slack ranks at 6.5% while robustness

of 99% requires 31.69% slack. The robustness metric for WP2 exposes 25% slack to be

below the minimum. A target for robustness of 99% requires 33.04% slack.

Table 16 – Duration, Slack and Rank – Worker Absence (duration in standard days)

6.6 Scenario IV – Deadline Robustness with Embedded QA

The distribution of QP1 is very right-skewed. QP2 is much less so. Both seem to have rather

high spread.


Baseline 398.00 0.00% #N/A 766.00 0.00% #N/A

Baseline +25% Slack 497.50 25.00% 3.40% 957.50 25.00% #N/A



90th percentile 521.00 30.90% 90.00% 1009.13 31.74% 90.00%

95th percentile 524.00 31.66% 95.00% 1012.67 32.20% 95.00%

99th percentile 530.68 33.34% 99.00% 1019.08 33.04% 99.00%

Min 487.10 22.39% 0.00% 965.47 26.04% 0.00%

Max 533.47 34.04% 100.00% 1026.54 34.01% 100.00%

Mean 511.44 28.50% 50.60% 997.01 30.16% 50.30%

Mode 513.23 28.95% 60.00% 995.64 29.98% 43.80%

Range 46.37 11.65% 61.07 7.97%

Std. dev. 7.57 9.43

Kurtosis 0.0956 -0.0246

Skewness 0.0792 -0.0191

Worker Absence (Std. Days)

WP1 WP2



Figure 28 –Distribution of Total Duration (QP1)

Figure 29 – Distribution of Total Duration (QP2)

The 25% slack level is beyond maximum in both QP1 and QP2. Robustness of 99%

requires 16.02% slack for QP1 and 11.81% slack for QP2.

Table 17 – Duration, Slack and Rank – Embedded QA


Baseline 398.00 0.00% #N/A 766.00 0.00% #N/A




90th percentile 446.47 12.18% 90.00% 839.06 9.54% 90.00%

95th percentile 453.29 13.89% 95.00% 844.55 10.26% 95.00%

99th percentile 460.30 15.65% 99.00% 856.46 11.81% 99.00%

Min 405.80 1.96% 0.00% 784.10 2.36% 0.00%

Max 469.30 17.91% 100.00% 872.92 13.96% 100.00%

Mean 423.69 6.45% 62.60% 818.80 6.89% 54.10%

Mode 413.93 4.00% 28.40% 805.74 5.19% 18.30%

Range 63.50 15.95% 88.82 11.60%

Std. dev. 13.72 14.63

Kurtosis 0.2936 0.115788

Skewness 1.0760 0.4112

Embedded QA (Std. Days)

QP1 QP2



6.7 Scenario V – Deadline Robustness with All Factors Combined

By visual inspection of the histograms it seems that the samples are fairly normal distributed

and ever so slightly right-skewed and with rather high spread.

Figure 30 –Distribution of Total Duration (AP1)

Figure 31 – Distribution of Total Duration (AP2)

Table 18 – Duration, Slack and Rank – All Uncertainty Factors (duration in standard days)


Baseline 398.00 0.00% #N/A 766.00 0.00% #N/A

Baseline +25% Slack 497.50 25.00% 0.90% 957.50 25.00% #N/A

Baseline +35% Slack 537.30 35.00% 61.90% 1034.10 35.00% 50.60%


90th percentile 555.02 39.45% 90.00% 1062.31 38.68% 90.00%

95th percentile 561.35 41.04% 95.00% 1068.36 39.47% 95.00%

99th percentile 575.82 44.68% 99.00% 1081.28 41.16% 99.00%

Min 490.48 23.24% 0.00% 977.76 27.64% 0.00%

Max 588.26 47.80% 100.00% 1095.36 43.00% 100.00%

Mean 532.80 33.87% 51.20% 1034.67 35.07% 51.50%

Mode 536.55 34.81% 60.80% 1032.76 34.83% 47.00%

Range 97.78 24.57% 117.60 15.35%

Std. dev. 16.82 20.26

Kurtosis 0.0092 -0.205503

Skewness 0.2898 0.1298

AP2AP1

All Factors (Std. Days)



AP1 shows a duration target range of 129.23 standard days while AP2 has a range of

only 117.60. The biggest impact comes from task duration variability.

Interestingly, in terms of slack percentage the range for AP1 is [19.60%; 52.07%]

which is quite a bit wider than the range for AP2 which is [27.64%; 43.00%]. Therefore

smaller changes in the slack percentage are needed for AP2 to cause the same rank change.

This means that overall higher robustness can be achieved by having two consecutive and

overlapping pipelines rather than one.

Using a standard of 25% slack added to the baseline is not enough to reach the 99%

percentile when applying the risk factors as defined for the model. Naturally these will not be

the same in real applications. The actual numbers here are not as important as the method

proposed. Managers and producers must model their actual pipelines and use risk factors that

fit their situation. Also they must choose the percentile that they feel comfortable with.

Figure 32 –Plot of Percentile Rank vs. Slack (AP1)

Figure 33 – Plot of Percentile Rank vs. Slack (AP2)

Calculations using distribution percentile ranks allows the producer and manager to

either evaluate the robustness of a selected slack percentage or pick a percentile rank and

calculate the needed slack percentage. The mean of one pipeline in AP2 produces the same

ranks and slack percentage although the range and standard deviation is halved.

The standard deviation for task duration variability is quite high compared with the

other factors.



7 Validation

The simulation program has undergone extensive debugging and heuristic testing including

validation of output in Excel and SPSS by use of checkpoint calculations and visual

inspection of variable distributions. Furthermore validation of the simulation output has been

done by creation of a simple meta model through linear regression.

All tests have shown output data to be consistent and valid for the model. Even if the

model itself, the model structure and input variables, cannot be validated empirically for

obvious reasons, and comparison with the baseline may not be statistically sound due to the

low number of simulation runs (1000), the results presented are valid for the purpose of this

thesis.

8 Conclusion

The results show that each risk factor affects the total schedule duration to some extent so on

average schedules have longer durations than the baseline. Still it is no trivial matter to assess

the impact beforehand when risk factors are combined. By simulation it is possible to capture

the dynamic nature of the production pipeline model and gauge the results of combinations of

risk factors that may exercise overlapping effects.

Simulation appears to be a strong tool for gaining valuable and detailed information

about how a milestone deadline is impacted by combinations of various risk factors. Even if

this thesis only analyzes the effect on deadlines the simulation program may store

distributions of many other variables that could potentially be analyzed as well to support the

decision-making process. The simulated distributions and robustness metrics for deadlines

allow managers and planners to jointly engage scheduling at a more informed level and may

help in defining realistic deadlines in real production settings.

Even though simulation may be a very useful tool for defining realistic deadlines in

games productions, the initial cost of setting them up can be high. Especially, simulation

modeling and programming in a general-purpose language is a time consuming endeavor. So

before taking on simulation as a support-tool for decision-making a cost-benefit analysis

should be completed.



Based on the experience with modeling and programming the simulation developed

for this thesis it does seem very plausible that a model with high enough detail to capture all

relevant dynamics of the game production could be developed. Obviously, the simulation

model must exhibit a close fit with the actual production system in order for the results to be

useful in decision-making. Exactly how close a fit is needed is yet unknown. This is

definitely something that could be interesting to test in future studies.

9 Further Studies

There are mainly three things that would be interesting to do in the future:

1. Enhance the model to include costs.

2. Expand the model to encompass the whole games development process.

3. Detail the model to fit an existing production pipeline and run empirical tests.

Incorporating costs in the model should be fairly simple, but interesting nonetheless.

Creating a model for the whole game development process is somewhat more

involving but the benefit from doing so possibly outweighs the effort.

One would be required to look into an actual production pipeline using empirical data

in order to create a highly detailed simulation model, taking into account all conceivable risks

and uncertainties, to approximate a good fit. The model should include elements like

information flow, learning curve, worker morale, crunch-time, over-time, etc. and be vary to

match real workflows, in order to capture enough information for reliable simulation results.

Simulation of actual days using a calendar to map workers’ scheduled time off and an

empirical model for sickness would be needed as well as simulation of scheduled breaks and

micro-breaks and startup times. Dynamic resource allocation for workers leaving, being laid

off or hired, partial re-use of asset from other productions – the list goes on. Having studied

all these elements it is possible that findings will show that some of these elements are

negligible and approximations can be found to create a good model fit with the real world

without too many variables in the simulation.



10 References

AUTODESK. 2012. Modern Pipeline [Online]. Available: http://usa.autodesk.com/adsk/servlet/index?id=16116742&siteID=123112 [Accessed April 24, 2012].

BARMBY, T. A., ORME, C. D. & TREBLE, J. G. 1991. WORKER ABSENTEEISM: AN ANALYSIS USING MICRODATA. Economic Journal, 101, 214-229.

CARTER, B. 2004. The Game Asset Pipeline, Delmar Cengage Learning. CHANDLER, H. M. 2010. The Game Production Handbook, Jones and Bartlett Publishers. GUSMÃO, F., ORTEGA, E. & CORDEIRO, G. 2011. The generalized inverse Weibull distribution.

Statistical Papers, 52, 591-619. HERROELEN, W. & LEUS, R. 2004. Robust and reactive project scheduling: a review and classification

of procedures. International Journal of Production Research, 42, 1599-1620. HERROELEN, W. & LEUS, R. 2005. Project scheduling under uncertainty: Survey and research

potentials. European Journal of Operational Research, 165, 289-306. HULETT, D. 2009. Practical Schedule Risk Analysis, Gower Publishing Company. LAW, A. 2007. Simulation Modeling and Analysis, McGraw-Hill Higher Education. MATSUMOTO, M. 1998. Mersenne Twister: A 623-Dimensionally Equidistributed Uniform Pseudo-

Random Number Generator. ACM Transactions on Modeling and Computer Simulation, 8, 3-30.

MSDN. 2010. Introduction to the C# Language and the .NET Framework [Online]. Microsoft. Available: http://msdn.microsoft.com/en-us/library/z1zx9t92.aspx [Accessed May 1, 2012].

RUBINSTEIN, R. Y. & KROESE, D. P. 2008. Simulation and the Monte Carlo Method, John Wiley & Sons.

VIEIRA, G. E., HERRMANN, J. W. & LIN, E. 2003. Rescheduling Manufacturing Systems: A Framework of Strategies, Policies, and Methods. Journal of Scheduling, 6, 39-62.

WIKIPEDIA. 2012. Polymorphism in object-oriented programming [Online]. Available: http://en.wikipedia.org/wiki/Polymorphism_in_object-oriented_programming [Accessed April 30, 2012].

http://usa.autodesk.com/adsk/servlet/index?id=16116742&siteID=123112

http://msdn.microsoft.com/en-us/library/z1zx9t92.aspx

http://en.wikipedia.org/wiki/Polymorphism_in_object-oriented_programming

Simulation Modeling of a Stylized Game Asset Pipeline...

Documents

Transcript of Simulation Modeling of a Stylized Game Asset Pipeline...