ARTIFICIAL NEURAL NETWORK APPLICATION

X. Cai C. Fang R. Guo W. Yu B. Qian S.Jiang

CSC7333 Machine Learning Project Presentation

April.30th, 2013

ARTIFICIAL NEURAL NETWORK APPLICATIONLEARNING TO PLAY THE GAME OF SUPER MARINO

Super Mario Bros is a 1985 platform video game developed by Nintendo, published for the Nintendo Entertainment System as a pseudo-sequel to the 1983 game Mario Bros.

introductionmethodology

experimentdiscussion



Mario AI Championship 2012 The 2012 Mario AI Championship, the successor to the

successful 2011 and 2010 Mario AI Competition, run in association with several major international conferences focusing on computational intelligence and games.



Our project gets inspiration from the competition:

The goal is to develop a best controller(agent) to play the game.

By using artificial neural network.



Method: artificial neural network.



Neural networks are used for modeling complex relationships between inputs and outputs or to find patterns in data.

Noise and error Tolerance. ANN can perform tasks that a linear program can not. When an element of the neural network fails, it can

continue without any problem by their parallel nature.



Design an ANN approach

By learning the environment around Marino, the controller's job is to win as many levels as possible. Each time step the controller has to decide what action to take (left, right, jump etc).

introduction

methodologyexperimentdiscussion

The linear world of Mario is filled with enemies, obstacles and power-ups.

The problem the agent faces when playing a game is how to correctly interpret what it sees in the environment and decide on the best action to take.

Input & Output:

introduction


Data set = input + output (one frame one data set)

Every frame our script will scan Mario code and extract environment matrix as well as keyboard action one time and obtain one data set. For example, A3 minutes monitoring of playing the game, we can obtain 3*60*24=4320 data sets.

Input=environment matrix + mario position

Output=keyboard action(0,1)Left, right, up, down, shoot, jump

0 0 0 0 0 1this is a jump action

introduction


introduction


Monitor data training:

introduction


Encapsulate pattern to Java Agent Encapsulate these patterns to Java style agents, so that

it can be used in Super Mario source code.

introduction


Test ANN Agents Import ANN Java library to Super Mario code.

Insert ANN agent for testing

introduction




First experiment : 13*19 + 1 + 6 13*19 : environment around Mario

1 : Mario’s state

6 : Human keyboard action (up, down, left, run, jump, fire)

Doesn’t work well: input data is too large



Second experiment : 10*10 + 1 + 2 10*10 : reduce the number of environment data

1 : Mario’s state

2 : Human keyboard action, but separate run & jump

Doesn’t work well: different for ‘run’ & ‘jump’, but Mario sometimes needs to run and jump all together.



Third experiment : 10*10 + 1 + 2 10*10 : keep the environment data small but effective

1 : Mario’s state

2 : merge run & jump together

Works better but not satisfied: too much data noise due to human’s playing training data



Fourth experiment : 10*10 + 1 + 2 Realized that the process human playing the game as

training data is critical for the experiment .

With extra carefulness during human’s playing.


experiment

discussion

How to obtain the best input/output pair? Input, remove decorative redundancy. Output, a good human player is critical.

About hidden layer Tuning is the best way.

Is ANN the best way of designing Mario agent?

Not really. The major problem is we cannot obtain a sufficient and accuracy training data from human beings.

Reinforcement learning maybe better.

ARTIFICIAL NEURAL NETWORK APPLICATION

Documents

Transcript of ARTIFICIAL NEURAL NETWORK APPLICATION