IntroductionIntroduction What is this ?What is this ?

This project is a part of a scientific research in machine This project is a part of a scientific research in machine learning, whose objective is to develop a system, based on the learning, whose objective is to develop a system, based on the existent machine learning algorithms, which will learn to play existent machine learning algorithms, which will learn to play foosball (table soccer game).foosball (table soccer game).

We present now the first part of the project, which does not We present now the first part of the project, which does not deal with machine learning at all. This software is a simulator, deal with machine learning at all. This software is a simulator, modeling the foosball environment. It consists of:modeling the foosball environment. It consists of:

The physical model of the game components (field, ball, players, The physical model of the game components (field, ball, players, etc.. )etc.. )

The graphical user interface (GUI) for a visual representation.The graphical user interface (GUI) for a visual representation. The framework for an agent.The framework for an agent.

The first two components are combined into a standalone The first two components are combined into a standalone application that can be used as an engine for development of application that can be used as an engine for development of different foosball agents.different foosball agents.

The Robotic FoosballThe Robotic Foosball

PlantDynamics Graphics

Data BankAgent

State

Data

Sta

te

Act

ion

m

Dynamics – The physics of the application.

Graphics – The visualization of the dynamics.

Agent – The brains of the application.

Data Bank – The data storage of the application.

In the real physical system the “Dynamics” and the “Graphics” are unified as the “Plant”.

The Project ObjectiveThe Project Objective

The objective of this project is to build the software simulation of the Plant and provide an interface for the agent.

The software simulation is the environment, which serves as the lab for the development of the agent. It is an idealized model of the real foosball table. The simulation consists of 2 independent modules, which communicate using a well-defined protocol.

1. Dynamics - Simulates the physical behavior of the foosball:

i. The motion of the ball and the players.

ii. The collisions between the ball and the players or the table borders.

2. Graphics – The graphic representation of all Foosball elements: the table, the ball and the players.

Agent

Clk

Plant

StateInstruction

Agent – Plant InterfaceAgent – Plant Interface

Time

Stam

p

The Environment State :Locations and velocities of all the axes, location and velocity of the ball at given time.

The Instruction issued by the agent to the foosball model:The desired location and the velocity of a certain axis at a given time.

The Fulfillment of an The Fulfillment of an InstructionInstruction

V

tt1 t2 T

V 0

V F

V m

a1 a2

The simulation works with accelerations as inputs.

)(taNeed to calculate:Given: X0, V0, XF, VF, T

Every requirement may be satisfied in this way:

The maximum velocity and acceleration are limited:

Vm <= Vmax ai<= amax

Generally, there is no need to reach Vmax , while executing the instruction.

The Fulfillment of an The Fulfillment of an InstructionInstruction

X

t

V

t

a

tt1 t

a1

a2

V1Vf

V0

X0

Xf

X1

t2

Thus, the implementation may be simplified in the following way:

If T is an odd number of clock cycles:

a1 and a2 are slightly different.

The Graphics module DescriptionThe Graphics module Description

y

x

The coordinates convention is: • X-axis – Vertical direction• Y-axis – Horizontal direction• Z-axis – Out of page plane direction

The Graphics module DescriptionThe Graphics module Description

The players are drawn using the The players are drawn using the following method:following method:

A player is drawn using several rectangles for A player is drawn using several rectangles for body parts, and a circle as a head.body parts, and a circle as a head.

The angle of rotation determines which parts The angle of rotation determines which parts of the player are visible.of the player are visible.

The visible parts are drawn after the invisible The visible parts are drawn after the invisible ones.ones.

Since all the players in a set are rotated Since all the players in a set are rotated simultaneously, the calculations are made simultaneously, the calculations are made only once. The rest of the players are simply only once. The rest of the players are simply shifted along the X-axis.shifted along the X-axis.

See the calculations belowSee the calculations below

The Graphics module DescriptionThe Graphics module Description

The original vector to be rotated by f about X-axis (orthogonal to the page plane).

The result of the rotation.

R

sinR

sinR

R

The Graphics module The Graphics module DescriptionDescription

In our graphic representation we are interested only in the 2D

projection of the player on the XY plane, therefore we don’t need

to calculate the coordinate.

Obviously, the rotation matrix operator about X-axis is:

The Graphics module DescriptionThe Graphics module Description

The player model in the application is simplified, as follows:

The names on the figure are representing the constants, defined in “config.h” file.

* Head diameter is equal to feet width.

The Physics module DescriptionThe Physics module Description

The physical module is responsible for the game dynamics:

• For each “time slice” the next state is calculated, according to the current state and the applied accelerations.

• A state is defined by: the positions of the players, their velocities (linear and rotational), the position and the velocity of the ball.

• Collision detection between the ball and different table elements ( borders , players).

• Collision affects the current state.

1. The The ball does not roll.

2. The ball does not take off the table

3. Players have infinite mass relatively to the ball.

4. The maximum velocity and acceleration are limited.

5. A high constant friction is applied to players’ motion when there are no acceleration instructions. This reflects the behavior of a real system, where the motion stops very fast, after no power is applied.

6. The ball has adjustable elasticity parameter.

7. There is adjustable friction between the floor and the ball.

The simulation assumptions

The Physics module DescriptionThe Physics module Description

A player kicks the ball: Analysis of player’s tangent velocity:The tangent velocity of a player is derived from his angular velocity. Only the projection of the tangent velocity on the XY plane makes sense, because the ball moves in XY plane only.

X

Y

ZPlayer Velocity

XY velocity Projection

Side View

Ball Velocity

Player Velocity

Side View

The Physics module Description - The Physics module Description - Ball Collision Ball Collision MechanismMechanism

1. Calculate the ball velocity relatively to a table element.

2. Calculate the Normal between the Ball and the table element.

Where:

The Physics module Description - The Physics module Description - Ball Collision Ball Collision MechanismMechanism

Notes:

1. The table borders are treated as players with fixed locations and zero velocity. 2. The friction and the elasticity factor can be defined within the application window.

3. Find the projection of the relative velocity on the normal.

4. Find the velocity of the ball after the collision.

r - Elasticity Coefficient

The Physics module Description - The Physics module Description - Collision Collision DetectionDetection

Every time cycle the effective shape of each player is calculated.

Effective shape is the shape “seen” by the ball, which is derived from the rotational angle and the body parameters of the player.

• The ball collides with the effective shape of a player. • The shape is always a rectangle, because the players’ feet are cubical.• Example: if a player is upside down, there is no effective shape at all – the ball doesn’t collide the player.The Effective Shape object calculations:• The closest point of the effective shape to the ball.• The horizontal (XY plane) velocity of the player.

The ball object calculations:•Whether the ball will reach any effective shape – the collision detection.•If a collision detected, the collision mechanism is activated (as described above).

• The Agent Framework is a standalone application, which controls the players in the simulation.

•The agent and the simulation maintain asynchronous communication.

• The communication is implemented as inter-process Windows messages (on the same machine).

• Data received from the simulation: current game state.

• Data sent to the simulation: the desired state of the players

• Any real agent may be installed within the framework. Currently, the agent simply issues random instructions to the server.

The Agent Framework description

Known ProblemsKnown Problems

The collision model fails at extremely The collision model fails at extremely high velocities, which are beyond the high velocities, which are beyond the limit.limit.

The game state is not updated correctly The game state is not updated correctly for extremely short time cycles.for extremely short time cycles.

The agents instructions are applied The agents instructions are applied only to one simulation, even though only to one simulation, even though several simulations run on the same several simulations run on the same machine. machine.

Future EnhancementsFuture Enhancements

Better collision modelBetter collision model Three dimensional game – the ability of Three dimensional game – the ability of

the ball to take off the table.the ball to take off the table. Take in.to account the ball spinTake in.to account the ball spin Provide more flexibility of the Provide more flexibility of the

simulation parameters.simulation parameters. Optimize the performance.Optimize the performance. Port the simulation to another Port the simulation to another

platforms.platforms.

Tools used The software is developed on Microsoft

Visual C++ platform for MS Windows. Fully object-oriented. MFC tools and libraries are used (mainly -

for GUI) Tested operating systems: Windows

2000/NT/XP