SWARMING PIXEL TRACKER - Parth Kanungo is certified that the work contained in the B.Tech. Project...
Transcript of SWARMING PIXEL TRACKER - Parth Kanungo is certified that the work contained in the B.Tech. Project...
SWARMING PIXEL TRACKER
B.Tech Project submitted by
Parth Kanungo
Computers and Communication Engineering,
The LNM Institute of Information Technology, Jaipur
2011
CERTIFICATE It is certified that the work contained in the B.Tech. Project entitled “Swarming Pixel Tracker” by Parth Kanungo (Y07uc078) has been carried out under our supervision and that this work has not been submitted elsewhere for a degree. May, 2011 ________________________ Soumitra Debnath, Assistant Professor, Communication & Computer Engg. The LNMIIT, Jaipur
________________________ Prithwijit Guha, Visiting Faculty,
Communication & Computer Engg. The LNMIIT, Jaipur
ABSTRACT
Tracking moving objects is a critical task in computer vision, with many practical applications such as vision based interface tasks, visual surveillance or perceptual intelligence applications. This project presents a method of visual tracking using a swarm intelligence metaphor. Swarm Intelligence refers to the collective intelligent emergent behaviour of simpler agents. This is a concept inspired by the social behaviour of insects. Each agent (or swarm particle) is "unintelligent" and follows a set of primitive rules like "stay close to a neighbour". The mutual interactions and movements make the entire system "intelligent" and robust for various applications. For object tracking, we have assigned a tracker (i.e. a swarm particle) to each pixel of the object read in the first frame. As the subsequent frames of video come by, the motion of the trackers (or the swarm particles) is governed by their neighbours and their respective confidence values. The interactions of individual particles results in an emergent object tracking behaviour.
ACKNOWLEDGEMENT
I would like to express my sincere gratitude to Prof. Prithwijit Guha for guiding and supporting
me throughout the duration of this project. I would also like to thank my friend Dushyant Goyal
for valuable discussions.
CONTENTS
1. Introduction to Swarms
a. Introduction
b. Mathematical Modelling of Swarm
2. Swarm Simulation
a. The Data Structure
b. Initialize a swarm
c. Step response of a swarm
d. Assigning goals to the swarm
e. Behaviour in a two-tank system
3. Application in Visual Tracking
a. Introduction
b. Mathematical Model
c. Results
4. Conclusions and Future Work
5. References
Chapter 1
Introduction to Swarms
Flocking birds
1.1 Introduction
Birds flock, ants forage, fish school. Evolution of creatures has produced swarming in so-many
contexts. Swarming has many individual and group benefits. For example – swarming provides
ability to forage more effectively. In case of birds, it reduces the overall air drag on the birds
giving them an ability to travel greater distances. Then there is a concept of safety associated as
well. Fishes in swarms are safer compared with when they are alone. The dynamics of predator-
prey algorithm are such that the predator gets confused when it sees multiple preys at the same
time, and is unable to attack.
This concept of swarms can be applied in computers to solve problems. In fact, many problems
are ideal candidates for swarm algorithms, including traveling salesman problems, data fusion on
distributed sensor networks, network load balancing, and the coordination of large numbers of
machines, e.g., cars on the highway or automatons in a factory.
Before going into the details, let us establish a concrete definition of a swarm and an autonomous
agent. An autonomous agent is an entity that is able to sense and affect its environment. It
interacts with its environment, which probably consists of other agents, but acts relatively
independently from all the other agents. It does not follow commands from a leader, or some
global plan. Such agent is referred to as a swarm-particle (or individual) and the collection of
these particles, which produces an emergent behaviour, is referred to as a swarm.
Using the swarms in computer science, we employ what is referred to as “swarm intelligence”, a
term first used by Beni, Hackwood, and Wang in regards to the self-organizing behaviours of
cellular robotic systems [1]. This interpretation of swarm intelligence focuses on unintelligent
agents with limited processing capabilities, but possessing behaviours that collectively are
intelligent. This behaviour can be applied in optimization in telecommunicate systems, robotics,
tracking patterns in transportation systems, and military applications.
Thus, Computer Scientists have used the concept of swarm to formulate various algorithms like
Particle Swarm Optimization and Ant colony Optimization. Then, there are BOIDs, which are
simulated flocking creatures created by Craig Reynold. They have a lot of advantages in
computer science – they are robust, simple agents, distributed & parallel and effort magnification
and scalability emerges in their behaviour. [2]
In this project, instead of using these standard algorithms, we follow a more primitive approach
of implementing basic motion rules to the swarm particles, as a result of which we get emergent
swarm behaviour.
1.2 Mathematical Modelling of Swarm
Swarm is an emergent behaviour arising from simple rules that are followed by particles (or
individuals) and does not involve any central coordination. The simplest mathematical models of
swarm follow the following rules:
1. Move in the same direction as your neighbours
2. Remain close to your neighbours
3. Avoid collisions with your neighbours
This can be made possible by implementing either of the two models as shown in the figure:
The Metric Distance Model consists of three zones around an animal. In the „zone of repulsion‟,
closest to the animal, the focal animal will seek to distance itself from its neighbours to avoid
collision. In the other zone „zone of alignment‟, the particles will move in the direction of the
neighbour. In the outermost zone, the animals attract each other, so as to make the swarm
cohesive.
In the other model known as the Topological Distance Model, the animal pays attention to only 6
or 7 animals closest to itself, regardless of the distance.
To end this section, I would like to quote some lines from the novel PREY written by the
Science Fiction author Micheal Chricton:
“In the last few years, artificial life had replaced artificial intelligence as a long-term computing
goal. The idea was to write programs that had the attributes of living creatures—the ability to
adapt, cooperate, learn, adjust to change. Many of those qualities were especially important in
robotics, and they were starting to be realized with distributed processing. Distributed
processing meant that you divided your work among several processors, or among a network of
virtual agents that you created in the computer. There were several basic ways this was done.
One way was to create a large population of fairly dumb agents that worked together to
accomplish a goal—just like a colony of ants worked together to accomplish a goal……The
emergent behavior is the sum of individual agent behaviors. There isn‟t anybody there to
„decide‟ anything. There‟s no brain, no higher control in that swarm.”[3]
Chapter 2
Swarm Simulation
2.1 The Data Structure
We implemented a data structure for the swarm and its individuals. The rules were assigned to the
individual swarm particles, we called CIndividual. (In this text, we shall use CIndividual and particle
interchangeably.)
In all the experiments, we made use of a single swarm with a swarm-id and an array of individuals
that comprised the swarm. The entire simulation was done using the OpenCV Library of C++ in
Ubuntu 9.10
CSwarm
{
int sName;
int* iArray[NUM]
}
CIndividual
{
int mID; // ID of the individual
int m_X, m_Y; // location on the image (x,y)
float m_vX, m_vY; //its velocity
int r,g,b; // the r,g,b components of its color
}
This is the primitive data structure. Depending on the specific goal of the program, some more
members (like a Confidence value or the distance_from_the_goal) shall be added to the CInidvidual
data structure later.
2.2 Initialize a swarm
Swarm Intelligence is used in swarm robots to help them achieve their goals. Initially, the robots are
kept near each other. Next, they need to self-arrange themselves into random positions such that they
satisfy certain minimum and maximum distance constraints. This step is essential for the overall
coordination of the swarm-robots, so that, the robots do not go farther than the range of the sensors.
This arrangement involves an algorithm called the swarm dispersion algorithm. [4]
Here, we simply initialized a swarm with 2 constraints –
1. There should be at least one particle in the neighbourhood of another particle.
2. The particles should not overlap, or else the overall number of swarm particles would get
reduced.
Thus, we initialized a swarm of N particles; we randomly picked up an N (100 in most of the
experiments).
Correctly initialized swarm Incorrectly initialized swarm
Precaution: Make sure that the swarm has a random shape, and not a square one. For that, assign the
location of particle at a random location around the particle. In case, we assign the
locations of all the particles with respect to the 1st particle only, the swarm will be of square shape,
which looks unrealistic.
2.2 Randomly moving swarm
Once we have initialized a swarm, the next step is to start the motion of the swarm, and make it
behave like a biological system. We will initialize the various forces amongst the particles,
which will result in an emergent random behaviour of the swarm. We shall introduce these forces
in the upcoming sections.
2.3 Step Response of a swarm
To observer the step response of the swarm, we select a peripheral element and give it a velocity and
see how the entire swarm responds to this stimulus.
On successful simulation of the step response, we observe the “subway effect”, which is the gradual
propagation of information to the swarm particles.
To understand the term, imagine a crowded subway. On a crowded subway, when the people at the
forefront walk, the entire crowd does not walk simultaneously. It happens gradually. Like initially
only first 2 rows start moving, then next 2, then next 3 and so on. It takes some time for the effect of
people walking at the forefront to reach the people at the last. This slow propagation of speed is the
„subway effect‟. In the same way, under the influence of stimuli, the velocities of the swarm particles
changes gradually and hence we observe the “subway effect.”
First step in this process is to find out a peripheral particle and pass on some stimuli to it.
2.3.1 To find the peripheral particle
We shall use the concept of SCAN LINE. To find the leftmost particle of the swarm, we shall start
scanning the image. Imagine a vertical line (called the Scan Line) spanning the entire height of the
image positioned at x=0.
We move this scan line from x = 0 to x = width_of_the_image, until we get a white pixel (or a
particle). As soon as the scan line detects this first particle, we break out of the loop, and report the
location of that particle to the main code.
The red line in the figure indicates the scan line, which travels from left to right and finds out the left
peripheral particle of the swarm.
2.3.2 The Mathematical Model
Once we impart the stimuli to the peripheral element, we need to establish mathematical relationships
among the other particles, so that, this stimuli can be propogated. Following is the model followed
for this:
Let the number of particles in the swarm be N.
Region of influence around a particle = (2M+1)*(2M+1)
Let be the velocity of the neighbour of a particle, and there are r neighbours around the
particle.
Then velocity of a particle is given by
where
Where is the euclidean distance between the particle and its neighbours
2.3.3 Important Points:
1. We need to assign some potential (which causes repulsion) at the boundaries. Otherwise, the
swarm may go out of the screen leading to Segmentation Fault, OR it may go to the left and
emerge from the right of the screen.
2. For keeping track of neighbours, we can introduce a new member in our existing CIndividual
data structure, i.e., an array to neighbours, using which we can calculate the neighbourhood
adjacency matrix.
2.4 Assigning Goal to the swarm
2.4.1 The Birds’ Analogy
This system is similar to the swarm of birds searching for food. Each bird is following the
primitive rules of attraction and repulsion. In addition, each bird is exploiting the eyes of every
other bird, by estimating their distance from the food, and following the one which is closest to
the food.
The simulation of this efficient approach is the subject of this subsection. Here, we show how
we implemented attraction and repulsion among the swarm elements, and their urge to reach the
goal.
2.4.2 Mathematical model
Attraction is necessary so to keep the swarm cohesive, otherwise there is a possibility of the
swarm getting split into two. Repulsion is needed in order to avoid collision of the swarm
particles. And, the urge is required to direct the swarm towards the goal (food).
[Attraction factor]
[Repulsion factor]
[Urge to reach the goal]
Here d1 is the distance between the swarm particles, and d2 is the distance of the swarm particle
from the goal.
The typical values that were suitable for animation were found after a lot of trial-and-error. Thus,
the values we used for these constants are
And
Where and denotes respectively the new and the old location of the swarm-particles.
We introduced attraction in the 5x5 neighbourhoods and repulsion in the 3x3 neighbourhoods.
Precaution: One problem that may arise during this process is that different characteristics can
interfere with each other. Too much attraction can drive to lose track of its goal. Too much
repulsion can break the swarm apart. Hence, the values mentioned above came after a number of
experiments.
2.4.3 Results
Our goal coordinates were = (300,200). Interestingly, the swarm takes a V kind of a shape in this
simulation.
2.5 Behaviour in a 2 tank system.
In this subsection, we will discuss the modifications needed to help the swarm travel through long
narrow corridors. The scenario is as follows:
The swarm has to move from region A to region B without colliding with the walls of the region C.
For a single agent, we can employ the path finding algorithms like the A* algorithm [10], which is
widely used in many computer games. However, for a swarm which learns and evolves, we need a
different approach.
We define new parameters to carry out this process. We define 2 global variables –
1. The centroid of the swarm
∑
∑
2. The radius of the swarm – It is the radius of the circle circumscribing the swarm.
In the simulation, the first step is detection of the wall. As the wall is coloured grey, its detection is
quite easy. Once the wall is detected, the wall potential will not allow the swarm to pass through it.
As soon as the swarm reaches the opening of the narrow corridor, it will reduce its radius and travel
through the tunnel. Once out of the tunnel, it can go back to its original radius, and reach the target.
We can control the radius by varying the values. And, the simulation stops when the centroid
of the swarm reaches the target area.
For simulation of walls here, we can also make use of the virtual swarm agents [6].
Chapter 3
Application in Visual Tracking
3.1 Introduction
The concept of swarm intelligence can be used to track objects [5]. We can employ algorithms like
predator-prey tracking the objects, wherein the object is the prey and the swarm particles which track it
are the predators.
The basic idea is that a swarm of particles fly around the image and learns and evolves as the frames go
by. The tracking window actually encloses the whole of the swarm, thereby detecting the moving object.
We applied the algorithm on the PETS2000 dataset and detected the various moving objects in the
dataset.
3.2 Mathematical Model
Here we are using the same concept with what we define as “CONFIDENCE VALUES”.
1. The object to be detected is initialized in the first frame. The entire object is marked white as
shown in the figure. These white pixels are actually the swarm elements which will act as tracker
pixels.
2. We define “confidence value”, which represents the confidence level. They were assigned on a
scale of 1 to 10. (1 representing min, 10 representing max)
Assume that in the frame, we have a red pixel at a position . When the new frame is
loaded, we search the neighbourhood of for the same red pixel in the frame. Let denote the
number of such red pixels detected in the neighbourhood.
While searching, there are 3 outcomes possible:
1. M = 1: which means there is exactly one red pixel in the neighbourhood of the old location of the
red pixel. Hence, we have found the new position of the swarm particle representing the red pixel.
Moreover, I can calculate the velocity of the swarm particle from this data. And, I set the
confidence value of this particle as 10.
2. M = 0: Confidence value is set to zero. The particle is not sure where to go. Hence, in this case it
shifts closer to the centroid of the swarm.
3. M > 1: for this case, we use the reduced „confidence values‟. The confidence values are reduced
by using the variance of confidence values of the neighbours.
After assigning the individual confidence values, we calculate the velocities of individual swarm particle
p as
,
where ‟s are the “Confidence values” of the particles.
The process is repeated for each swarm particle in each new frame, and we observer the emergent object
tracking behaviour.
(The swarm particles covering the car, which help in object detection.)
Chapter 4
Conclusions and Future Work
Swarming Pixel Tracker is an intelligent system of tracking objects. However, a lot of
improvements and analysis are possible.
1. Stability of swarms is a critical issue. We need to assign proper values of attraction and
repulsion factors. A slight incorrectly estimated value may result in the swarm breaking
away into fragments, or collapsing into its centre. This issue of stability has although
been studies [8, 9], yet it still remains an open problem.
2. We have manually detected the object in the first frame. This detection can be
automatized.
3. We found that in various applications, the actual number of particles in the swarm does
not matter, as long as the number is large enough to get our goal accomplished. Hence, in
our applications we may give rise to or kill the swarm particles.
4. Real-time simultaneity is not easily achievable. This is a limitation of the simulation
which can be overcome, if we build better algorithms for swarm simulation.
5. Predicting the exact behaviour of the swarm is also a problem that can be studied.
6. As long as the swarm particles accomplish the assigned goal, the actual number of
particles in a swarm does not matter. In many cases, they may die or new ones may take
birth.
7. Research is possible in the swarm simulation also. We can study how 2 or 3 swarms
interact with each other. That observation will provide us a deeper insight into the swarm
behaviour.
8. Swarming algorithms can be used for various life-like simulations. It is here to stay for
long. This research can be further expanded towards building a road traffic model.
Chapter 5
References
1. Beni, G., Wang, J. Swarm Intelligence in Cellular Robotic Systems, Proceed. NATO
Advanced Workshop on Robots and Biological Systems, Tuscany, Italy, June 26–30
(1989).
2. Reynolds CW (1987) "Flocks, herds and schools: A distributed behavioural model"
Computer Graphics, 21(4), 25–34
3. Prey by Micheal Chricton
4. MA KOVACINA - SWARM ALGORITHMS: SIMULATION AND GENERATION
5. Swarm Intelligence Based Dynamic Object Tracking – by Yuhua Zheng and Yan Meng.
2008 IEEE Congress on Evolutionary Computation (CEC 2008)
6. Exerting Human Control Over Decentralized Robot Swarms by Zsolt Kira and Mitchell
A. Potter
7. Swarm Intelligence: Literature Overview by Yang Liu and Kevin M. Passino, The Ohio
State University
8. K. Jin, P. Liang, and G. Beni, Stability of synchronized distributed control of discrete
swarm structures," in IEEE International Conference on Robotics and Automation, pp.
1033{1038, 1994}
9. G. Beni and P. Liang, Pattern reconfiguration in swarms {convergence of a distributed
asynchronous and bounded iterative algorithm," IEEE Trans. on Robotics and
Automation, vol. 12, June 1996
10. http://www.gamasutra.com/view/feature/3096/toward_more_realistic_pathfinding.php