RECONNAISSANCE AND SURVEILLANCE INURBAN TERRAIN WITH UNMANNED

AERIAL VEHICLES

Tamir Hegazy ∗ Ben Ludington ∗

George Vachtsevanos ∗

∗Department of Electrical and Computer Engineering,Georgia Institute of Technology

Abstract: Unmanned vehicles endowed with attributes of intelligence are findingapplications in reconnaissance, surveillance, and rescue operations in both militaryand civilian domains. We are introducing a methodology to locate a number ofsuch Unmanned Aerial Vehicles (UAVs) optimally over an urban environment andidentify and track potential ground targets using a particle filtering framework.The particle filtering framework, when used in conjunction with novel initializationand adaptation techniques, is a robust, reliable state estimation tool that avoidsmany of the drawbacks of classical techniques. Simulation results support theefficacy of the proposed approach and validate the effectiveness of the algorithmicdevelopments. Copyright c©2005 IFAC

Keywords: Target tracking, Manoeuvring target, Agents, Autonomous vehicles

1. INTRODUCTION

The term target tracking is often used to referto the task of finding/estimating the motion pa-rameters (mainly the location and direction) ofa moving target in a time sequence of measure-ments. This task is achievable as long as the targetis within the sensor’s field of view (FOV). If ithappens that the target keeps moving away tothe point it runs off the FOV, the target trackingtask will fail to track the moving target untilthe target re-enters the sensor’s FOV. To addresssuch problems, the sensor is mounted on a movingplatform such as a UAV. We call the new setup(the sensor plus the UAV) an agent. Thus, we canstart a second task, other than the target trackingtask, to (reactively or proactively) move the sensorto guarantee that the target stays in view. Thatsecond task is what we call the agent placementtask. The work presented in this paper is of the ac-tive sensing-based target tracking variety, in whichboth tasks discussed above are integrated.

The problem of interest to this paper has been thefocus of attention of some researchers in the pastfew decades. Such agent placement approaches asA-CMOMMT (Parker, 2002), CEP (Hegazy andVachtsevanos, 2004), and spreading out (Batalinand Sukhatme, 2002) have addressed the agentplacement problem for a general region of interest.The main difference between those approachesand the approach presented in this paper is thatthis paper is mainly interested in surveillance andreconnaissance in urban environments. Therefore,the problem is formulated accordingly to betteraccommodate the nature of urban environments.

2. AGENT PLACEMENT

2.1 Problem Statement

Introduction We use a formulation of the vari-ety of Weighted Cooperative Multi-robot Observa-tion of Multiple Moving Targets (W-CMOMMT)

Fig. 1. A possible view from a mini-UAV hoveringover a street intersection.

Fig. 2. Inserting excessive observation points.

(Werger and Mataric, 2001) since it captures themultiple-observer-multiple-target scenario with tar-get prioritization. W-CMOMMT can be shown tobe NP-hard (Hegazy and Vachtsevanos, 2004).

In this paper, the choice is made to limit theobservation points (OPs) to street intersections.The rationale behind this choice is illustrated inFigure 1, which shows a possible view from a mini-UAV hovering over a street intersection. As evi-dent from the picture, the agent can easily observeall the moving vehicles on the street as well as thebuilding surfaces all the way to the next streetintersection. All street segments around an agentcan be visually observed simultaneously using ei-ther multiple cameras or a single panoramic cam-era. Thus, the region of interest (ROI) is modelledas a graph G = (V , E), where V is a set of vertices;each vertex vi represents an OP, and E is a set ofedges; each edge ei represents a street segment.Starting with a street map, G can be created byplacing vertices at the street intersections. ExtraOPs can be inserted as needed. For example, whena street segment is too long to be covered bya single agent’s FOV, one or more OPs can beinserted along the segment to split it into two ormore smaller segments. This idea is illustrated inFigure 2.

Problem Formulation Given:

• G = (V , E): a graph representing the ROI,• R: a team of m agents that are located on

U ⊆ V and have observation sensors witha FOV that can cover the longest streetsegment represented in G ,

• O (t): a set of n targets, such that, at time t,target oj (t) is located on some edge ek ∈ E ,and

• U: a vector of utility values such that uj

reflects the utility value gained by observingoj .

Define an m×n Boolean matrix Γ (t), where γij (t)is equal to 1 if and only if oj is located on an edgethat is incident to the vertex where ri is located.

The goal is to provide a dynamic agent placementpolicy to maximize the normalized global utilityfunction

G =

T∑t=0

n∑j=1

uj

∨mi=1 γij (t)

Tn∑

j=1

uj

, (1)

where∨

is the logical disjunction operator, andT is the time interval during which the trackingmission is carried out.

2.2 Approach

Coarse Motion Model A target currently residingon an edge e connecting the vertices v1 and v2

may stay on the same edge e or move to anyother edge incident to v1 or v2. Target transitionsamong street segments follow a stochastic modeldescribed by an M th-order Markov chain, whichcan be parsimoniously represented using the gen-eralized mixture transition distribution (GMTD)model (Berchtold and Raftery, 2002) as

X(t) =M∑

i=1

λiQiX(t− i), (2)

where λi’s add up to 1, X(t) is a vector represent-ing the probability distribution of target locationover E at time t, Qi is the i-step transition matrix.

Agents can use the model to predict the targetlocations at future time instant h as a probabilitydistribution using

Xj (t + h) =M∑

i=1

λiQiXj (t + h− i) , (3)

where Xj is the best estimate of the target lo-cation as a probability distribution. The lagsXj (t + h− i) are obtained in the light of theoutput of the target tracking task.

Distributed Algorithm Every sampling period t,each agent ri broadcasts its location vi (t) ∈ V ,the set of locally observed targets, and their loca-tions. Each agent then builds a weighted version ofthe graph H = (V , E , w), where w : E → R can be

described by probable edge outcomes for a futuretime horizon H, which we define as

w (e,H) =∑

oj∈O

uj

(H∑

h=1

vh

(Xj (t + h) ·Ye

))(4)

where vh describes the contribution of the futureprediction at time t+h to the placement decision(vh’s add up to 1), and Yc is a binary vector withall zeros and a one at the location correspond-ing to e. Equation 4 sums up the probability ×utility quantities over all observed targets over afuture horizon H. Note that since targets changetheir locations over time, the edge weights w aretime-dependent, and so is H . However, the time-dependence notation is left out for clarity.

The algorithm attempts to maximize the globalutility defined in Equation 1 by searching for aset of OPs U ⊆ V that maximizes the coverage ateach time step, which can be defined as

g =∑e∈E

adjU (e) w (e), (5)

where adjU : E → {0, 1}, such that adjU (e) is 1if and only if e has at least one end in U. Thisis known as the max vertex cover problem, whichis an NP-hard problem (Han et al., 2002). Con-sequently, we are only interested in approximatesolutions.

An iterative greedy algorithm (Hochbaum, 1995),provides a performance guarantee of 1−e−1 of thethat obtained by an optimal policy 1 . Each itera-tion of the algorithm starts by assigning weights tothe vertices such that the weight of v is the sum ofthe weights of all incident edges. Then, it picks thevertex vmax with the maximum weight and addsit to the cover set. Finally, vmax and the incidentedges are removed. The process is repeated in thenext iterations until m vertices are obtained.

The algorithm can be decentralized by dividingthe set of vertices V into m disjoint subsets; eachagent ri is assigned a subset Vi. Each agent ri

computes the weights of each vertex v ∈ V i asdescribed above. ri broadcasts the pair (vi, wi)containing the best local candidate vi and its cor-responding weight wi. ri waits on similar broad-cast messages from other agents. The global bestcandidate v is determined and added to the so-lution set. Then, All edges incident to v are re-moved. If v ∈ Vi, the agent removes that vertexfrom Vi. Thus the distributed greedy algorithmentails sending O (m) messages and guaranteesa solution that is at least 1 − e−1 ≈ 63% ofthe optimal solution. Figure 3 shows a pseudo-code representation of the distributed greedy al-gorithm. A clock symbol “�” placed next to a

1 e here refers to the natural logarithm base (≈ 2.7183).

0. Initialize L = φ.1. For each local vertex v ∈ V i, compute vertex

weight w (v) =∑

adj{v}(e)=1

w (e).

2. Pick the vertex vi with the maximum weightwi = w (vi) (if exists).

3. Broadcast the pair (vi, wi) (if exists, otherwisesend a NULL message).

� 4. Wait on a message from each agent rj contain-ing the pair (vj , wj).

5. Choose (v , w) as the pair with the highest w .6. Remove each edge e (if exists) that satisfies

both adj{v} (e) = 1 and adj{Vi} (e) = 1.

7. Add v to the solution set L.8. If v ∈ Vi , remove v from Vi .

9. If∣∣L∣∣ = m, end.

10. Goto Step 1.

Fig. 3. Pseudo-code representation of the distrib-uted greedy algorithm.

step number indicates that a timeout is used toavoid waiting indefinitely for a message from afailed agent.

3. VIDEO TARGET TRACKING

Each agent is equipped with video cameras thatdeliver videos of street segments it is currentlyobserving. Those videos need to be processed andimportant targets are to be tracked. Particle filters(PFs) have recently been successful in trackingmobile targets in video (Perez et al., 2004), (Perezet al., 2002). Information such as size, color, andmotion characteristics of targets, is known a pri-ori. This information is used to initialize the PF inthe first few frames. Thereafter, using a dynamicmodel, the state of each particle is updated as thevideo progresses. At each step, color and motiondata is collected for each particle to determinewhich particles have a high probability of correctlytracking the target. On the next iteration, parti-cles are drawn according to this probability. Thus,successful particles “survive”, while the other par-ticles “die.”

3.1 Particle Filtering in a Bayesian Framework

The objective of Bayesian state estimation is toestimate the posterior pdf p (xk|z1:k) of a statexk based on all previous measurements z1:k. Thispdf can be determined in two steps: predictionand update. In the prediction step, the stateupdate model is used to determine the priorpdf p (xk|xk−1). If a first-order Markov model isassumed, then the prior is given as

p (xk|z1:k−1) =∫p (xk|xk−1) p (xk−1|z1:k−1) dxk−1. (6)

After the measurement, zk is made, the prior isupdated using Bayes’ rule:

p (xk|z1:k) =p (xk|zk) p (xk|z1:k−1)

p (zk|z1:k−1)(7)

Generally, the above equations cannot be deter-mined analytically. A well-known exception is theKalman filter, which can handle linear systemswith Gaussian distributions. PFs are one way toestimate the above equations for nonlinear, non-Gaussian processes.

A PF iteratively approximates the posterior pdfas a set

Sk ={〈x(i)

k , w(i)k 〉

∣∣i = 1, ..., n}

, (8)

where x(i)k is a point in the state space, and w

(i)k

is the weight associated with this point. w(i)k ’s are

non-negative and sum to unity. At each iteration,the particles are updated using the system dynam-ics and sampling from p

(x(i)

k ,x(i)k−1

).

Measurements are then taken at each particle andthe weights are updated 2 using

w(i)k ∝ w

(i)k−1p

(zk|x(i)

k

). (9)

The posterior pdf estimated using particle filter-ing converges to the actual pdf as the numberof particles increases (Arulampalam and Maskell,2002).

3.2 Tracking using Particle Filtering

Initialization In much of the previous work (Perezet al., 2004), (Perez et al., 2002), the filter is ini-tialized by manually selecting a region of interestin one of the early frames of a video sequence.Since the PF is used on an autonomous vehicle,an automatic initialization routine is developed,where the prior knowledge of the targets’ color,motion, and size, is used. First, two adjacentframes are selected and sent through a brick wallchromatic filter such that only pixels that are nearthe color of the target are kept. The filtered framesare subtracted pixel by pixel to gain motion infor-mation. The remaining regions in this differenceimage are grouped. Particles are assigned to theregions based on size; regions with a sizes closeto the target size are assigned more particles thanothers.

Fine Motion Model A motion model similar to(Perez et al., 2004) is used. This model describesthe target motion at a finer scale than the one used

2 If the particles are resampled at each iteration, theprevious weights may be neglected.

for the agent placement. Although, the modelassumes that targets move smoothly from oneframe to the next, a term is included to allowthe model to “lean” towards motion regions. Thisaccounts for instances where the filter loses lock.The model is described as

p (xk) = βRW N(xk−1, σ

2)

+ (1− βRW )N(yk, σ2

y

), (10)

where 0 < βRW < 1 is an adaptable term to allowthe random-walk portion to become a contributorto the model, xk is the state estimate at time k,N

(x, σ2

)is a Gaussian distribution with mean x

and variance σ2, yk is a region where motion isdetected, and σ2

y is the associated variance.

Measurements After each particle is updated us-ing Equation 10, color and motion measurementsare taken. The color data is collected in a man-ner similar to (Perez et al., 2002). A histogramis populated using only pixels whose saturationand value (in the HSV sense) are above certainthresholds. The histogram is compared to a refer-ence histogram, which is created using the priorcolor information. The comparison is done by mul-tiplying the reference histogram by the particlehistogram bin by bin. A distance measurementDC (h,hr) is then defined as a function of theproduct

DC (h,hr) =

(1−

B∑

i=1

√hihi,r

) 12

. (11)

DC (h,hr) is then used to determine the colormodel

p(yC

∣∣x)= exp

(−DC (h,hr)

σ2C

), (12)

where σ2C is an adaptable term that determines

how “spread” p(yC

∣∣x)is.

Motion data is collected at each particle by sub-tracting the pixel values (in the HSV sense) minusthe corresponding values from the previous frame.The difference is summed and normalized by thearea. This sum is finally used to determine amotion distance

DM (x) =(

1− M

A

) 12

, (13)

where M is the sum of the difference pixels withinthe particle, and A is the area of the particle.DM (x) is used to determine the motion measure-ment model

p(yM |x) ∝ exp

(−DM (x)

σ2M

), (14)

where σ2M is analogous to σ2

C .

It is assumed that the color measurements are in-dependent from their motion counterparts. There-fore, the total measurement model may be fac-tored as

p (y|x) = p(yC |x)λC

p(yM |x)λM

, (15)

where λC,M are adaptable terms that are adjustedthroughout the filtering process.

Adaptation The iterative nature of the PF allowsfor various parameters to be updated through-out the filtering process. (Kwok et al., 2003) and(Fox, 2003) have shown that the number of parti-cles may be updated during the filtering process inorder to increase efficiency. In this case, the num-ber of particles as well as model and measurementparameters are automatically adjusted. For exam-ple, when a target is in a cluttered environment,it is advantageous for the motion measurementsto make a larger contribution to the measurementmodel and to allow the state update model to leanmore towards regions of high motion. The adapta-tion is determined by examining the goodness ofthe“lock” that the filter had on the target, whichcan be measured using (1) the weights that wouldbe obtained if only color data was collected and(2) the spatial dispersion of the best particles. In aframe with a good lock, the standard deviation ofthe color weights and the maximum color weightare both higher than in frames with a poor lock.

Confidence Level Determination In order to usethe PF as part of a decision support tool, itis essential to know how well the targets arebeing tracked. This can be accomplished usinga neural network (NN), which takes data fromthe filter such as the maximum particle weight,standard deviation of particle weight, the spatialdispersion of the best particles, and the numberof particles, and outputs a confidence level as adiscrete value. In our case, 4 confidence levels areused. A confidence level of 1 corresponds to havinga poor lock on the target where none of the topten particles are on the target. A confidence levelof four corresponds to having a strong lock onthe target where all of the ten best particles areon the target. The NN was trained off-line usingdata collected while running a PF and manuallyassigning confidence levels.

4. PERFORMANCE EVALUATION

The integrated approach is evaluated on an urbanzone whose street map is similar to the one shownin Figure 2. A first-order Markov chain is definedto determine how targets move from a streetsegment to another within the urban zone. As anexample of how a single target is tracked on astreet segment, a PF is used to track a soldier as he

manoeuvered on a street segment form an urbanenvironment 3 . Frames are grabbed from a movieat a rate of 30 Hz. The results of the integratedapproach are shown in Figure 4

Figure 4(a) shows the simulation results of apply-ing different algorithms to the scenario describedabove. In addition to the predictive greedy algo-rithm introduced above, a random dynamic policyand a fixed policy are tested. A nonpredictiveversion of the greedy algorithm is also tested,where the weight of an edge is computed simplyby adding the utility values of the targets withoutpredicting future target locations. It is clear thatthe predictive iterative greedy algorithm exhibitsthe best performance among all of the algorithmsunder evaluation. The conclusions to be drawnfrom the figure confirm two intuitions. First, care-fully designed policies, such as the distributedgreedy algorithms, significantly outperform sim-ple random and fixed policies. Second, a nonpre-dictive policy, where agents base their placementdecisions on current target locations only, is out-performed by a predictive policy.

Figure 4(b) shows the initialization of the PFgiven the prior knowledge about the soldier men-tioned above. Sixty particles (shown as boxes)are assigned, and 51 of them are placed on thetarget. It is noteworthy that the filter is initializedproperly even when the target is camouflaged.Figure 4(c) shows 6 sample frames of tracking thesoldier. The box represents the weighted averageof the ten best particles. The set of “lights” in theupper left corner of each frame are used to indicatethe output of the NN. If the lowest “light” is“illuminated,” the NN has output the lowest con-fidence level. If the second lowest is illuminated,the NN has output the second lowest confidencelevel. If the middle two are illuminated, the NNhas output the second highest confidence level. Ifthe top three are illuminated, the NN has outputthe highest confidence level.

5. CONCLUSION

This paper introduces an integrated approach toreconnaissance and surveillance in urban envi-ronments. The approach relies on a multi-leveltarget motion model and entails algorithms forautonomous dynamic UAV placement and visualtarget tracking. Algorithms are developed andevaluated using urban warfare scenarios. Exper-imental results reveal the effectiveness of the ap-proach and its applicability to real-life scenarios.Therefore, this paper contributes (1) an integratedapproach to UAV placement and target tracking

3 McKenna Military Operations in Urban Training(MOUT) site at Fort Benning.

(a) Performance of different agent placementpolicies as the target-to-agent ratio increases.(PIG: Predictive iterative greedy, NIG: Nonpre-dictive iterative greedy.)

(b) Automatic particle initialization.

(c) Tracking a single target in an urban environment.

Fig. 4. Results of applying the integrated ap-proach in an urban environment.

for urban reconnaissance and surveillance, (2) adistributed UAV placement algorithm for targettracking applications and (3) a particle filter-based target tracking algorithm.

ACKNOWLEDGEMENT

This work has been partially funded by DARPAunder the Software-Enabled Control program andNSF under grant number CCR-0209179. We are

gratefully acknowledging the support from thesetwo organizations.

REFERENCES

Arulampalam, A. and S. Maskell (2002). A tu-torial on particle filters for online nonlin-ear/ non-Gaussian Bayesian tracking. IEEETransactions on Signal Processing 50, 174–188.

Batalin, M. and G. Sukhatme (2002). Spreadingout: a local approach to multi-robot cover-age. In: Proc. of the Sixth International Sym-posium on Distributed Autonomous RoboticSystems. Fukuoka, Japan. pp. 373–382.

Berchtold, A. and A. Raftery (2002). The mixturetransition distribution model for high-orderMarkov chains and non-gaussian time series.Statistical Science 17(3), 328–356.

Fox, D. (2003). Adapting the sample size in par-ticle filters through KLD-sampling. Interna-tional Journal of Robotics Research pp. 985–1002.

Han, Q., Y. Ye, H. Zhang and J. Zhang (2002). Onapproximation of max-vertex-cover. EuropeanJournal of Operational Research (2), 207–220.

Hegazy, T. and G. Vachtsevanos (2004). Dynamicagent deployment for tracking moving tar-gets. In: Proc. of the Twelfth MediterraneanConference on Control and Automation. Ku-sadasi, Aydin, Turkey.

Hochbaum, D. (1995). Approximate Algorithmsfor NP-Hard Problems. PWS PublishingCompany.

Kwok, C., D. Fox and M. Meila (2003). Adap-tive real-time particle fitlers for robot lo-catlization. In: Proc. of the IEEE Interna-tional Conference on Robotices & Automa-tion. pp. 2836–2841.

Parker, L. (2002). Distributed algorithms formulti-robot observation of multiple movingtargets. Autonomous Robots 12(3), 231–255.

Perez, P., C. Hue and J. Vermaak (2002). Color-based probabilistic tracking. In: Proc. of theEuropean Conference on Computer Vision.pp. 134–149.

Perez, P., J. Vermaak and A. Blake (2004). Datafusion for visual tracking with particles. In:Proc. of the IEEE. Vol. 92. pp. 495–513.

Werger, B. and Maja J. Mataric (2001). Frominsect to internet: Situated control for net-worked robot teams. Annals of Mathematicsand Artificial Intelligence 31(1-4), 173–197.