Indoor Simulation of Wildfire Detection and Monitoring ... · Indoor Simulation of Wild re...

Indoor Simulation of Wildfire Detection and Monitoringusing UAV

by

Hao Cheng Zhu

A thesis submitted in conformity with the requirementsfor the degree of Masters of Applied Science

Graduate Department of Aerospace EngineeringUniversity of Toronto

c© Copyright 2014 by Hao Cheng Zhu

Abstract

Indoor Simulation of Wildfire Detection and Monitoring using UAV

Hao Cheng Zhu

Masters of Applied Science

Graduate Department of Aerospace Engineering

University of Toronto

2014

Wildfires have been a major source of environmental and economical damage for Canada.

Wildfire detection and monitoring using UAVs have garnered great interest. This thesis

project investigates the use of colour-based vision feedback for tracking a dynamically ex-

panding wildfire simulated in a fully coloured environment. A custom Indoor Simulation

Platform is developed to test and debug various navigation algorithms using a physical

quadrotor. The proposed fire perimeter tracking navigation algorithm has been tested

in both computational simulation and on the Indoor Simulation Platform, and could be

used in an outdoor mission with high confidence of success.

ii

Acknowledgements

First and foremost, I would like to thank Dr. Hugh H.T. Liu for his continuous support

and guidance over the course of these two years. His mentor-ship has been critical in

the completion of this thesis project. Furthermore, I would like to thank my two RAC

members, Dr. Grant and Dr. Damaren, for their feedbacks and comments throughout

this project. I would also like to thank Yasir Malang, a summer intern at the FSC

lab, for his contribution in the initial construction of the Indoor Simulation Platform.

Additionally, Yasir assisted in the configuration of the ground station computer. I would

also like to acknowledge Marlon Koga, another summer intern at the FSC lab, for his

contribution in parameter estimation based on experimental data. Finally, I have to give

special thanks my friends and family for their continued encouragement and support over

these two years, providing me with hope and motivation that are necessary to complete

my project.

iii

Contents

1 Introduction 1

1.1 Background and Motivation . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Research Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2.1 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2.2 Constraints and Limitations . . . . . . . . . . . . . . . . . . . . . 3

1.3 Literature Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.3.1 Fire Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.3.2 Quadrotor Dynamic Model . . . . . . . . . . . . . . . . . . . . . . 5

1.3.3 Edge Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.3.4 Edge Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.3.5 Visual Servoing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1.4 Thesis Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2 Fire Perimeter Tracking 9

2.1 Fire Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.1.1 Static Elliptic Approximation . . . . . . . . . . . . . . . . . . . . 10

2.1.2 Dynamic Elliptic Approximation . . . . . . . . . . . . . . . . . . 12

2.1.3 Approximation under Non-Uniform Conditions . . . . . . . . . . . 17

2.2 Edge Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

2.3 Perimeter Reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

2.3.1 Reverse Elliptic Approximation . . . . . . . . . . . . . . . . . . . 25

2.3.2 Full Perimeter Reconstruction . . . . . . . . . . . . . . . . . . . . 27

2.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

3 Quadrotor Navigation 31

3.1 Quadrotor Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.1.1 Translational Dynamics . . . . . . . . . . . . . . . . . . . . . . . 32

3.1.2 Rotational Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . 33

iv

3.1.3 Motion Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

3.2 Quadrotor Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

3.3 PBVS Navigation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

3.3.1 Reference Frame Transformation . . . . . . . . . . . . . . . . . . 38

3.4 Parameter Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

3.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

4 Indoor Simulation Platform 46

4.1 Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

4.2 Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

4.3 Experimentation Phases . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

4.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

5 Numerical Results 57

5.1 Quadrotor Navigation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

5.1.1 Controller Validation . . . . . . . . . . . . . . . . . . . . . . . . . 57

5.1.2 Way-Point Navigation . . . . . . . . . . . . . . . . . . . . . . . . 58

5.1.3 Quadrotor Fire Perimeter Tracking . . . . . . . . . . . . . . . . . 62

5.2 Fire Perimeter Reconstruction . . . . . . . . . . . . . . . . . . . . . . . . 63

6 Experimental Results 65

6.1 Quadrotor Navigation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

6.1.1 Way-Point Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . 65

6.1.2 Target Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

6.1.3 Line Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

6.1.4 Dynamic Edge Tracking . . . . . . . . . . . . . . . . . . . . . . . 71

6.2 Fire Perimeter Reconstruction . . . . . . . . . . . . . . . . . . . . . . . . 73

6.3 Outdoor Extension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

7 Conclusions 78

References 78

v

List of Tables

2.1 Cell properties table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

4.1 Hardware specifications for FPV259 quadrotor frame . . . . . . . . . . . 49

4.2 Hardware specifications for ArduPilot Mega 2.6 . . . . . . . . . . . . . . 50

vi

List of Figures

2.1 The ROS-based elliptic approximation method . . . . . . . . . . . . . . . 10

2.2 Simulated fire propagation after 5, 20, and 100 seconds . . . . . . . . . . 12

2.3 The contour that identifies the new fire front lies tangent to two adjacent

ellipses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.4 Geometric method of finding the tangent between two circles. . . . . . . 14

2.5 Fire perimeter simulation using dynamic elliptic approximation . . . . . 17

2.6 Finding the tangent between two different circles. . . . . . . . . . . . . . 18

2.7 Fire perimeter simulation under non-uniform conditions . . . . . . . . . . 20

2.8 The top down view of a wildfire projected onto the ISP. . . . . . . . . . . 21

2.9 The propsed process of edge extraction . . . . . . . . . . . . . . . . . . . 21

2.10 Side profile of quadrotor and location of new way-point on image . . . . 24

2.11 The spiral path created by the UAV when tracking a continuously expand-

ing fire perimeter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

2.12 The reverse elliptic approximation result and comparison . . . . . . . . . 27

2.13 The path of a UAV tracking the fire front over three time steps . . . . . 27

3.1 The “X” configuration used by the ArduPilot Mega [26] . . . . . . . . . . 32

3.2 Principle of quadrotor dynamics . . . . . . . . . . . . . . . . . . . . . . . 32

3.3 Controller structure used for the quadrotor . . . . . . . . . . . . . . . . . 37

3.4 PBVS flow chart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

3.5 The image reference frame . . . . . . . . . . . . . . . . . . . . . . . . . . 39

3.6 The pixel-centimetre ratio at a certain altitude . . . . . . . . . . . . . . . 39

3.7 The contribution of quadrotor’s pitch angle to total pitch angle difference

for target . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

3.8 The experimental parameter estimation block diagram [28] . . . . . . . . 44

4.1 The overall system schematic of the ISP. . . . . . . . . . . . . . . . . . . 47

4.2 The environment projection platform with angled projection. . . . . . . . 47

4.3 An example of the projected wildfire environment. . . . . . . . . . . . . . 48

vii

4.4 The two quadrotor frames that have been used in this project . . . . . . 49

4.5 Structure of the ISP software system . . . . . . . . . . . . . . . . . . . . 51

4.6 The interaction between ground station and vision client for edge extraction 52

4.7 The reverse elliptic approximation result and comparison . . . . . . . . . 53

4.8 The test plan for simulation and physical experiment . . . . . . . . . . . 55

5.1 Step response for angle tracking . . . . . . . . . . . . . . . . . . . . . . . 58

5.2 The simulated way-point based quadrotor mission . . . . . . . . . . . . . 59

5.3 Yaw angle tracking in simulated way-point navigation mission . . . . . . 59

5.4 Altitude tracking in simulated way-point navigation mission . . . . . . . 60

5.5 Position tracking in simulated way-point navigation mission . . . . . . . 60

5.6 Roll and pitch angle tracking in simulated way-point navigation mission . 61

5.7 Lateral error in simulated way-point navigation mission . . . . . . . . . . 62

5.8 Simulated fire edge tracking with quadrotor . . . . . . . . . . . . . . . . 62

5.9 Finding local fire perimeter position using reverse elliptic approximation 63

5.10 Full perimeter approximation using elliptic curve fitting technique . . . . 64

6.1 Way-point tracking experiment position results . . . . . . . . . . . . . . . 66

6.2 Way-point tracking experiment heading angle and altitude results . . . . 67

6.3 The target used in the PBVS experiment . . . . . . . . . . . . . . . . . . 67

6.4 Determining the centre position of the target object via thresholding . . 68

6.5 Top down view of the target estimation test . . . . . . . . . . . . . . . . 69

6.6 Top down view of the PBVS tracking test . . . . . . . . . . . . . . . . . 70

6.7 X and Y-axis PBVS tracking result . . . . . . . . . . . . . . . . . . . . . 71

6.8 Static straight edge being projected onto the glass floor . . . . . . . . . . 71

6.9 Top down view of static line tracking . . . . . . . . . . . . . . . . . . . . 72

6.10 The initial state of fire perimeter and the segment projected . . . . . . . 73

6.11 The progression of the dynamically growing fire in three time frames . . . 73

6.12 Dynamic edge tracking result . . . . . . . . . . . . . . . . . . . . . . . . 74

6.13 Detected edges and approximation of the segment . . . . . . . . . . . . . 74

6.14 Comparison of estimated, projected, and actual fire perimeters . . . . . . 75

6.15 FSC Octorotor flying a aerial imaging mission with full payload . . . . . 76

6.16 Edge extraction of outdoor aerial wildfire image . . . . . . . . . . . . . . 77

viii

Nomenclature

ROShead Rate of Spread in the direction of the wind

ROStail Rate of Spread opposite of the wind

ROSside Rate of Spread perpendicular to the wind

a Major axis of ellipse

b Minor axis of ellipse

p Distance between ignition point and centre of ellipse

θwind Direction of the wind

Tellipse Transformation matrix that transforms ellipse into a circle

Ci Centre of circles

Pi Intersection points between tangent line and circles

ri Radius of circles

ξ Neighbouring pixel

c(ξ, x) Geometric closeness between two pixels

s(f(ξ), f(x)) Photometric similarity between two pixels

Gx, Gy Gradient strength in x and y direction

F1−4 Thrust from each motor

θ, φ, ψ Pitch, roll, yaw Euler angle

p, q, r Roll, pitch, yaw rate in body-fixed frame

x, y, z x, y, z acceleration in inertial frame

U1−4 Intermediate input values

Ω1−4 Rotational velocity of each motor

Ixx, Iyy, Izz Moment of inertia for x, y, and z axis

ix

Acronyms

FSC Flight Systems and Control lab

UAV Unmanned Aerial Vehicle

LASE Low Altitude Short Endurance

ISP Indoor Simulation Platform

PBVS Position Based Visual Servoing

IBVS Image Based Visual Servoing

ROS Rate of Spread

ESC Electronic Speed Controller

RGB Red-Green-Blue colour space

HSV Hue-Saturation-Value colour space

APM ArduPilot Mega

PID Proportional-Integral-Derivative

PI Proportional-Integral

IMU Inertial Measurement Unit

GPS Global Positioning System

MP Mission Planner

LiPo Lithium Polymer

DCM Direction Cosine Matrix

x

Chapter 1

Introduction

1.1 Background and Motivation

Canada experiences over 5,000 wildfire occurrences on average each year, affecting al-

most 2 million hectares of land [1]. In 2014, the area damaged by wildfires stands at

a staggering 3.5 million hectares in less than 4,000 occurrences. Real-time monitoring

and early detection of wildfires is a very important field of research due to the massive

environmental and economic damage done by wildfires around the world and in Canada.

Current wildfire detection and monitoring methods include using ground-based watch

towers, satellite imaging, as well as manned aircraft missions. Ground-based watch tow-

ers are of limited use due to their inflexible nature. Even with more modern automatic

fire and smoke detection systems, the detection range for each tower is restricted to ap-

proximately 20km in good visibility [2]. Satellites used for fire detection can be divided

into Low Earth Orbit (LEO) satellites and geostationary satellites. LEO observation

satellites have an orbiting period, which limits their value in tracking wildfires in real

time. The LEO Landsat satellites can provide a resolution of 15m per pixel, but have an

orbiting period of 14 days [3]. The Geostationary Operational Environmental Satellites

(GOES), in contrast, can provide an updated image every 15-30 minutes, but the reso-

lution is only 1km for visual image and 4km for thermal images [3]. In addition, satellite

images can only be taken from a fixed angle, making them impossible to actively avoid

any smoke or cloud cover obstructions. Manned aircraft are much more manoeuvrable

compared to satellites, flies below clouds and can avoid smoke; however, manned aircraft

missions can be risky for the operators and these missions are expensive to carry out.

The use of unmanned aerial vehicles (UAVs) to complement existing wildfire detection

and monitoring methods such as manned missions has become an active area of research

due to the unique attributes of a UAV mission.

1

Chapter 1. Introduction 2

Low altitude UAVs are greatly desired for wildfire detection and monitoring missions

because they can safely fly at an altitude that can avoid cloud cover. They are able to

survey the field at close range and take high resolution images. Low altitude UAVs are

relatively small in size compared to some of the high altitude ones used by the military;

therefore, they generally have shorter endurance. Multirotor is one type of low altitude

UAV that has very high manoeuvrability and is capable of vertical take-off and landing.

They are very useful in missions that have forested areas where sufficiently large clear

areas for take-off and landing are hard to find.

In general, the development of vehicle and its fire detection function needs to be

tested at each design stage, from numerical simulations to field experiments. Numerical

modelling and simulation may not be accurate enough, and field testing is time consum-

ing. Thus, the development of an indoor test platform for hardware-in-loop simulation

is proposed to bridge the gap, and to provide an efficient design-validation cycle.

1.2 Literature Review

Fire edge detection using UAVs requires the integration of multiple sub-systems and spans

multiple areas of research. These components and areas of reserach include environment

simulation, micro-UAV control, autonomous edge tracking, vision-based edge detection,

and navigation using visual servoing.

1.2.1 Fire Model

A fire model in some form is necessary in the simulation in order to move the navigation

algorithm from the more deterministic line tracking mission to a more stochastic envi-

ronment. The so-called physical fire model and approximation fire model are two main

types of methods to describe the spread of a fire. Although they both use cell-based sim-

ulation where each grid cell contains a number of states, such as unburned, burning, and

burnt status, they differ fundamentally in how these states are determined. A physical

fire model, such as the Rothermel model, calculates the temperature of each cell based

on physical parameters of the cell [4]. Fuel parameters, including the mineral content,

moisture content, as well as vegetation density, and environment parameters, including

the wind speed, wind direction, as well as terrain incline, are used as direct inputs to

the physical fire model. The physical fire model uses these input parameters and vari-

ous thermal equations to calculate the temperature of each cell at every time interval.

The states of these cells are assigned based on the temperature. For example, a cell is


labelled as burning when it reaches a certain temperature threshold for a period of time.

In general, the physical fire model is more computationally intensive, but it is also more

accurate than other approximation methods. The physical fire model is used when the

estimated temperature of each cell is desired.

The approximation fire model, on the other hand, uses geometry to estimate the

location of the fire perimeter. The most common and most basic approximation method

is the elliptic approximation. The simplest case of an elliptic approximation is when all

parameters are constant and uniform over the simulated area. The fire expands out from

the original ignition point in an ellipse shape depending on the wind direction and speed.

The wind parameters are converted into head, tail, and side rates of spread (ROS) before

being passed into the ellipse approximation model. These ROS would be used to calculate

the angle, major-axis, and minor-axis of the ellipse that would represent the perimeter

of the fire [5]. This perimeter would then be overlaid onto the simulation field consisting

of grid cells, and the states of these cells are determined by where the cells are located

relative to the elliptic perimeter, such as in the simulation developed by Trunfio [6]. The

elliptic approximation method could be extended to include changing wind parameters

and non-uniform fuel parameters.

The type of fire model to be used for this project is mainly determined by the com-

plexity and accuracy requirement of the fire simulation. The environment simulation for

this project does not have to be overly complex since the ISP is largely a proof of concept.

The approximation model would be the better choice over the physical model due to its

simplicity. Additionally, the output of the fire model for this project is the position of

the fire perimeter, which is direct output of the approximation fire model. While the

approximation model used in this particular project will only take into account a limited

number of field conditions, the model itself is expandable to make the simulation much

more realistic.

1.2.2 Quadrotor Dynamic Model

Although fixed-wing aircrafts are better suited for outdoor fire detection and monitoring

missions, they cannot be operated in the limited indoor lab space. As a result, the more

manoeuvrable quadrotors are used to in the indoor set up. A dynamic model for the

quadrotor is needed for controller development in both simulation and on the physical

vehicle. The quadrotor dynamic model has been presented by many researchers, both

in the original non-linear and linearized form. The full non-linear dynamic model of a

quadrotor can be derived from Newton’s Second Law, as presented by many researchers


including Leishman and Beard [7][8]. The Stanford Testbed of Autonomous Rotorcraft

for Multi-Agent Control (STARMAC) team presented a simplified non-linear model as

well as a linearized quadrotor model for simulation and development [9]. A small roll

and pitch angle assumption is used in the simplified non-linear model to reduce the need

of a rotation matrix between body-fixed frame angles and inertial frame angles. Small

angle approximation is used during linearization to make the model simple and straight

forward. This linearization is done at the equilibrium state of hover, and the linearized

model describes the behavior of the quadrotor within a small roll and pitch angle limit.

The same derivations from basic physic laws can be used to find the quadrotor dy-

namic model. This project will adopt the simplified non-linear model in mathematical

simulation for a balance of accuracy and complexity.

1.2.3 Edge Detection

While many of the edge tracking strategies proposed assume that the edge information is

available, it is important that a realistic edge detection and extraction strategy is imple-

mented to complete the feedback loop. Visual cameras will be the feedback sensors used

in this project. A number of different image processing algorithms and filters can help

with edge extraction from a visual image. Canny Edge Detector, originally proposed by

Canny [10], is the most common image processing method for detecting edges. Canny

Edge Detector utilizes the colour gradient of an image to find the areas that have the

highest contrast, or greatest change in colour. These areas are extracted as edges via hys-

teresis. Canny Edge Detector is very sensitive to noise, which means filtering the image

beforehand is important if a specific edge is desired. Thresholding is a common filtering

technique used to extract an object of interest that has a unique colour. Thresholding

works the best when the object’s colour makes it stand out from the background. A more

advanced thresholding technique is described by Hong [11]. The thresholding technique

described by Hong uses fuzzy sets to determine the object of interest. Qiu [12] presented

an image based autoadaptive edge detection technique that is specifically designed to

detect the edges of a real flame, based on the luminance property of a fire. Multiple

image processing techniques can be combined to increase accuracy and robustness, such

as the curve tracking algorithm proposed by Chen [13]. In some cases, categorizing the

detected edges based on their shapes is just as important as detecting them. Generic

Edge Tokens (GET) is one of the many methods to organize detected edges [14]. The

GET model contains a set of segments and partition points, and can be applied to any

edge. These edges are then classified into one of four types of straight lines, or one of


four types of curves [14].

For the purpose of this physical experiment conducted in the ISP, Canny Edge De-

tector will be used in conjunction with various filters to ensure that the fire edge is the

only edge being extracted. The exact method of image processing will be determined

based on the actual experimental set up and the quality of the images collected.

1.2.4 Edge Tracking

Aerial fire edge tracking is often simplified into a line following problem, as many re-

searchers, such as Casbeer and Sujit [15][16], make the assumption that the fire can be

defined by a clear distinctive edge in their papers regarding fire edge tracking. Casbeer

presented a fire edge tracking method that would keep the UAV a certain distance away

from the fire to minimize damage. His method assumes that the edge of the fire can be

specified by a curve and that it is in view at all times. In the case where the fire edge

is not guaranteed to be in view of the camera, Kawano [17] proposed a line following

algorithm with a dead-reckoning technique that can estimate the position of the robot.

The dead-reckoning technique computes the position of a robot by taking into account

of its velocity vector and time passed since previous update. This method is relatively

accurate on ground robots where the velocity with respect to ground can be accurately

measured. Dead-reckoning may not be as accurate on UAVs as ground robot due to the

increased uncertainty with the operating environment, such as effect of gusts and air

density. Holt and Beard [18] presented a road following algorithm for UAVs based on

proportional navigation. The experimental results included show good tracking accuracy.

Simpler control strategies, such as a bang-bang controller, have been presented by various

researchers including Clark [19] and Kemp [20] to be used as benchmarking baseline for

other more complex algorithms.

The environment simulation created in the ISP is purely visual based. The UAV does

not need to maintain a certain distance away from the fire for safety reasons, as described

in Casbeer’s paper. Therefore, the tracking algorithm used for this project will attempt

to ensure the UAV is on top of the fire edge at all times, using a combination of Casbeer’s

method for heading control and a simpler bang-bang controller for lateral error control.

The advantage of a bang-bang lateral error controller is that it is easy to create an offset

for the vehicle, with or without an angled camera. In a real fire tracking mission, it

would be a good idea for the UAV to fly a certain distance away from the actual fire to

prevent heat damage and turbulence. The fire in the simulated environment is assumed

to be expanding much slower than the velocity of the UAV in the ISP; thus, it can be


assumed that the fire edge is in view of the camera at all time.

1.2.5 Visual Servoing

Visual servoing is a technique that can be applied to different types of mobile robots,

including UAVs. There are two types of visual servoing techniques for navigation: Po-

sition Based Visual Servoing (PBVS) and Image Based Visual Servoing (IBVS). PBVS

feedback requires the knowledge of the absolute position of both the vehicle and the tar-

get, whereas IBVS only needs the knowledge of relative position between the two. Many

researchers, such as Rangavjhala [21], Thuilot [22], and Bachiller [23], have presented

papers on PBVS control for robotics. On the other hand, the IBVS technique has been

used by researchers such as Begum [24] and Fu [25], on mobile robotics navigation. The

total loss of target is a major problem for navigation based on visual servoing. Fu at-

tempts to negate this problem with the use of an odometry-based fail-safe system. In

this case, the robot would continue to navigate using on board Lidar sensors.

Both visual servoing techniques can be used for the navigation control of a UAV to

track the edge of a fire. The ultimate goal of fire edge tracking is to gather location data

of the fire edge. For this purpose, PBVS would be the better choice because it computes

the target’s global position while navigating. IBVS, on the other hand, only calculates

the relative position of the fire edge, which means an extra step is needed to find the

global position of the fire edge afterwards.

1.3 Research Objectives

1.3.1 Objectives

The primary goal of this thesis project is to develop a wildfire perimeter tracking al-

gorithm for UAVs. This trajectory tracking algorithm needs to be able to detect the

location of a dynamically expanding fire perimeter, and guide the UAV to fly along the

perimeter over the course of the mission. This algorithm should be tested in both com-

putational simulation, such as MATLAB, and a hardware-in-the-loop environment with

a physical aerial vehicle in a simulated environment using visual feedback. The following

two objectives summarizes this goal:

1. To design a fire perimeter tracking algorithm that makes the aerial vehicle follow

the expanding fire edge and relay position data of the fire to the ground station,

and


2. To develop an integrated hardware-in-the-loop Indoor Simulation Platform (ISP)

that allows the testing of such navigation algorithms inside the lab.

1.3.2 Constraints and Limitations

One of the primary requirements of the navigation algorithm is that it has to make the

UAV track the edge of a wildfire. The sensors that could be used by a UAV to detect a

wildfire in real life are visual or thermal cameras. In fact, both types of cameras are often

used in the real world scenario to provide backup and to increase detection accuracy. The

extraction of a fire edge from a thermal image and a visual image is similar, although

edge extraction is more robust with thermal image due to less background noise. A visual

camera is used for this thesis project because the it is not practical to generate a simulated

environment with temperature difference indoors on the first iteration of the ISP, which

is only a proof of concept system. This indoor constraint limits the fire environment

to a simplified representation of an actual wildfire. Specifically, the environment used

for testing will have varying wind conditions, but uniform and constant vegetation and

terrain parameters.

1.4 Thesis Outline

This thesis includes five chapters. Chapter 1 is an introduction to the problem that

this research project attempts to solve. It also provides a summary of related work

previously done by other researchers. Chapter 2 describes the fire model used to generate

a simulation environment, provides an edge extraction algorithm that can detect the

fire edge from visual images, and proposes a fire perimeter reconstruction technique

to analyze the data captured by the UAV. Chapter 3 presents the quadrotor model,

control structure, and the navigation algorithm developed for experimentation. Chapter

4 contains an overview of both the hardware and software components of the Indoor

Simulation Platform that is being developed for the experiments. Chapter 5 presents the

numerical results of the quadrotor controller tests, as well as the fire edge tracking tests

in MATLAB simulation. Experimental results for both preliminary tests, such as target

position estimation and PBVS target tracking, and the final fire edge tracking tests on

the ISP can be found in Chapter 6. Finally, Chapter 7 provides a summary for this

project and discusses possible future research.

Chapter 2

Fire Perimeter Tracking

A fire model is needed in order to generate a simulated environment with a dynamically

growing wildfire. Once this simulated environment is created, the UAV would use an edge

detection algorithm to extract the edge of the fire perimeter for navigation purposes. Data

collected from the quadrotor tracking mission would be used for perimeter reconstruction

for the local segment that is tracked, and potentially for the entire fire.

2.1 Fire Model

The physical model and the approximation model are two common ways to create a

wildfire simulation. The elliptic approximation model is chosen for this research project

for two reasons. First of all, the scope of this project constrains the wildfire simulation

to constant fuel parameters. This means the vegetation density, wind speed, and wind

direction are the same across the entire simulation space, which in turn means that the

rate of spread at every point in the simulated field is the same. This suggests that

the elliptic approximation fire model is sufficient of the purpose of this project, and the

additional complexity of the Rothermel model is not needed. Secondly, the desired output

of the fire simulation is a top down view of an expanding fire perimeter, as opposed to

temperature at each point on the grid. The elliptic approximation model generates the

fire perimeter directly, whereas the physical fire model needs to compute the temperature

first before creating the fire perimeter. Thus, the elliptic approximation model is chosen

to generate the wildfire simulation.

8

Chapter 2. Fire Perimeter Tracking 9

2.1.1 Static Elliptic Approximation

The basis of the elliptic approximation model is that under a no wind condition, a field

with constant fuel would result in the same rate of spread (ROS) in all directions. This

would mean that the fire perimeter could be approximated as a circle. Under windy

conditions, the front ROS, which has the same heading angle as the wind direction, will

increase. The tail ROS, which is opposite to the wind direction, will decrease. The

two side ROS, which are perpendicular to the wind direction, will remain the same.

The circular fire perimeter will turn into an ellipse in this case. Figure 2.1 shows the

relationship between the ignition point, the ROS, the wind direction, and the elliptic

shape of a fire perimeter.

Figure 2.1: The ROS-based elliptic approximation method

The ellipse created by the wind has the same direction as the wind direction. The

following equations calculate the total spread of the fire in all four directions for every

time period dt: headcurrenttailcurrent

sidecurrent

=

headprevtailprev

sideprev

+ ∆t×

ROSheadROStail

ROSside

(2.1)

a =headcurrent + tailcurrent

2(2.2)

b = sidecurrent (2.3)

p = a− tailcurrent (2.4)


C = (Ix + p× cos(θ), Iy + p× sin(θ)) (2.5)

where a and b are the major and minor axes of the ellipse respectively. For plotting

purposes, the centre of the ellipse C is needed, which is distance p away from ignition

point I = (Ix, Iy). The ellipse can be created using the general ellipse formula:

(x cos(θwind) + y sin(θwind))2

a2+

(x sin(θwind)− y cos(θwind))2

b2= 1 (2.6)

The pseudo code for a fire simulation using static elliptic approximation method is

as follows:

Create simulation field of x× y grid cellsSet all cell values to 0Set all cell state to unburnedwhile time is less than final time do

Calculate current head, tail, and side spread from Ignition Point (Eqn 2.1)Calculate a, b, p, C based on current head, tail, and side distances (Eqn 2.2 - 2.5)Update ellipse perimeter arrayfor Each cell in the simulation field do

if cell is on the ellipse perimeter AND cell is unburned thenSet cell state to burning

else if cell state is burning thenIncrement cell value

else if cell state is burning AND cell value reaches threshold thenSet cell state to burnt

end ifend forColour simulation field with predetermined colour schemeIncrement time

end while

The simulated field is a square grid with 100 cells on each side. Each cell is associated

with one of three states: unburned, burning, and burnt. Each cell is also associated with

a colour and a value that represents how much fuel has been burned. The unburned cells

are coloured green and have a numerical value of 0, representing healthy vegetation. The

burning cells start with yellow colour with a numerical value of 8, but quickly changes

to orange and then red where the value would eventually become 10. This is made

to distinguish between those that have just started burning and those that have been

burning for a while. The burnt cells have a brown colour with a numerical value of 10,

and are not considered to be on fire because they have used up all available fuel. The

comparison between cell colour, value, and state can be found in Table 2.1.


Value Colour State0 Green Unburned

1 - 7 NA NA8 Yellow Burning9 Red Burning10 Brown Burnt

Table 2.1: Cell properties table

Figure 2.2 demonstrates how elliptic approximation fire model propagates the fire

perimeter over 100 seconds. The environment used for this demonstration is a 100 × 100

grid.

(a) t = 5s (b) t = 20s (c) t = 100s

Figure 2.2: Simulated fire propagation after 5, 20, and 100 seconds

2.1.2 Dynamic Elliptic Approximation

The elliptic approximation can be expanded with dynamic wind conditions. This means

that while the fuel parameters on the ground remain constant and uniform, the wind

speed and direction can vary with time. The effect of the wind on every grid cell at each

time step is uniform. Under such conditions, the fire edge cannot be approximated by

a single ellipse. Instead, elliptic approximation would need to be applied to a number

of discretized points along the original fire front. The Dynamic Elliptic Approximation

includes three steps:

1. Discretize the fire front at time t = 0 into a number of points.

2. Each of the discretized points act as a new ignition point, and apply basic elliptic

approximation on each of these ignition points.


3. Find a contour that envelops all the individual new ellipses, and this contour be-

comes the new fire front at time t = 1.

The calculations for the small individual ellipses are the same as the static elliptic

approximation, which requires the knowledge of ignition point, wind speed, and wind

direction. The contour that envelops these small ellipses would lie tangent to all of them;

thus, the task of finding this tangent can be broken down into finding tangent lines

between every two adjacent ellipses, as illustrated in Figure 2.3.

Figure 2.3: The contour that identifies the new fire front lies tangent to two adjacentellipses

Points P1,t and P2,t are two points on the original fire perimeter at time t. They

become the two adjacent ignition points after discretization has been applied to the

original fire perimeter. After one time period has passed, the two ignition points expand

based on the basic elliptic approximation rules and becomes points P1,t+1 and P2,t+1

respectively. The line that is tangent to P1,t+1 and P2,t+1 forms a segment of the new fire

perimeter at time t+ 1.

The method that is used to find these tangent lines has three steps:

1. Find and apply the transformation matrix T to transform the two adjacent ellipses

into circles.

2. Find a line that is tangent to both circles.

3. Apply inverse transformation matrix T−1 to transform the two circles back into the

original ellipses, along with the newly determined tangent line.


The transformation matrix has 2 components. It must first rotate the ellipses back

to a 0 degree position, and then compress the major axis of the ellipses by a factor of

b/a. The transformation is defined by Tellipse = TCompTRot:[XT

YT

]= Tellipse

[x

y

](2.7)

TComp =

b/a 0

0 1

TRot =

cosθ sinθ

−sinθ cosθ

(2.8)

where θ is the angle of the ellipses, which is the same as the wind direction; a and b

are the major and minor axes of the ellipse respectively;[x y

]Tare the points on the

original ellipse; and[XT YT

]Tare the points on the circle after the transformation.

The entire ellipse is first rotated clockwise by an angle of θ such that the new ellipse is

horizontal with respect to the x-axis. The ellipse is then shrunk by a factor of b/a along

its major axis so that it becomes a circle with radius r = b. Once the two ellipses have

been transformed into two circles, a line segment representing the outer tangent of these

two circles needs to be calculated. Figure 2.4 shows the geometric method of finding a

tangent line between two circles with same radius.

Figure 2.4: Geometric method of finding the tangent between two circles.

The two circles are centred at C1 = (xc1, yc1) and C2 = (xc2, yc2), and both have radius

r. Line P1P2 is the line that is tangent to both circles. AP1 and BC2 are perpendicular


to the x-axis. 6 AP1C1 and 6 BC1C2 are equal and are named 6 α. 6 α can be described

by the following relationships based on 4BC1C2:sin(α) = yc2−yc1d

cos(α) = xc2−xc1d

(2.9)

The intersection points P1 and P2 can be determined using the knowledge of sin(α)

and cos(α).

P1,x = xc1 − r × sin(α)

P1,y = yc1 + r × cos(α)

P2,x = xc2 − r × sin(α)

P2,y = yc2 + r × cos(α)

(2.10)

Once P1P2 are found, an inverse transformation Tinv = R−1ellipse would be applied so

that this line is tangent to the two original ellipses. In order to create a fire simulation

based on dynamic elliptic approximation, the static elliptic approximation procedure

needs to be converted into a function. This function would take the following inputs:

ignition point, head ROS, tail ROS, and side ROS; and would provide the following

outputs: 2× n array of x and y position of the discretized ellipse perimeter.

function createEllipse(I, ROShead, ROStail, ROSside)Calculate head, tail, and side spread distancesCalculate ellipse parameters a, b, p, C based on head, tail, and side spread distancesreturn Ellipse parameters

end function

Using the function above, a fire simulation with the dynamic elliptic approximation

can be created.

Figure 2.5 demonstrates the use of the dynamic elliptic approximation to simulate

fire perimeter growth over one time step.

The black circle is the original ignition point of the fire. The black ellipse is the original

fire perimeter. The small blue ellipses are the results of individual elliptic approximations

applied to the discretized fire perimeter. Finally, the red ellipse is the new fire perimeter,

which envelopes the individual ellipses.


Require: createEllipseCreate simulation field of x× y grid cellsSet all cell values to 0Set all cell state to unburnedwhile time is less than final time do

if time is 0 thenCall createEllipse on original ignition pointCompute perimeter array based on returned ellipse parameters

elsefor Each point in perimeter array at time t-1 do

Call createEllipse with selected point as ignition pointSave ellipse parameters into parameter array

end forfor Each entry in ellipse parameter array do

Calculate outer tangent line between current and next ellipse using Eqn 2.9and 2.10

Save tangent line points in perimeter arrayend forDetermine cell state using same logic as static case

end ifColour simulation field with predetermined colour schemeIncrement time

end while


Figure 2.5: Fire perimeter simulation using dynamic elliptic approximation

2.1.3 Approximation under Non-Uniform Conditions

The dynamic elliptic approximation can be expanded to simulate non-uniform conditions.

A non-uniform condition is defined as wind and fuel conditions that changes with both

time and location. At any time instance t, the wind speed, wind direction, and fuel

parameter on the simulation field can be different depending on where the wind data is

being measured. For example, if a part of the simulation field is set to contain higher

density vegetation, then there is more fuel for burning at that location compared to rest

of the field. Generally, the rate of spread increases when there is more fuel present or

the wind speed is higher, and vice versa.

The difference between dynamic elliptic approximation under uniform conditions and

non-uniform conditions is that two different transformation matrices, and by extension,

two inverse transformation matrices, are needed. The environment parameters could

be different between two neighbouring cells, resulting in different ROS for each cell.

This implies that the radii of the two neighbouring circles after transformation could be

different. As a result, the general version of the tangent line equations are used. Figure

2.6 shows the geometric method of finding tangent line between two circles with different

radii.


Figure 2.6: Finding the tangent between two different circles.

Once again, the two circles are centred at C1 = (xc1, yc1) and C2 = (xc2, yc2). Circle

1 has radius r1 and Circle 2 has radius r2. Line P1P2 is the line that is tangent to both

circles. AP1 and BC2 are perpendicular to the x-axis; C1P1 and C2P2 are perpendicular

to the tangent line P1P2; and C1C is parallel to the tangent line P1P2. 6 AP1C1 is 6 δ,

where 6 δ = 6 α + 6 β. 6 α and 6 β can be described by the following relationships based

on 4BC1C2 and 4C1C2C:

sin(a) = yc2−yc1d

cos(a) = xc2−xc1d

sin(b) = rc2−rc1d

cos(b) = C1Cd

(2.11)

where d =√

(yc2 − yc1)2 + (xc2 − xc1)2

C1C =√d2 − (r2 − r1)2

(2.12)

sin(δ) = sin(α + β) = sin(α) cos(β) + cos(α) sin(β)

cos(δ) = cos(α + β) = cos(α) cos(β) + sin(α) sin(β)(2.13)


Finally, the locations of P1 and P2 can be calculated using the following equations:

P1,x = xc1 − r1 × sin(δ)

P1,y = yc1 + r1 × cos(δ)

P2,x = xc2 − r2 × sin(δ)

P2,y = yc2 + r2 × cos(δ)

(2.14)

The pseudo code for applying the non-uniform condition version of dynamic elliptic

approximation in to the fire model is very similar to that of the uniform version described

before. The only differences lie within the need to calculate ROS for each grid cell and

the use of Eqn 2.11 - 2.14 for calculating tangent line segments.

Require: createEllipseCreate simulation field of x× y grid cellsSet all cell values to 0Set all cell state to unburnedwhile time is less than final time do

if time is 0 thenCall createEllipse on original ignition pointCompute perimeter array based on returned ellipse parameters

elsefor Each point in perimeter array at time t-1 do

Calculate ROS values for the selected pointCall createEllipse with selected point as ignition pointSave ellipse parameters into parameter array

end forfor Each entry in ellipse parameter array do

Calculate outer tangent line between current and next ellipse using Eqn 2.11and 2.14

Save tangent line points in perimeter arrayend forDetermine cell state using same logic as static case

end ifColour simulation field with predetermined colour schemeIncrement time

end while

Figure 2.7 shows the result of applying non-uniform condition extension to the dy-

namic elliptic approximation. The lower right area is impacted by wind blowing hori-

zontally towards the right, whereas the wind direction for the rest of the simulation field

are towards the top right. The ROS can be affected by many different sources, including


wind parameters, fuel parameters, and terrain parameters. The dynamic elliptic approx-

imation can be used to simulate a growing wildfire as long as these parameters can be

quantitatively converted into the three ROS values at each cell location.

Figure 2.7: Fire perimeter simulation under non-uniform conditions

2.2 Edge Detection

The environment being projected for this research project is the aforementioned wildfire

expansion scenario. There are four distinct colours involved in this projection: green,

yellow, red, and brown, as shown in Figure 2.8.

The dynamic wildfire environment will be projected onto a 2m x 2m glass floor, and

a 250mm diameter quadrotor will fly on top and attempt to follow the edge of the fire

from one side to the other. Further details of the experimental set up of the ISP is

provided in Chapter 4. Due to the limited space, a portion of the environment image

from Figure 2.8 would be projected. The camera on board the micro-quadrotor would

capture a portion of the projected image, and sends it to the ground station for edge

detection. In theory, a good image should contain the edge of the fire, as well as both

the green unburned area and the burning area. The tasks of the ground station include


Figure 2.8: The top down view of a wildfire projected onto the ISP.

detecting the edge, finding a target way-point from the detected edge, and converting

the target way-point from image coordinate to the global Optitrack coordinate before

sending it as a way-point command to the autopilot.

Based on the fact that the colour information for the region of interest in the images

is known before hand, a colour-based thresholding edge extraction technique is proposed.

One of the major issues with the captured images is that the image can be noisy due to

the pattern on the semi-transparent film used on the ISP. As a result, pre-filtering the

raw image before running the edge extraction algorithm is required. The proposed steps

for extracting the edge from each image is shown in Figure 2.9.

Figure 2.9: The propsed process of edge extraction

There are a number of smoothing filters that can be used to reduce noise on an

image, such as gaussian filter, median filter, and Bilateral filter. Both the gaussian filter

and median filter blurs the entire image, including the edges. The Bilateral filter, in

contrast, blurs the surface while keeping the sharpness of the edges, making it ideal for

edge detection.


A bilateral filter is used first to remove the texture of the semi-transparent film by

blurring areas with similar colours while keeping edges sharp. Red Green Blue (RGB)

to Hue Saturation Value (HSV) colour-space conversion transforms the image into the

intensity-based colour-space, so that the thresholding process afterwards can pick out

different shades of the same colour easier. Thresholding would then find areas in the

image with red colour and turn that white, while changing everything else to black. Once

the image is in black and white, a Canny edge detector is used to find the edges. Since

Canny edge detection could potentially result in multiple broken edges, morphological

transformation is used to connect these edges based on spacial distance. Finally, contours

are extracted from the image with edge data, in the format of arrays of x and y positions

for further processing. Edge extraction for this research project is implemented with the

OpenCV library.

The bilateral filter contains both a domain filter and a range filter. A domain fil-

ter targets the geometric distance between target pixels and its neighbours; whereas a

range filter targets the photometric similarities [27]. Together, the bilateral filter can be

described by the following equation:

h(x) = k−1(x)

∫ ∞−∞

f(ξ)c(ξ, x)s(f(ξ), f(x))dξ (2.15)

where h(x) is the output image matrix, f(x) is the input image matrix where the target

pixel x is compared with its neighbouring pixels ξ. c(ξ, x) is the geometric distance

between neighbouring pixels ξ and the target pixel x. s(f(ξ), f(x)) is the photometric

similarity, or how closely the two pixels are related in their colour properties. Finally,

k(x) is the normalization term, defined as:

k(x) =

∫ ∞−∞

∫ ∞−∞

c(ξ, x)s(f(ξ), f(x))dξ (2.16)

In essence, the bilateral filter only averages the colour of neighbouring pixels if their

colour properties and distances are closely related. Visually, this means that areas with

similar colours, or non-edges, get averaged; whereas areas with different colours, or edges,

remain sharp.

Thresholding in HSV colour space is a hysteresis process including the hue, saturation,

and value ranges. HSV colour space has the advantage over RGB colour space in that the

colour information is stored in the Hue range alone. The Saturation and Value ranges

indicate the colourfulness and the brightness of the colour defined by Hue. This means

that when thresholding for red colour, as in this research, the Hue hysteresis curve is

the most critical. As long as the Hue is correctly thresholded, Saturation and Value


ranges can be determined by how big the range in different shades of red and brightness

is needed. The sample image shows that although the colour is relatively uniform, the

brightness could change significantly due to the bright spot produced by the projector

bulb. In fact, any region could be turned into a white colour if the bright spot falls on it.

This means that both Saturation and Value range should be relatively large to account

for this variation.

The OpenCV library specifies that each of the three ranges are defined by numbers

from 0 to 179. The red colour in the OpenCV Hue range is between 160 and 179. The

edge extraction program uses this set of minimum and maximum Hue values to find red

areas in the image regardless if it’s lighter or darker in colour. All red pixels are turned

into white and all non-red pixels are turned into black in the binary output.

Canny Edge Detection was originally proposed by Canny as a four step computational

edge detection process [10]. The four steps are Gaussian filter, intensity gradient, non-

maximum suppression, and hysteresis. The Gaussian filter smooths the image; however,

unlike the bilateral filter that is applied before, Gaussian filter averages over the entire

image and blurs the edges as well. An intensity gradient is the first derivative of colour,

and it can be applied to the image to find the rate of change of colour for every pixel. An

edge pixel would have a very high intensity gradient, whereas a surface pixel would have

relatively low intensity gradient. Canny Edge Detector uses the Sobel Operator, which

is based on the intensity gradient of a visual image, for edge detection.

Gx =

−1 0 +1

−2 0 +2

−1 0 +1

(2.17)

Gy =

−1 −2 −1

0 0 0

+1 +2 +1

(2.18)

The intensity gradient has two directions: x and y direction. The overall gradient

strength and direction of an edge can be calculated as:

G =√G2x +G2

y (2.19)

θ = arctan(Gy

Gx

) (2.20)

The combination of non-maximum suppression and hysteresis techniques ensure that

false edges are eliminated from the final result. A single pixel is used to represent each po-


tential edge with the application of non-maximum suppression, following which hysteresis

eliminates potential edge pixels based on a set of minimum and maximum threshold val-

ues that defines edge pixels. Edge pixels lower than the minimum threshold value or

higher than the maximum value are considered non-edges. Those that lie within the

threshold range are only considered an edge if they are connected to another edge.

The OpenCV library contains pre-made functions for many of the filters used in the

edge detection process, including bilateral filter and canny edge detector. In an OpenCV

implementation, converting from edge images to contour arrays is necessary if further

processing on the extracted edge is needed, as arrays of x and y values are much more

useful than a binary image of 1’s and 0’s. The concept of determining a new target

way-point is shown in Figure 2.10.

(a) Position of camera and field of view (b) Edge image captured by the camera

Figure 2.10: Side profile of quadrotor and location of new way-point on image

While the quadrotor system will be discussed in detail in Chapter 4, it is important to

note where the camera is mounted. The left image shows the position of the camera with

respect to the quadrotor, and the camera’s field of view. Since the camera is mounted

towards the front of the quadrotor, the vertical centre of this image is at a location in

front of the quadrotor’s position. This is illustrated on the right image, where the blue

dotted line is the vertical centre of the image and the black dotted line below is the centre

of the quadrotor. Assuming that the fire edge is in the image, and that the quadrotor is

flying relatively parallel to the fire edge, then the red solid curve is the fire edge that is

in view of the camera. The position of the quadrotor would be the intersection between

the horizontal centre line and the black dotted line. The intersection between the fire

edge and the vertical centre line, the green dot, would be a position on the fire edge but

in front of the quadrotor. If this green dot acts as the next way-point, then it can be

certain that the quadrotor would always move forward and track the fire edge.

The pseudo code for an OpenCV implementation of the fire edge extraction algorithm

is presented below, which would extract the fire edge from the image and determine the


location of the next way-point in the image coordinate.

Image resolution is 640× 480while exit key not pressed do

Receive RGB image from cameraUse Bilateral Filter to remove noiseConvert from RGB to HSV colour spaceThreshold image for red colour, return binary imageUse Canny Edge Detector, return binary image with white edges on black back-

groundPerform morphological transformation to remove gaps in edgesConvert edges from binary image to arrays of contoursFind longest contourFind the point from the longest contour with y value of 240

end while

2.3 Perimeter Reconstruction

The ultimate goal of the fire perimeter tracking mission is to recreate the fire perimeter

and be able to both move backwards to see the history of the fire progression and move

forward to predict the fire growth. The quadrotor that flies alongside the fire would result

in a spiral path as the fire continuously expands over time, similar to the one shown in

Figure 2.11. Reverse elliptic approximation is a method developed to synchronize all the

location data captured over a time period, similar to the spiral UAV path in Figure 2.11,

into a single time instance.

2.3.1 Reverse Elliptic Approximation

Reverse elliptic approximation is fundamentally the same as the dynamic elliptic ap-

proximation, except that it works in reverse. The reverse elliptic approximation steps

backwards in time instead of forward, finding the inner tangents as opposed to the outer

tangents of two ellipses. This is accomplished by using a negative angle α in Equa-

tion 2.10. Figure 2.12 shows the result of applying the reverse elliptic approximation

procedure after a forward elliptic approximation.

Figure 2.12a shows a forward elliptic approximation from the original fire perimeter in

black to the new fire perimeter in blue; Figure 2.12b shows a reverse elliptic approximation

being applied to the new fire perimeter; and Figure 2.12c shows the difference between

the original fire perimeter in black and the new approximated perimeter at the same


Figure 2.11: The spiral path created by the UAV when tracking a continuously expandingfire perimeter

time instance in red. Any difference between the black and red perimeters are due to

perimeter discretization.

If the fire is assumed to be much larger than the distance that a single UAV can

cover, then the local fire perimeter at different time instances can be approximated to be

parallel to each other, as illustrated in Figure 2.13.

P1, P2, and P3 are three unique positions on the fire perimeter. The fire perimeter

expands upwards with time, and the UAV tracks the fire perimeter from right to left.

The UAV reaches point P1 at time t, point P2 at time t + ∆t, and finally P3 at time

t + 2∆t. The overall UAV path is indicated by the solid arrow. The recorded positions

of the fire perimeter from the UAV’s perspective are: P1,t, P2,t+∆t, and P3,t+2∆t. In order

to synchronize the three positions into a single time frame, a series of reverse elliptic

approximations need to be applied. The reverse elliptic approximations used here are

slightly modified, in that the line segment to be determined is the tangent to an ellipse

and an adjacent point. Based on the scenario in Figure 2.13, for example, the red ellipse

with an ignition point at P2,t+∆t can be created. It would have its head direction pointed

downward. The tangent between point P1,t and this ellipse would form the line P1,tP2,t.


(a) Forward Approximation (b) Reverse Approximation (c) Result Comparison

Figure 2.12: The reverse elliptic approximation result and comparison

(a) (b)

Figure 2.13: The path of a UAV tracking the fire front over three time steps

In this example, P2,t+∆t would be called the Reverse Ignition Point, P1,t is the Static

Point, and P2,t is the Intersection Point. Applying this technique twice on P3,t+2∆t would

create the line P2,tP3,t. This line segment can be combined with the line segment P1,tP2,t

to create the desired local fire front at time t. The pseudo code for calculating the fire

perimeter at time t = 0 using the UAV’s location data from time t = tfinal is as follows:

2.3.2 Full Perimeter Reconstruction

If the quadrotor is capable of perfect tracking, then the result of the reverse elliptic ap-

proximation would be the same as the fire perimeter that has been tracked. However,

perfect tracking does not exist, and the position data obtained from a quadrotor prob-

ably does not describe the fire perimeter perfectly. Elliptic curve fitting and 2nd order

polynomial curve fitting are two methods can be used to approximate the fire perimeter

being tracked.

Assuming that the overall fire perimeter grows in an elliptic fashion, then elliptic

curve fitting would provide an approximation on the entire fire perimeter, as opposed to


UAVPath (size 2×n) contains UAV’s x and y position data, where n = tfinal× intervalCreate empty UAVX and UAVY matrices (size n× n)for k increments from 1 to n do

UAVX k’th diagonal value = UAVPath x at index kUAVY k’th diagonal value = UAVPath y at index k

end forfor i increments from 2 and n do

for j decrements from i to 2 doFind head, tail, side ROS at current timeReverse Ignition Point P1 = (UAVX(j, i), UAV Y (j, i))Static Point P2 = (UAV X(j − 1, i− 1), UAV Y (j − 1, i− 1))Apply reverse elliptic approximation on P1

Find tangent line between ellipse and P2, where Intersection Point Pinter is thepoint where the tangent line intersects the ellipse

UAV X(j − 1, i)andUAV Y (j − 1, i) are defined by Pinterend for

end for

only the section that the quadrotor has monitored. The quadratic form of the general

ellipse equation is as follows:

Ax2 + 2Bxy + Cy2 + 2Dx+ 2Fy +G = 0 (2.21)

where x and y are points representing the ellipse, and A,B,C,D, F,G are parameters

that need to be calculated. This means that a minimum of six unique points that lie

on the ellipse are needed. A visualization of the ellipse can be drawn once these six

parameters are obtained. The centre of the ellipse is:x = CD−BFB2−AC

y = AF−BDB2−AC

(2.22)

and the semi-axes lengths of the ellipse are:a =

√2(AF 2+CD2+GB2−2BDF−ACG)

(B2−AC)[√

(A−C)2+4B2−(A+C)]

b =

√2(AF 2+CD2+GB2−2BDF−ACG)

(B2−AC)[−√

(A−C)2+4B2−(A+C)]

(2.23)

Finally, the angle between the major axis of the ellipse and the x-axis can be calculated

with one of the four cases:


θellipse =

0 if b = 0 and a < c

12π if B = 0 and A > C

12

cot−1(A−C2B

) if B 6= 0 and A < C

π2

+ 12

cot−1(A−C2B

) if B 6= 0 and A > C

(2.24)

Due to the fact that the ellipse that is being fitted is free in size, location and angle, the

best fitted ellipse may not necessarily be the closest to the original fire perimeter if there

are noises in the data. In an extreme example, it is possible for the fitted ellipse to be 90

degrees offset from the actual perimeter. This means that the data that are suppose to

be on the major axis are fitted to the minor axis, making the fitted ellipse very different

from the original. In these cases, the approximated perimeter that are further away from

the location of data collection will contain too much error to be of any use. If the segment

that is close to the data collection location is the only portion that is useful, then the

curve can also be approximated using a 2nd order polynomial curve fitting technique.

2nd order polynomial curve fitting is much simpler. The general 2nd order polynomial

equation is as follows:

y = Ax2 +Bx+ C (2.25)

2nd order polynomial curve fitting requires a minimum of 3 sets of data points, and

the data can be fitted using the least square method. It should provide a result that is

very close to the tracked segment of the actual fire perimeter.

2.4 Summary

An elliptic approximation fire model is chosen for generating the dynamically growing

wildfire because it provides the position of the fire perimeter without the need to create

a temperature model for the environment. This elliptic approximation model can be

expanded to include both dynamic and non-uniform field conditions. The edge detection

method proposed for this project includes a bilateral filter to pre-process the image, a

Canny Edge Detector to extract the edges, and a few other post-process filters to ensure

that only a specific edge is being extracted. Both elliptic curve fitting and 2nd order

polynomial curve fitting have been proposed for reconstructing the perimeter. Elliptic

curve fitting can provide a good estimate of the entire fire perimeter if the noise in data

is low. If the data contains a lot of noise, however, it is possible that elliptic curve

fitting does not return a valid solution. Additionally, fire does not necessarily grow in an


elliptic fashion, which prevents effective prediction of how the fire expands in areas not

monitored by the quadrotor. In these scenarios, 2nd order polynomial curve fitting may

be a better alternative. 2nd order polynomial can only provide a good approximation of

the section of the fire perimeter that has been monitored, but it is much less susceptible

to noise.

Chapter 3

Quadrotor Navigation

A quadrotor dynamic model and controllers are required to test the navigation algorithm

in simulation. The quadrotor model can be derived from basic physics laws such as the

Newton’s Second Law. The controller structure used in the simulation will match that of

the actual autopilot for the micro-quadrotor. Finally, a PBVS navigation method will be

used in conjunction with edge detection to tracking the fire perimeter via visual camera.

3.1 Quadrotor Model

A standard non-linear quadrotor model is used for fire edge tracking simulation in MAT-

LAB. While many papers use the simpler “plus” configuration of quadrotor model, a

slightly more complex “X” type quadrotor model is needed for this project in order to

match with the hardware configuration. The “X” configuration is used on the micro-

quadrotor so that the camera can be mounted towards the front of the vehicle without

significantly disrupting air flow from the propellers. ArduPilot Mega is an open source

autopilot that is being used on the physical micro-quadrotor [26]. The ArduPilot Mega

has its own way of numbering the motors, and the simulation model will reflect this cus-

tom motor order for consistency. Figure 3.1 shows the APM version of the “X” quadrotor

configuration.

Quadrotor attitude and position can be calculated based on the thrust and the torque

each propeller produces during flight. The translational and rotational dynamics of a

quadrotor can be modelled separately. Figure 3.2 shows the principles behind quadrotor

dynamics.

Each motor provides a certain amount of torque to the rotor, and the rotor creates an

upward thrust, indicated by red arrows, as well as a certain amount of torque, indicated

by blue arrows. The quadrotor rolls and pitchs by creating a difference in thrust between

30

Chapter 3. Quadrotor Navigation 31

Figure 3.1: The “X” configuration used by the ArduPilot Mega [26]

Figure 3.2: Principle of quadrotor dynamics

pairs of motors. For example, the quadrotor rolls to the left if motors 2 and 3 generate

less thrust than 1 and 4. At the same time, the quadrotor would start moving towards

the left because the total thrust vector is always perpendicular to the quadrotor frame.

The quadrotor can yaw counter-clockwise if the motors that generate counter-clockwise

torques are faster than those that generate clockwise force. In this case, a counter-

clockwise yaw can be produce by having motors 1 and 2 turn faster than 3 and 4,

creating a difference in overall torque.

3.1.1 Translational Dynamics

The dynamics of a quadrotor are governed by Newton’s Second Law, F = ma, where m

is the mass of the quadrotor. This can be expanded in the inertial reference frame:


xyz

=1

m

FxFyFz

(3.1)

The thrusts provided by the motors, F1−4, are defined in the body-fixed frame. Thus,

the rotation matrix RIB1 transforming from body-fixed frame to inertial frame is required.

RIB1 =

cψcθ −sψcφ + cψsθsφ sψsφ + cψsθcφ

sψcθ cψcφ + sψsθsφ cψsφ + sψsθcφ

−sθ cθsφ cθcφ

(3.2)

where c denotes cos and s denotes sin. φ is the roll angle, θ is the pitch angle, and ψ is

the yaw angle of the quadrotor. The overall translational dynamics becomes:xyz

=1

mRIB1

0

0

F1 + F2 + F3 + F4

− 0

0

mg

(3.3)

This can be simplified into three equations that describes the quadrotor’s translational

motion: x = 1

m(sin(φ)sin(ψ) + cos(φ)cos(ψ)sin(θ))U1

y = 1m

(cos(φ)sin(θ)sin(ψ)− cos(ψ)sin(θ))U1

z = 1mcos(θ)cos(φ)U1 − g

(3.4)

where U1 = F1 + F2 + F3 + F4.

3.1.2 Rotational Dynamics

The rotational dynamics of a rigid body in the body frame is governed by the Euler’s

equation:

T = IRα + ω × IRω (3.5)

where T is the torque, α is the angular acceleration, and ω is the angular rate in body-

fixed frame. IR is the moment of inertia matrix and is defined as:

IR =

Ixx Ixy Ixz

Iyx Iyy Iyz

Izx Izy Izz

(3.6)


where Ixx, Iyy, Iz are the moments of inertia of the quadrotor with respect to the roll,

pitch, and yaw axis. The rest of the terms in the matrix are the products of inertia.

Due to the fact that quadrotors have mirror symmetry on the XZ and Y Z planes, the

products of inertia are 0. Expanding the above equation would result in the following

equations in terms of angular rates p, q, r in the body-fixed frame:p = 1

Ixx(Mx − (Izz − Iyy)qr)

q = 1Iyy

(My − (Ixx− Izz)pr)

r = 1IzzMz

(3.7)

where Mx,My,Mz are the moments with respect to the body frame. To convert angular

rates in the body-fixed frame to Euler angular rates, θ, φ, ψ in inertial frame, the rotation

matrix RIB2 is used:

RIB2 =

1 sφtθ cφtθ

0 cφ −sφ0

sφcθ

sθcθ

(3.8)

φθψ

= RBI2

pqr

(3.9)

where s denotes sin, c denotes cos, and t denotes tan.

3.1.3 Motion Equations

Combining the translational and rotational dynamic equations, a quadrotor’s motions

can be described by the following set of equations:


x = 1m

(sin(φ)sin(ψ) + cos(φ)cos(ψ)sin(θ))U1

y = 1m



p = 1Ixx

(U2 − (Izz − Iyy)qr)

q = 1Iyy

(U3 − (Ixx − Izz)pr)

r = 1IzzU4

φ = p+ qsin(φ)tan(θ) + rcos(φ)tan(θ)

θ = qcos(φ)− rsin(φ)

ψ = q sin(φ)cos(θ)

+ r sin(θ)cos(θ)

(3.10)

The inputs, Ui, are intermediate values and are defined as:

U1 =∑4

i=1 Fi

U2 = l(−Fleft + Fright)

U3 = l(Ffront − Fback)

U4 = KtorqueKthrust

(F1 + F2 − F3 − F4)

(3.11)

Ffront = F1 + F3

Fback = F2 + F4

Fleft = F2 + F3

Fright = F1 + F4

(3.12)

Since the rotors of a quadrotor are mostly perpendicular to the direction of travel, the

impact of air speed through the rotors on the thrust they generate is small. In addition,

translational velocity of the quadrotor is very low in the indoor space, so that any changes

in thrust due to change in air flow are also small and negligible. The thrust and torque

can be approximated by the following relationship with the propellers’ angular velocity

Ωi. l is the length of the motor arm of the quadrotor.Fi = KthrustΩ2i

Ti = KtorqueΩ2i

(3.13)

Due to limited space in the indoor environment that the ISP resides in, the body angles

of the quadrotor can be assumed to be small. As a result, the Eular angular rates φ, θ,

ψ and the body angular rates p, q, r can be approximated to be the same. Additionally,


gyroscopic effects generated by moments of inertia in the off axes are negligible when the

angular rates in all three axes are small. As a result, Eqn 3.10 can be simplified to the

following:

x = 1m

(sin(φ)sin(ψ) + cos(φ)cos(ψ)sin(θ))U1

y = 1m



φ = 1IxxU2

θ = 1IyyU3

ψ = 1IzzU4

(3.14)

The mass moments of inertia of the quadrotor on the x, y, and z axis can be approx-

imated with a disk, such that:

Ixx = Iyy =1

2Izz (3.15)

3.2 Quadrotor Control

The quadrotor has four output states: longitudinal position, lateral position, altitude,

and heading angle. Longitudinal position is controlled by the pitch angle of the quadrotor,

while lateral position is controlled by the roll angle. Pitch, roll, yaw, and altitude are

determined by the thrust on each motor. The overall control structure is provided by the

ArduPilot, and the controllers in the simulation are made to match this structure. The

controller structure is shown in Figure 3.3.

The quadrotor is entirely controlled by PID controllers. Four control loops are used for

longitudinal and lateral position control, shown in Figure 3.3 as XB and YB respectively,

while two are used for altitude and heading angle control. A PI controller is used to

convert displacement error to desired velocity, as well as to convert angular error to

desired angular velocity. The PI controller ensures that the relatively slow changing

parameters, position and angle, have no steady state error. In real world operations,

flying in constant wind can result in steady state error in both quadrotor’s pitch angle,

roll angle, and horizontal positions, if the integral terms are not tuned. In contrast, a PID

controller is used for horizontal velocity and angular rate. These controllers are designed

to provide quick responses to the relatively fast changing parameters. The heading angle

control structure is similar to that of the pitch or roll angle. There is a PI controller for

angle error, and a PID controller for angular rate error. The altitude control structure,

on the other hand, is similar to the outer loop of horizontal position control, which has


Figure 3.3: Controller structure used for the quadrotor

a PI controller for altitude error and a PID controller for climb rate. The PID gains

are custom tuned for the simulation and hardware setup. The tuning for the numerical

simulation model is done mostly by analyzing the step response behavior, whereas the

tuning for the micro-quadrotor is done by a combination of analyzing behavior based on

telemetry log files and trial and error.

On the ArduPilot, the IMU on the autopilot outputs the angles θ, φ, ψ and the angular

rates θ, φ, ψ. The inertial navigation, on the other hand, estimates the position X, Y, Z

and linear velocity X, Y , Z of the quadrotor in both the body-fixed frame and the inertial

frame.

Both the simulated autopilot and the ArduPilot need to be able to operate in two

modes: way-point command mode and velocity command mode. In way-point command

mode, the user specifies way-points for the quadrotor. A way-point contains X, Y, and

Z parameters. The way-point command mode can be used by any generic navigation

algorithms that has target locations as outputs. The longitudinal error represents how far

the quadrotor is from the target; and the lateral error represents how far the quadrotor has

drifted from its intended path between two way-points. In contrast, velocity command

mode is used when the user not only cares about where the quadrotor needs to go, but

also how it gets there. Many edge tracking based navigation algorithms uses velocity

command mode to ensure that the vehicle moves along the edge at a constant velocity


and the vehicle does not deviate from the edge being tracked. In this case, the horizontal

velocities are constrained, which requires the full PID controllers.

3.3 PBVS Navigation

Position Based Visual Servoing (PBVS) has been chosen as the navigation technique over

Image Based Visual Servoing (IBVS) for this thesis project primarily because the position

of the target, which in this case is the fire perimeter, is desired for post experiment data

processing. In this case, PBVS provides this position data while navigating. Figure 3.4

shows how the PBVS navigation algorithm works. Additionally, the ArduPilot is set up

as a way-point based autopilot that responds to way-point commands.

Figure 3.4: PBVS flow chart

A PBVS navigation algorithm can be divided into three parts. Firstly, the vehicle

needs to have an autopilot that uses the way-point as navigation commands. Secondly,

there needs to be a way to determine the target way-point. Lastly, the target way-points

have to be transformed into the inertial coordinates. In this particular case, both the

autopilot in the simulation and ArduPilot do indeed work with way-point commands.

The target way-point is determined by the edge detection algorithm described in Section

2.2 in experiment, and is simulated with the actual fire perimeter with Gaussian noise

added in MATLAB simulation. The edge detection algorithm outputs way-points in

the image coordinate, but ArduPilot requires target way-points in the format of x and

y position, in the inertial coordinates that is being used by Optitrack. Therefore, a

reference frame transformation is needed to connect the two existing blocks.

3.3.1 Reference Frame Transformation

There are three steps involved in transforming a set of coordinates from the image refer-

ence frame into the global reference frame. Assuming that the camera is pointed vertically

downwards when an image is taken, then any pixel that is not at the centre of the image

could be represented by a variation in pitch and roll angles, ∆θi and ∆φi where subscript

i denotes the image frame, as shown in Figure 3.5.


Figure 3.5: The image reference frame

C is the centre of the image, P is the point of interest, and O is the origin in the

image frame. ∆θi and ∆φi are calculated by first converting all pixel distances in image

frame into distances in the global frame via a pixel-to-centimeter ratio, and then taking

the arctangent of the horizontal and vertical distances. Although the pixel to centimetre

ratio is different depending on the altitude of the camera, the angles created by the

distance are the same because they form similar triangles, as shown in Figure 3.6. Thus,

the angles can be calculated using a specific pixel-centimetre ratio with a specific altitude

regardless of the altitude that the image is actually taken at.

Figure 3.6: The pixel-centimetre ratio at a certain altitude

The image that the on board camera captures has a resolution of 640× 480px, which

means that xi = 640px and yi = 480px. When the camera is pointed vertically downwards

at a height of h = 35cm, the centimetre distances that is captured by the camera has


been experimentally determined to be x = 42cm and y = 31.5cm. The calculations for

∆θ and ∆φ using this set of experimental data are as follows:∆θi = atan(∆yi ×K)

∆φi = atan(∆xi ×K)

K = 31.5(480×35)

(3.16)

where K is a constant that is related to the pixel-centimetre ratio and the specific altitude

for this ratio.

Due to the fact that camera is fixed downwards in relation to the quadrotor frame

and the projection plane is flat, any tilt on the image frame relative to the global frame

are induced by the pitch and roll of the quadrotor, as illustrated in Figure 3.7.

Figure 3.7: The contribution of quadrotor’s pitch angle to total pitch angle difference fortarget

The offset distance between the centre of the camera and the centre of the quadrotor is

very small in relation to the altitude that the quadrotor is flying at; thus, the camera can

be assumed to be at the centre of the quadrotor for this reference frame transformation

process. Based on Figure 3.7, the total pitch and roll angle needed to reach the target

is the combination of the quadrotor’s pitch and roll (θ and φ), and the target’s relative

pitch and roll in the image frame (∆θi and ∆φi):θtotal = θ −∆θi

φtotal = φ−∆φi(3.17)

In this particular case, a position pitch angle in the image plane corresponds to a


negative pitch angle in the inertial frame of the quadrotor. The same rotation matrix

between body frame and inertia frame (Eqn 3.2) can be applied to transform the angles

experienced in the image frame into the global frame. Both the x, y, z position and the

φ, θ, ψ attitude angles of the quadrotor are known, which means that the reference frame

transformation can be done with the following calculations:R1 = cos(φtotal) sin(θtotal) cos(ψ)− sin(φtotal) sin(ψ)

R2 = cos(φtotal) sin(θtotal) sin(ψ) + sin(φtotal) cos(ψ)

R3 = cos(θtotal) cos(φtotal)

(3.18)

xtarget = xquad + L×R1

ytarget = yquad + L×R2

ztarget = zquad + L×R3

(3.19)

Although the parameter L is an unknown, the altitude of quadrotor is known and the

altitude of the target is zero. Therefore, parameter L can be calculated as:

L =−zquadR3

(3.20)

The pseudo code for PBVS implementation in MATLAB can be found below. The

PBVS navigation implementation for the physical experiment uses the same concept, but

is different because the Indoor Simulation Platform is an integration of many separate

components. As a result, data needs to be sent between components to ensure proper

execution of the algorithm. The implementation of this algorithm on the physical system

will be discussed in the next chapter.

while time < tfinal doFind fire edge in pre-defined image windowDetermine next target way-point based on intersectionTransform way-point to global coordinateSend new way-point command to autopilotwhile way-point has not been reached do

Check distance to way-pointend while

end while


3.4 Parameter Estimation

The above simulations are completed with the theoretical quadrotor model, where var-

ious parameters are estimated based on geometry of the physical quadrotor. Some of

these parameters, such as mass, can be easily measured; however, other parameters, such

as moments of inertial, are more difficult. A more accurate parameter estimation of the

quadrotor can be accomplished by using maximum likelihood method. This method has

been investigated by Marlon Koga, a summer intern at the FSC lab. This method is

based on the idea that a mathematical model can be found given the input sequence

to the autopilot and the physical output results from the quadrotor [28]. In theory, the

calculations for the model parameters would be trivial. In the real world, however, the

noise in the physical system means that Kalman Filter is required to estimate these pa-

rameters. The new state-space equations for a system that contains random disturbance

is as follows:

x(t) = Ax(t) +Bu(t) +Bww(t) (3.21)

y(t) = Cx(t) +Du(t) (3.22)

z(i) = y(i) + v(i), i = 1, 2, 3, 4...N (3.23)

where Bw is the disturbance matrix, w(t) is the random disturbances, z(i) is the mea-

surements, and v(i) is the measurement noise given in discrete time steps. Both w(t)

and v(i) are Gaussian distributions. The likelihood for a series of measurements ZN =

[z(1), z(2), ..., z(N)]T given a theoretical model M and a set of inputs is given by the

following equation:

L(ZN |M) =n∏i=1

(2π)−n0/2|β(i)|−1/2exp[−1

2νT (i)β−1ν(i)] (3.24)

where the predicted measurement yp, innovation ν, and innovation covariance matrix β

are defined as:

yp ≡ E[z(i)|Zi−1,M ] = E[z(i)|M ] (3.25)

ν(i) ≡ z(i)− yp(i) (3.26)

β(i) ≡ Cov[z(i)|Zi−1,M ] = E[ν(i)νT (i)] (3.27)

The desired system parameters γ that approximates the real parameters γ can be

found by maximizing the likelihood equation:


γ = argmax(L(ZN |M)) = argmin1

2

N∑i=1

[νT (i)β−1(i)ν(i) + ln|β(i)|] (3.28)

Thus, the problem has been reduced to a form that supports a discrete time Kalman

Filter algorithm where the prediction equations are:

xp(i) = Adx(i− 1) +Bdu(i− 1) (3.29)

Pp(i) = AdP (i− 1)ATd +BwdQBTwd (3.30)

innovations are:

β(i) = CdPp(i)CTd +R (3.31)

ν(i) = z(i)− Cdxp(i)−Ddu(i) (3.32)

and the measurement updates are:

x(i) = xp(i) +K(i)ν(i) (3.33)

K(i) = Pp(i)CTd β−1(i) (3.34)

P (i) = [I −K(i)Cd]Pp(i) (3.35)

where Ad, Bd, Cd, Dd and Bwd are the discrete time version of the state space matrices.

Figure 3.8 shows the block diagram for parameter estimation with experimental data,

incorporating the Kalman Filter estimation method.

The experimental input data consists of the 4 desired forces the autopilot sends to the

motors. The output data are the angles and angular rates from the IMU, the position

Optitrack, and estimated linear velocities from inertial navigation. These data are not

logged at the same frequency; thus, data synchronization is needed to match the frequency

of the output data. In this case, all data are modified to have a frequency of 50Hz, even

if they are actually logged at a lower frequency. The experimental inputs are passed into

the Kalman Filter as inputs, and the predictions of the Kalman Filter are compared with

the experimental output data. The cost function and the model are updated based on the

discrepancies between the two outputs. Three parameters are subjected to estimation:

m, Ixx, and Iyy. The result of this estimation are as follows:


Figure 3.8: The experimental parameter estimation block diagram [28]

m = 511grams

Ixx = 0.0010kgm2

Iyy = 0.0014kgm2

(3.36)

The mass parameter can be experimentally verified to be highly accurate. The actual

weight of the quadrotor is measured to be 510 grams. There are two ways of calculating

the moments of inertia theoretically. The first method involve breaking up the quadrotor

into separate simple geometry components and add up all the individual moments of

inertia. The second method simple approximates the quadrotor as a disk, and calculate

the moments of inertia using the volume and dimensions of the quadrotor. Although the

break-up method would provide a more accurate result, both methods would normally

produce the same value for Ixx and Iyy, and neither can fully take into account of the

mechanical imperfections of the physical hardware. The quadrotor has many different

components that are not mounted at the centre of gravity, such as batteries, radio, and

camera. While much effort has been made to balance the centre of gravity, there is

only so much that can be done given the limited space on the micro-quadrotors and

the required payload for this mission. Therefore, the moments of inertia for the actual

quadrotor on the roll and pitch axes are most likely not the same. The experimentally

estimated parameters above indicate this important difference. Due to the fact that the

quadrotor used in indoor experiments have gone through multiple hardware upgrades,


this estimation is only done on the final hardware setup. Therefore, this estimation result

is not used in the numerical simulations for this thesis project.

3.5 Summary

The quadrotor dynamic model used in the numerical simulation is the “X” configuration

of the simplified non-linear model used by many other researchers. The “X” configuration

ensures maximum thrust with the camera mounted in front of the micro-quadrotor. A

set of intermediate inputs is needed for the “X” configuration compared to the standard

“Plus” configuration. The controller has four control signals controlling four states: X

position, Y position, altitude, and heading angle. X and Y are the left and forward

directions in body-fixed frame respectively, which translate into desired roll and pitch

angles. The controller gains for the simulation model and the physical micro-quadrotor

are tuned separately based on step response behaviors. PBVS navigation has been chosen

for the fire edge tracking mission. The input of the PBVS algorithm is target way-point

provided by edge detection. A reference frame transformation is required to transform

the target way-point from image frame in pixels to the global frame in cm. A parameter

estimation method has been investigated by the summer student Marlon, and has proven

to be accurate in estimating the mass and moments of inertia of the quadrotor being

used for experiments.

Chapter 4

Indoor Simulation Platform

The ISP is an integrated hardware-in-the-loop experimentation environment being de-

veloped at the Flight Systems and Control (FSC) Lab. The system used for this thesis

project is the first iteration in both hardware and software, and serves as a proof of con-

cept for future developments. This version of ISP is capable of generating a simulated

environment and having a single micro-quadrotor navigate within the environment via

visual feedback.

4.1 Hardware

There are four major hardware components and four major software components for the

ISP. The four hardware components are: a quadrotor that would act as the vehicle for

carrying out various navigation missions, an environment projection that can simulate

a top down view of any environment, an indoor navigation system that helps locate the

quadrotor, and finally a ground station that is capable of synchronizing all the data and

commands the quadrotor. The overall system schematic can be seen in Figure 4.1.

The black dotted box encircles everything that is on board the micro-quadrotor, which

includes the autopilot, various sensors, and other quadrotor hardware. The black arrows

indicate wired connections; the blue dotted arrows show wireless connections; and the

red dotted arrows describe relationships other than data transmission.

The environment projection platform was designed and constructed previously by

Yasir Malang, a summer intern at the FSC lab. It consists of an angled projector that

projects the image downwards at a mirror. The image is reflected by the mirror upwards

to a semi-transparent film under a large piece of glass floor. The projection platform

setup can be see in Figure 4.2.

An example of the projected environment can be seen in Figure 4.3. The projected

45

Chapter 4. Indoor Simulation Platform 46

Figure 4.1: The overall system schematic of the ISP.

Figure 4.2: The environment projection platform with angled projection.

elliptical fire edge can be seen from the top of the glass floor, so that the image can be

captured by a downward looking camera mounted on the quadrotor for position feedback.

The glass floor measures about 2m × 2m, which limits horizontal dimension of the

simulation space. The glass floor is also 1.5m above ground, due to the angled mirror

assembly underneath. This restricts the vertical dimension of the simulation space to no

more than 2m. A set of six Optitrack cameras are used as the indoor positioning system.

The cameras are V100:R2 models. In order for the cameras to provide a trackable area

of 2m × 2m horizontally, the vertical tracking is essentially limited to a maximum of 1m.

The lack of large open space in the lab limits the size of the aerial vehicle used on in

the experiment to a micro-sized quadrotor. A micro-quadrotor has very limited payload


Figure 4.3: An example of the projected wildfire environment.

capacity, which has to be split between an autopilot, a battery, and a camera system. In

order to ensure reasonable flight time, the autopilot and the camera system would have

to be small and lightweight. A small autopilot means limited computational resources

and some of the more resource intensive operations, such as image processing, would have

to be done on the ground station instead. In turn, this means that data transmission

between the ground station and the autopilot is crucial. A lightweight camera system,

on the other hand, means that the image quality would suffer, and that there would be

very limited, if any, physical stabilization of the camera. With these operational limits

in mind, a micro-sized quadrotor with diagonal arm length of less than 250mm is needed

for ISP operation.

A number of hardware options have been considered for the ISP micro quadrotor.

The main constraints on selecting an appropriate micro quadrotor for this project are

the size, payload, and endurance. The micro quadrotor must have a diameter no more

than 250mm; it must be able to carry a fully functional autopilot that is capable of

carrying out high level navigation commands, as well as a small camera with associated

video transmission module; finally, it should have a maximum flight time of at least 5

minutes.

The quadrotor frame and autopilot controller are selected separately as there are a

number of mix and match combinations. The frame selection included a FPV250 frame

from HobbyKing, Kmel Robotics quadrotor package, and a custom built carbon fiber

frame. The Kmel Robotics package, which contains both a carbon fibre frame and a

Gumstix based autopilot, was rejected because a flexible open source system is desired.

The custom built carbon fibre frame was chosen initially due to the flexible nature of

the design. Although there are a number of autopilot options to choose from within

the price range, only the ArduPilot is open source in both software and hardware. As

such, the ArduPilot is chosen as the autopilot for the ISP. Two carbon fibre frames


were constructed for ISP; however, the design and available materials could not meet

the vibration requirements of the ArduPilot. A switch is made to the FPV250 frame,

and a number of damping solutions are applied on the ArduPilot, so that the vibration

requirements could finally be met.

(a) Custom built frame (b) FPV250 frame

Figure 4.4: The two quadrotor frames that have been used in this project

Figure 4.4 shows the two quadrotor frames that have been tested. The one on the left

is the custom built carbon fibre frame that ultimately did not pass the vibration test.

The one on the right is the FPV250 frame from HobbyKing, which is currently used on

the ISP. The FPV250 frame has the following technical specifications:

Manufacturer HobbyKingConfiguration Quad X

Diagonal Length 250mmPropeller 5 x 3

Motor Multistar 1704Electronic Speed Controller 12Amp ESC

Battery 1300mAH 3 Cell LiPoTotal Weight 450gEndurance 5minAutopilot ArduPilot Mega 2.6

Table 4.1: Hardware specifications for FPV259 quadrotor frame

The ArduPilot Mega 2.6 (APM) is an open source autopilot module. It has the

following specifications:

The ArduPilot has an integrated Inertial Measurement Unit (IMU) that contains a

3-axis rate gyro and accelerometer. Additionally, it uses Direction Cosine Matrix (DCM)

for attitude estimation and inertial navigation. It has an internal task scheduler and


Manufacturer 3D Robotics

Sensors

3-Axis Gyro3-Axis Accelerometer

CompassBarometric Altimeter

Battery Current SensorOptional GPS

Chipset ATMega 2560On Board Memory 4MB Dataflash Chip

Table 4.2: Hardware specifications for ArduPilot Mega 2.6

runs four loops at different frequencies. The fastest loop, running at a frequency of

100Hz, is the IMU loop. Its tasks include monitoring and logging IMU data as well as

motor control. The medium loop, running at 50Hz, contains inner loop controls such

as pitch and roll angle controllers. The slow loop, running at 10Hz, handles navigation

control. Lastly, the very slow loop running at 1Hz is tasked with miscellaneous activities

that are not as time sensitive as quadrotor navigation, such as some of the logging and

communication tasks.

4.2 Software

The software, on the other hand, includes an autopilot that can be modified for various

navigation algorithms, an image processing module for controller feedback, an environ-

ment simulation module to generate the environment projection, and data synchroniza-

tion on the ground station to ensure that every module is communicating with each other

properly. Figure 4.5 shows the relationship between different ISP modules that need to

be developed. The blue dotted box is the software package on board the micro quadrotor,

whereas the red dotted box is the software on the ground station.

The software package being carried by the micro quadrotor consists of the autopilot

and a video transmission module. The images taken by the camera is transmitted to a

receiver that is attached to the ground station computer wirelessly. The autopilot controls

the movement of the quadrotor, which includes both high level navigation controllers

and low level stabilization controllers. The navigation commands are generated by the

ground station and sent to the autopilot. Due to the limited size and payload capability

of the micro quadrotor, the on board computational resource is very limited. As a result,

some essential calculations, such as position tracking, image processing, and navigation


Figure 4.5: Structure of the ISP software system

command generation, would be done remotely on the ground station.

The ground station software used for ISP is a modified version of the Mission Planner

(MP), which is the default open source ground station software used by ArduPilot. MP

is made up of multiple threads. The most important of these is the Mavlink thread.

Mavlink is the data transmission protocol used by the Ardupilot to communicate with

its proprietory ground station software, Mission Planner(MP). Mavlink itself is an open

source communication library developed with UAV data transmission as its primary

purpose. The Mavlink thread handles the flow of data between the autopilot and the

ground station. Information such as quadrotor attitude would be received by the ground

station; and in return, navigation commands would be transmitted to the quadrotor.

The radio telemetry that handles the transmission of data operates at 900MHz and has

a baud rate of 57600 bits per second. Due to the fact that MP is programmed in C#,

and Optitrack library is created in C, it would be very difficult to integrate the Optitrack

library directly into MP. Therefore, a separate Optitrack Client is created in C to decode

the data being sent by the Optitrack cameras. Data from the Optitrack Client are sent

to the MP via internal network. Optitrack Listener is a thread created in MP for the

sole purpose of receiving and processing data being sent by the Optitrack Client on the

internal network.

A similar configuration has to be made for image processing. The OpenCV graphics


library is used for the ISP, but it only supports C or C++ environments. A Vision Client

is created to receive the image from the camera receiver and process the image before

sending valuable information to the MP via a separate internal network. Once again, a

Vision Listener thread is made in the MP to receive this information so that all these

external data can be integrated in the MP before creating the appropriate navigation

commands. For simplicity in data encoding and decoding, the Mission Planner does not

send any real data to the external client. As a result, data can only travel from the Vision

Client to the Mission Planner while the Mission Planner can only send acknowledgements

to the Vision Client. This results in the interaction process shown in Figure 4.6.

Figure 4.6: The interaction between ground station and vision client for edge extraction

Processes in the Mission Planner are inside the red dotted box, and processes that

are part of the Vision Client are inside the blue dotted box. The interaction starts as

soon as the Vision Client receives an image from the camera. A request to save the

current states of the quadrotor is sent by the Vision Client to the Mission Planner.

The states saved include the altitude, heading angle, pitch angle, yaw angle, and the

position. These information will be used to convert target way-point information from

image pixel coordinate to global optitrack coordinate for navigation purposes. When the

Mission Planner receives this request, it would save the current states data, and return

an acknowledgement to the Vision Client. The image processing sequence starts once

the acknowledgement has been received. The Vision Client would then send the point of


interest data, which in this case would be the target way-point in image pixel coordinates,

to the Mission Planner after image processing has been completed. The Mission Planner

is responsible to convert this target way-point from image pixel coordinates into global

coordinates. X and Y are usually longitude and latitude values in GPS coordinates for

outdoor missions and Z is usually the altitude above ground measured in meters. In

an indoor experiment, however, the quadrotor would be tracked by Optitrack instead of

GPS. Therefore, X, Y, and Z are the coordinates in the Optitrack reference frame with

units in mm. The simulation developed in MATLAB for this thesis project will use the

Optitrack reference frame. Finally, the global target way-point defined by X, Y, and Z

is sent to the autopilot as a navigation command, completing the PBVS feedback cycle.

As with all wireless data transmission, noise can be occasionally added to the data

during transmission. Figure 4.7 shows a comparison of a sample image captured by the

on board camera with and without noise.

(a) Normal image (b) Image under interference

Figure 4.7: The reverse elliptic approximation result and comparison

Figure 4.7a shows the image captured under normal circumstances, while Figure 4.7b

shows images with occasional interference. Interference could potentially come from two

sources. Due to the size of the micro-quadrotor, all the electronic components are very

close to each other. The four motors and the four ESCs could generate large amounts

of electro-magnetic interference for both the camera and wireless transmission module.

In addition, the wireless transmission module is relatively low power because of the

battery limit on board the micro-quadrotor; therefore, noise from the surroundings could

also degrade the image quality during wireless transmission. The chance of receiving

an image with as much interference as the one shown in Figure 4.7b is low; thus, it is

possible to completely discard images like these during image processing as they offer no

real value in terms of navigation information.

The pseudo code for the Vision Client and Mission Planner are shown below. The

Vision Client code can be changed for different PBVS missions. For example, a target

tracking PBVS mission would use an image processing algorithm that detects the target


object, and sets the location of the target object as the next way-point. The Mission

Planner component of the PBVS code does not need to be changed.

Vision Clientwhile Exit key not pressed do

Receive image from cameraSend request to MP to save quadrotor stateswhile MP acknowledgement not received do

Wait for MP responseend whileFind fire edge from image and determine next way-point (image coordinate)Send way-point (image coordinate) to MPwhile MP acknowledgement not received do

Wait for MP responseend while

end while

Mission Planner Vision Threadwhile Exit key not pressed do

while Request from Vision Client not received doWait for request

end whileSave current quadrotor statesSend acknowledgement to Vision Clientwhile Way-point from Vision Client not received do

Wait for way-pointend whileUse saved quadrotor states to transform way-point into Optitrack reference frameSend way-point (Optitrack coordinate) to autopilot

end while

The Optitrack Client transmits quadrotor’s attitude and position data to the MP

at 10Hz, and the Vision Client transmits the new target way-point to the MP at 1Hz.

Although the maximum update rate for Optitrack hardware is 25Hz and the maximum

refresh rate for image processing is 10Hz, these two components are throttled because

such a high update rate on quadrotor position control has been found to not be necessary.

The environment generation for the ISP is independent of everything else in the

system. The current design of ISP requires a pre-rendered environment, either static or

dynamic, to be projected on to the the glass floor via a separate computer. The wildfire

projection used in this research, for example, is a video captured from a MATLAB

simulation. The rest of the system, such as the ground station and the autopilot, are not

connected to the environment projection in any way.


4.3 Experimentation Phases

A test plan that consists of multiple phases has been created for both computational sim-

ulation and physical experiments. This test plan, illustrated in Figure 4.8, ensures that

the system is fully tested on its basic functions before moving on to the final objectives.

Each of the blocks in Figure 4.8 is an experiment.

The ones on the left side in red are the experiments that need to be done in the

simulation. The first of these tests is controller verification. This would validate both

the quadrotor model and the controllers, including controller gains, in the simulation. A

well tuned controller is essential to future quadrotor experiments, and individual control

channels will be checked. Once the controllers have been tested, the way-point navigation

experiment will be done in simulation. This experiment validates the quadrotor’s ability

to follow way-point commands, which is important for a PBVS navigation. Finally, the

dynamic fire edge tracking can be tested in simulation.

Figure 4.8: The test plan for simulation and physical experiment

The indoor experiments are listed on the right side, in blue boxes. Due to the fact

that ArduPilot is a commercial open source autopilot, the confidence in the controller im-

plementation is high and the controller gains are the only variables that need to be tuned.

Therefore, the indoor physical experiment starts with way-point navigation test, which

would validate the quadrotor’s navigation functions using manual way-point commands.


A target estimation test is needed afterwards to verify the image processing algorithm

and reference frame transformation code. Once the target can be estimated accurately,

the quadrotor would need to run a PBVS target tracking test. This ensures that the vi-

sion feedback loop is working properly and navigates based on image processing results,

instead of manual way-point commands. An intermediate edge tracking experiment needs

to be carried out to test the edge extraction code on a static fire edge. Finally, the full

dynamic fire edge tracking experiment would be completed once all sub-systems of the

experiment set up has been validated.

4.4 Summary

The ISP is an integration of multiple hardware and software systems, such as micro-

quadrotor, Optitrack cameras, ground station software, and image processing program.

The hardware of ISP, the micro-quadrotor in particular, gets updated over the course

of this thesis project in order to meet the specifications of the system. The software of

ISP is concentrated on the ground station computer, where three separate programs are

inter-connected in operation via an internal network. Together, they provide real-time

tracking, image processing, way-point identification, as well as wireless communication

capabilities. The indoor experiment has been broken down into several phases, each one

designed to verify the capabilities of the entire system while minimizing the chance of

catastrophic failure.

Chapter 5

Numerical Results

Numerical simulations of a quadrotor fire edge tracking mission has been carried out

in MATLAB. Quadrotor controllers, including attitude and position control, have to be

verified first to ensure that the quadrotor behavior in simulation is correct. Then, the

quadrotor is simulated with the dynamically expanding fire to see how well the edge

tracking strategy works. Finally, an attempt is made to reconstruct the fire perimeter

based on the data collected by the quadrotor while tracking the fire edge.

5.1 Quadrotor Navigation

In order to develop a simulation of the quadrotor edge tracking mission and verify the

fire perimeter estimation method proposed in this research, a simulated quadrotor with

functional navigation controllers is required. A non-linear quadrotor model is created

in MATLAB with the controller structure presented in Section 2.2.2. Various system

parameters used by this model are either measured, such as the weight of the quadrotor,

or approximated, such as the moments of inertia, based on the micro-quadrotor that is

being used on the ISP.

5.1.1 Controller Validation

Although the behaviours of the simulated quadrotor should be close to that of the actual

quadrotor, they are not an exact match. As a result, the various PID gains used for

the simulated controllers are different from those used in the ArduPilot. The controller

structure can be divided into two controller loop categories. The outer loop includes

the position controller, the yaw controller, and the altitude controller. Controllers in

this category are directly affected by the navigation commands. The inner loop includes

56

Chapter 5. Numerical Results 57

the roll and pitch controllers. These two controllers use the outputs of the outer loop

position controllers as reference signals. Inner loop PID controllers are tuned first and

usually require fast response to any changes in reference signals. After which, the outer

loop controllers can be tuned with the assumption that the inner loops are effectively a

part of the model.

On a quadrotor, roll and pitch behaviours should be the same in theory; therefore,

the roll and pitch controllers would use the same set of PID gains. Figure 5.1 shows the

simulated step response of roll angle tracking using the inner controllers.

(a) Angle and angular rate vs. time (b) Roll Moment vs. time

Figure 5.1: Step response for angle tracking

U2 from Eqn 3.14 is the roll moment. The pitch angle response would be exactly the

same. The quadrotor is capable of achieving the desired roll angle of 1 radian within 0.4

seconds. The force applied has a limiter of ±2N that prevents the model from generating

a force greater than the actual quadrotor could.

5.1.2 Way-Point Navigation

The three outer loop controllers of the quadrotor can be examined with a simulated way-

point based mission. The quadrotor starts at the origin on the ground and a series of

two way-points are defined for this mission, in the format of[x y altitude

]with unit

in meters . The first way-point is located at[0 5 1

], and the second way-point is at[

4 8 2]. The overall result of this simulation is shown in Figure 5.2

The simulation ran at a discrete time step of 0.01 second. The green dots indicate the

location of the home position of the quadrotor and the two way-points. The blue circles

represent the location of the quadrotor over time, and the vertical blue lines represent


Figure 5.2: The simulated way-point based quadrotor mission

the altitude of the quadrotor at every 100 time steps. The yaw angle tracking result is

shown in Figure 5.3.

Figure 5.3: Yaw angle tracking in simulated way-point navigation mission

The reference yaw signal is shown in red and the output of the quadrotor is shown

in blue. The yaw angle controller is capable of reaching the reference yaw angle within

1 second, and has no steady state error and minimum overshoot. Figure 5.4 shows the

altitude tracking and vertical velocity of the quadrotor in simulation.

A vertical velocity limit has been placed on the quadrotor model in order to produce


Figure 5.4: Altitude tracking in simulated way-point navigation mission

a more gradual climb that is similar to commercial autopilots, including the ArduPilot.

The climb rate limit prevents any noticeable overshoots in the system. Figure 5.5 shows

the position tracking results.

Figure 5.5: Position tracking in simulated way-point navigation mission

The X and Y position tracking of the quadrotor is limited by a maximum linear

velocity of 0.5m/s. Although most quadrotors in the real world would travel between

5m/s and 10m/s, these velocities are too fast for the ISP application. The velocity of

0.5m/s is more reasonable when flying indoors. A minor cross channel interference can

be observed at the start, when the quadrotor tries to turn from its initial heading of

0 deg to a reference heading of 90 deg in order to point at the next way-point. This


type of cross channel interference, between heading angle and position, is due to the

quadrotor attempting to move in a certain horizontal position, thus having a roll or pitch

angle, while changing its heading. The direction of horizontal movement changes in the

inertial frame when the quadrotor has a non-zero roll angle and a changing heading

angle. Although the PID controller will eventually fix this drift once the heading angle

stabilizes, it cannot provide an instant response to prevent such movement. Figure 5.6

shows the roll and pitch response of the quadrotor based on the requirements of the

position controllers.

Figure 5.6: Roll and pitch angle tracking in simulated way-point navigation mission

The reference roll and pitch angles are the outputs from the horizontal position con-

trollers. It can be observed that the responses are very quick and accurate. The steady

state errors in roll and pitch angles are also minimum. Finally the lateral error of the

quadrotor can be seen in Figure 5.7.

The lateral error plot shows how far off the quadrotor is from the intended track

linking between two way-points. Overall, the lateral error is constantly decreasing and

is negligible, even though it does not quite reach zero.

The way-point mission simulation proves that the PID gains for the controllers as

well as the controller structure can achieve good navigation results. The quadrotor in

simulation is capable of moving from way-point to way-point with very good tracking

performance and minimum error. All four control channels are stable and the simulated

quadrotor can be used in different way-point based missions.

Scaling up is possible because everything in the ISP, from fire simulation to the micro-

quadrotor, are scalable. In fact, wildfires in real world are most likely to be much larger


Figure 5.7: Lateral error in simulated way-point navigation mission

in size compared to the UAV. As a result, the local fire edge should appear more like a

straight line than a curve, making tracking easier for the UAV.

5.1.3 Quadrotor Fire Perimeter Tracking

A simulation of a quadrotor tracking the expanding fire edge is created to see how well the

fire perimeter could be approximated from the position data collected by the quadrotor

in theory. The environment used is a wildfire scenario with uniform but dynamic wind

conditions. Figure 5.8 shows the edge tracking result of a simulated quadrotor on a

dynamically expanding fire perimeter with ideal tracking.

(a) Isometric view (b) Bird’s eye view

Figure 5.8: Simulated fire edge tracking with quadrotor

Figure 5.8a is the isometric view of the simulation and presents the quadrotor’s move-


ment in the 3-dimensional space. Figure 5.8b presents the bird’s eye view of both the fire

perimeter over time and the quadrotor movement. Each of the red ellipses represent the

location of the fire perimeter at a time instance. The wind initially blows at a 45 degree

angle, but changes to 90 degrees in the middle of the simulation. This can be observed

from the difference in the spacings between the red ellipses in Figure 5.8b.

5.2 Fire Perimeter Reconstruction

The reverse elliptic approximation can be applied to the position data that forms the

quadrotor’s flight path to determine the fire perimeter location and shape at a single

time instance. Figure 5.9 shows the result of reverse elliptic approximation using the

technique presented in Section 2.3.1.

Figure 5.9: Finding local fire perimeter position using reverse elliptic approximation

The green ellipse represents the ground truth position of the fire perimeter at the time

instance when the quadrotor first encounters the fire. The goal is to estimate the shape

and position of this particular perimeter using the position data of the quadrotor and

the available wind information. The black line is the estimation result using the reverse

elliptic approximation technique. The estimation result is very similar to the actual fire

perimeter when the quadrotor is capable of ideal tracking. In fact, it can be said that

the estimated perimeter is a discretization of the fire perimeter.

The next step is to approximate the rest of the fire perimeter using the estimated local

fire perimeter. The aforementioned elliptic curve fitting technique is used to approximate

the entire fire perimeter. Figure 5.10 shows the result of this curve fitting technique

presented in Section 2.3.2.


(a) No noise (b) Gaussian noise

Figure 5.10: Full perimeter approximation using elliptic curve fitting technique

The quadrotor in Figure 5.10a has ideal tracking capabilities and experiences no noise

on its sensors. As a result, the ellipse curve fitting produces excellent results. In con-

trast, the elliptic perimeter approximated is very different from the actual perimeter if

the quadrotor tracking is not perfect, as can be seen in Figure 5.10b, where a gaussian

error with a mean of 0m and a standard deviation of 1m is added to the position data of

the quadrotor. A quadrotor’s tracking capability heavily relies on the sensor accuracy,

autopilot implementation, and the integration electromechanical components. As a re-

sult, ideal tracking does not exist in the real world and the results to be expected from

the ISP experiment should be closer to that of Figure 5.10b.

In the non-ideal tracking case, only the segment close to the reverse elliptic approxi-

mation result is accurate enough for any further processing. The rest of the ellipse does

not provide any useful information regarding the shape of the entire fire perimeter. In

the segment tracked by the quadrotor, 2nd order polynomial curve fitting provides bet-

ter results than ellipse fitting. Additionally, ellipse curve fitting can only provide good

estimation of the entire ellipse if the fire is assumed to be expanding elliptically. This is a

valid assumption in the constant and uniform parameter conditions, as well as dynamic

wind condition with limited change in wind parameters. However, this assumption would

not be valid in a real world scenario.

Chapter 6

Experimental Results

6.1 Quadrotor Navigation

Flight tests in the indoor environment can be dangerous for both the vehicle and po-

tentially the occupants; thus, the experimental phase of this research project is divided

into multiple phases, each assigned a small objective. The result of meeting each of these

objectives would mean that the ISP system is ready to move on to the next phase of the

experiment.

The progression of the experiment starts with verification of the quadrotor’s way-point

tracking capability within the Optitrack environment, which involves position feedback

control. Once this capability has been met, the next objective would be target tracking.

The quadrotor would be tasked with tracking a moving target using vision feedback.

Then the quadrotor needs to be able to switch from tracking a single object to a static

line. It would need to detect and follow the edge of the line from one side of the ISP

to the other. Finally, the quadrotor would be used to track a dynamically expanding

fire perimeter. The data collected from this test would be used for the fire perimeter

reconstruction experiment.

6.1.1 Way-Point Tracking

Position-based navigation is built upon the quadrotor’s ability to track a way-point. This

capability is evaluated in terms of the quadrotor’s horizontal position, heading angle, and

altitude. The quadrotor should be able to not only maintain its position at a single way-

point, but also move from one way-point to another.

In the first phase of the experiment, the quadrotor is commanded to hover at an

initial way-point, with additional way-point commands sent to the quadrotor manually.

64

Chapter 6. Experimental Results 65

Figure 6.1 shows the quadrotor’s way-point tracking capabilities in this experiment.

(a) X-axis position control (b) Y-axis position control

Figure 6.1: Way-point tracking experiment position results

Figure 6.1a shows the position of the quadrotor in the x-axis, along with the reference

position, which is the position of the way-point. Figure 6.1b shows the desired and actual

position data in the y-axis. The quadrotor takes off in manual mode, with the autopilot

switched on at the 10 second mark. The way-point moves in the x-axis at the 20 second

mark, and moves in the y-axis at approximately 45 second mark. Overall, the deviation

from the desired position is less than ±5cm. The quadrotor also tracks any changes

in the way-point very closely, provided that the change is gradual. Gradual changes in

way-points are needed to prevent excessive overshoot.

Figure 6.2a shows the heading angle of the quadrotor throughout this experiment.

The heading angle is fixed to 90 degrees in the Optitrack reference frame. The quadrotor

is capable of maintaining this heading angle command to an error of ±5 degrees.

Figure 6.2b shows the altitude of the quadrotor. The command altitude for this

experiment is 60cm. The altitude controller is engaged at the 10 second mark, and

any altitude changes prior to this is due to manual throttle control. The quadrotor can

maintain its altitude fairly well during hover, but has exhibits an initial drop in altitude

when it is commanded to move to another position.

Although the position of the way-points in this experiment are manually entered into

the system, the system is set up so that the ground station can generate its own way-

point commands based on the image processing results, which would become a part of

the PBVS control loop.


(a) Heading angle control (b) Altitude control

Figure 6.2: Way-point tracking experiment heading angle and altitude results

6.1.2 Target Tracking

A PBVS target tracking experiment is carried out to make sure that the quadrotor is

capable of navigating using automated way-point commands. The PBVS target tracking

experiment requires the quadrotor to follow a uniquely coloured object above the glass

floor of the ISP. The quadrotor’s altitude is manually specified to be 40cm, and its heading

angle is fixed to 180 degrees. The target object used in this experiment is a piece of blue

foam fitted with Optitrack tracker balls on top of a long rod, as illustrated in Figure 6.3.

Figure 6.3: The target used in the PBVS experiment

The long rod allows the target object to be moved around on top of the ISP glass

floor manually. The target is tracked by the Optitrack system via the tracker balls, so

that the quadrotor’s movement can be compared with the ground truth position data of

the target from Optitrack.

A relatively simple colour thresholding object tracking algorithm is used in this PBVS


experiment, because the target has a unique blue colour compared to the white back-

ground. A colour threshold filter for the blue colour is applied to the raw image, which

turns the pixels that are considered part of the target white, and pixels that are not the

target black. Then, an area based centre of mass calculation, provided by the OpenCV

library, is used to determine the location of target’s centre within the image. Figure 6.4

shows the result of applying the blue threshold filter on the raw image captured by the

on board camera on the left, and the position of the centre of the target object as the

quadrotor is hovering on the right.

(a) Threshold result (b) Target tracking

Figure 6.4: Determining the centre position of the target object via thresholding

The red line in Figure 6.4b indicates the movement of the centre of the object within

the image frame. In this particular case, however, this is a result of the movement of

the quadrotor in hovering mode with the target object being stationary. In the PBVS

target tracking test, the location of the centre of the target would be transformed from

the image coordinates to the global Optitrack coordinates. This location would become

the new target way-point for the quadrotor. The target way-point is being updated by

the ground station continuously throughout the target tracking mission. As a result, it is

important to make sure that this reference frame transformation process is tested before

the actual PBVS experiment is carried out. Figure 6.5 shows the result of the target

position estimation test.

The quadrotor is commanded to hover around the way-point[1600 1200

], which

represents 1600mm on the x-axis and 1200mm on the y-axis. While the quadrotor is

hovering, the target is moved around on the ISP glass floor, still within the field of view

of the on board camera. The estimated target position in blue follows the actual target

position in red very closely. The actual target position is taken from Optitrack and is

considered as the ground truth in this experiment. The maximum error in the X direction


Figure 6.5: Top down view of the target estimation test

is 60mm with a mean error of 7.8mm, whereas the maximum error in the Y direction

is 120mm with a mean error of only 6.5mm. The image processing and reference frame

transformation processes produce very accurate position estimates of the target object.

Finally, the PBVS experiment can be carried out knowing that the ISP could accurately

estimate the position of the target object. Figure 6.6 shows the result of this PBVS test.

The separate tracking responses for x and y-axis are shown in Figure 6.7.

The red line represents the position of the target over the duration of the PBVS

test, as measured by Optitrack. The blue line represents the estimated target position,

calculated by the Vision Client and ground station from the images taken by the on

board camera. The black line represents the flight path of quadrotor, whose goal is to

track the target by ensuring that the target remains at the centre of the image. The

target estimation is very accurate and the tracking by the quadrotor is good although

not exact. It can be observed that on both axes, the quadrotor’s position lags behind

the position of the target, which is to be expected in an object tracking test.

6.1.3 Line Tracking

The PBVS line tracking test is an extension of the target tracking test. In this test,

the target way-point is a point in front of the quadrotor that is also on the line that

has been detected. The line comes from a projection of a static image that includes a

straight fire edge. For scaling reasons, the projected fire edge has to be large relative to


Figure 6.6: Top down view of the PBVS tracking test

the quadrotor. Figure 6.8 shows the original image that is being projected.

The fire is located at the bottom of the image, and the fire edge would be expanding

upwards if this was a dynamic image instead of a static one. The fire edge retains the

yellow outer edge that represents newly burning areas, the red inner edge that represents

the areas that have been burning for a while, and the dark red areas at the bottom have

been burnt out. Figure 6.9 shows the top down view of the static line tracking result.

Due to the position of the projector relative to the ISP, the fire edge does not appear in

the middle of the glass floor as it does in the original image. The actual position of the

edge is measured by Optitrack markers based on the projection.

The green line represents the total width of the fire edge, which includes the yellow

and the red areas. The red line is the centre of the fire edge; the blue line is the position

of the fire edge estimated by the ground station, which is effectively the position of the

target way-points; and finally the black line is the path that the quadrotor took to track

the fire edge. The estimated fire edge position varies in the y-axis but overall remains

bounded to the upper and lower fire edge boundaries. The quadrotor’s roll angle affects

the accuracy of edge detection by altering the lighting of the raw image. The closeness

in colour space between yellow and different shades of red can affect the accuracy of the

estimation as well. As a result, it is acceptable for the estimated fire edge to be between

the upper and lower boundaries of the projected image. The x-axis tracking result for

the quadrotor is not as important as the y-axis, since the target way-point would always


Figure 6.7: X and Y-axis PBVS tracking result

Figure 6.8: Static straight edge being projected onto the glass floor

be a set distance in front of the quadrotor and the quadrotor should move forward in the

x-axis at a relatively constant velocity. Additionally, this plot depicts that the quadrotor

both overshoots and lags behind when trying to track the target way-points in the y-

axis. The lag is considered normal behaviour for tracking. Neither the overshoot nor the

lag are detrimental to the quadrotor’s line tracking capability and the test is considered

successful.

6.1.4 Dynamic Edge Tracking

The environment being projected on to the ISP glass floor is a simulation of dynamically

expanding fire edge created in MATLAB. This projection also uses the same scale as that


Figure 6.9: Top down view of static line tracking

of the static fire edge, which is a portion of the full perimeter simulation. Figure 6.10

shows the initial state of the entire fire perimeter, as well as the segment of the perimeter

that is actually projected.

The red ellipse is the fire perimeter on the simulation field. The blue rectangle

is the window used for capturing the video of the simulation. Everything within the

blue window is projected onto the ISP glass floor. Figure 6.11 shows the change of the

projected fire perimeter over time.

Figure 6.11a shows the initial state of the fire perimeter segment that is projected

onto the glass floor; 6.11b shows the perimeter segment after 40 seconds; and finally,

6.11c shows the final state of the fire perimeter segment after 80 seconds. The quadrotor

is capable of tracking this edge from right to left, flying counter-clockwise around the

fire, within 80 seconds. Figure 6.12 shows the quadrotor dynamic edge tracking result.

Similar to the static edge tracking result, the blue line is the estimated target position,

and the green line is the quadrotor flight path. The red lines, in this case, are the locations

of the fire perimeter at different time instances. The lowest red line is the initial fire

perimeter, which corresponds to Figure 6.11a; whereas the highest red line is the final

fire perimeter position, which corresponds to Figure 6.11c. The expansion of the fire

perimeter can be observed in the estimated target position, as the blue line moves from

lower right side to the upper left side. The quadrotor was able to keep the fire edge in


Figure 6.10: The initial state of fire perimeter and the segment projected

(a) (b) (c)

Figure 6.11: The progression of the dynamically growing fire in three time frames

view at all times and followed the expansion of the fire edge throughout this experiment.

6.2 Fire Perimeter Reconstruction

An attempt was made to use the quadrotor position data to approximate the location

of the fire perimeter, similar to the simulation; however, the quadrotor could not track

the actual fire perimeter as accurately as hoped. Although the quadrotor was able to

maintain the fire edge in view of the camera at all times, its own position was not as stable

as in the simulation. This means that the position of the quadrotor cannot accurately

represent the position of the fire edge. The estimated target positions, on the other

hand, are much more accurate and are used for fire perimeter reconstruction instead.

The images are sent from the on board camera to the Vision Client at a rate of 30 frames


Figure 6.12: Dynamic edge tracking result

per second; however, the Vision Client is set to only receive and process the images at 2

frames per second. Every 10th image that has been processed is saved for post analysis

purposes, which results in 1 image every 5 seconds. Over the course of 80 seconds of

flight, 16 images are saved for fire perimeter reconstruction. Figure 6.13 shows the raw

positions of the detected edges over the course of the dynamic edge tracking experiment

on the left, and the result of second order polynomial curve fitting after aligning the

curves to a single time step on the right.

(a) (b)

Figure 6.13: Detected edges and approximation of the segment


Figure 6.13a includes 16 edges. Each of these are from a different time instance, at

least 5 seconds apart, with the first edge on the right side and last on the left. The

locations of these edges show the progression of the fire perimeter, which moves from

bottom to top. These edges are synchronized into a single time instance using the reverse

elliptic approximation technique presented in Section 2.3.1. The resulting edges at the

initial time instance are coloured in red in Figure 6.13b. The blue line is the 2nd order

polynomial curve fitting result, which should be similar to the actual projected curve.

A comparison between the estimated perimeter, projected perimeter, and the actual

perimeter in simulation are shown in Figure 6.14.

(a) (b)

Figure 6.14: Comparison of estimated, projected, and actual fire perimeters

The red ground truth perimeter is the elliptic curve from the original MATLAB simu-

lation of a dynamically growing fire, after being converted into the Optitrack coordinate.

The blue projected perimeter is the curve that has been projected onto the ISP glass

floor. These two curves should be exactly the same in theory, and they do appear to

be very similar. The green detected curve is the same second order polynomial approx-

imation from Figure 6.13b. In this case, the detected curve is slightly higher than the

original. This can be explained by the fact that colour based detection for a projected

image is not always exact.

Figure 6.14b shows the ground truth perimeter in red, the second order approximation

curve in blue, and an elliptic estimation of the full fire perimeter in green. As with the

full fire perimeter result from the simulation experiment, the elliptic curve fitting can

only ensure reasonable approximation of the fire perimeter around the area where data

is collected. The perimeter is less likely to match that of the ground truth the further

away it is from the quadrotor’s tracking path. Elliptic curve fitting is very sensitive to

noise in the data. In certain cases, the parameters returned by the elliptic curve fitting


algorithm are imaginary numbers due to bad data and the total freedom in placing the

ellipse. It may be possible to minimize this by restricting the angle of the ellipse, since

the historic wind data could be obtained.

6.3 Outdoor Extension

The goal of the ISP is to provide a testing platform for various aerial trajectory tracking

and path planning algorithms. Although the wildfire perimeter tracking project is limited

to indoor experimentations, the Flight Systems and Control lab has an octorotor platform

for outdoor experimentations. The FSC octorotor consists of a Tarot T15 octorotor frame

with Pixhawk PX4 autopilot. The payload consists of a point and shoot camera with

satellite tagging and a 2 degrees of freedom gimbal. Figure 6.15 shows the octorotor

system.

Figure 6.15: FSC Octorotor flying a aerial imaging mission with full payload

There is a very close connection between this particular system and that of the ISP.

The PX4 autopilot is an open source autopilot developed by 3DRobotics, the same com-

pany that made the ArduPilot Mega. In fact, the PX4 and the ArduPilot uses the same

code. Additionally, they share the same ground station software and use the same com-

munication protocol library Mavlink. This means that modifications that have been done

on the ISP ground station software and the ISP quadrotor autopilot could also be done

on the outdoor octorotor platform. The same wildfire edge tracking algorithm, for ex-

ample, could be loaded onto the octorotor platform with relative ease, making the flight

tests completed on the ISP an integral part of any future outdoor octorotor flight tests

using the same algorithm. An edge detection analysis has been completed on an actual

aerial wildfire image using the same fire edge extraction algorithm used in the indoor

experiment. The result reveals that the algorithm developed for the indoor experiment


can be used for real world outdoor fire edge tracking experiment with minor adjustments.

Figure 6.16 shows the edge detection process being done on the sample aerial wildfire

image taken by CTV News in BC [29].

(a) Original[29] (b) Bilateral Filter (c) Thresholded

(d) Edge Detection (e) Selected Contour

Figure 6.16: Edge extraction of outdoor aerial wildfire image

The edge detection process used for the outdoor wildfire edge detection is very similar

to the process used for fire edge detection in the ISP experiment. First, a bilateral filter

is applied to the raw image to smooth out the noise. Then, a colour based threshold

filter is applied to pick out the colour that represents the object of interest. This would

be red in the case of fire detection. A Canny edge detector is used to pick out the edges

from the thresholded image. For way-point generation, the longest edge is picked as the

contour of interest and a new way-point can be found at the intersection between this

contour and the vertical centre line of the image. The image processing algorithm is able

to pick out the edge of the fire with relatively good accuracy and the way-point output

from this algorithm could be used as the feedback for navigation controller, similar to

the process integrated in the ISP system.

Chapter 7

Conclusions

The primary objective of this project was to develop a fire edge tracking algorithm

that could be tested in realistic scenarios. These scenarios consisted of a fully coloured

dynamically expanding fire perimeter, and a physical quadrotor that had to track this

moving fire edge. This navigation algorithm has been tested in both computational

simulation and on a physical indoor test bed, the ISP. The quadrotor was able to track

the expanding fire edge, and provided the desired position data of the fire. Although

approximation of the entire fire perimeter is unlikely in a real world scenario, the segment

of the fire that the quadrotor tracked could be accurately plotted. This allows for future

fire growth prediction, and provide valuable information for disaster relief personnel.

The Indoor Simulation Platform created for this thesis project provides indoor test-

ing capabilities of a wild fire tracking algorithm. This integrated test bed also allows for

experimentation of other various UAV path planning and trajectory tracking algorithms.

The development of this platform increases algorithm debugging and functionality im-

provement efficiency greatly by moving many of the validation tests from outdoor to the

much more easily accessible indoor environment.

In addition, this system is fully compatible with the FSC octorotor system that is

used for outdoor flight tests. The octorotor uses an autopilot that shares the same code

as the one used for the ISP; therefore, it is relatively easy to insert the tracking code into

the octorotor system. This outdoor experiment could further verify the usefulness of the

ISP as an intermediate testing platform for UAV navigation algorithms.

77

References

[1] N. R. Canada, “National Wildland Fire Situation Report,” http://cwfis.cfs.nrcan.

gc.ca/report, accessed: 2014-07-08.

[2] D. Schroeder, “Evaluation of Three Wildfire Smoke Detection Systems,” http:

//wildfire.fpinnovations.ca/61/EvaluationOfThreeWildfireSmokeDetectionSystem.

pdf, accessed: 2014-09-17.

[3] NASA, “Global Fire Monitoring - NASA and NOAA Missions for Monitoring Global

Fires,” http://earthobservatory.nasa.gov/Features/GlobalFire/fire 5.php, accessed:

2014-09-17.

[4] R. Rothermel et al., “A Mathematical Model for Predicting Fire Spread in Wildland

Fuels,” in USDA Forest Service Research Paper, Ogden, Utah, 1972.

[5] D. Anderson et al., “Modelling the Spread of Grass Fires,” in Journal of the Aus-

tralian Mathematical Society, 1982, pp. 451–466.

[6] G. Trunfio et al., “A New Algorithm for Simulating Wildfire Spread Through Cel-

lular Automata,” in ACM Transactions on Modelling and Computer Simulation,

2011.

[7] R. Leishman et al., “Quadrotors and accelerometers: State estimation with an im-

proved dynamic model,” Control Systems, IEEE, vol. 34, no. 1, pp. 28–41, Feb 2014.

[8] A. Salih et al., “Modelling and PID Controller Design for a Quadrotor Unmanned Air

Vehicle,” in International Conference on Automation Quality and Testing Robotics,

Cluj-Napoca, Romania, 2010, pp. 1–5.

[9] S. Waslander et al., “Multi-Agent Quadrotor Testbed Control Design: Integral Slid-

ing Mode vs. Reinforcement Learning,” in Intelligent Robots and Systems, Edmon-

ton, Canada, 2005, pp. 3712–3717.

78

References 79

[10] J. Canny, “A Computational Approach to Edge Detection,” IEEE Transaction on

Pattern Analysis and Machine Intelligence, vol. PAMI-8, no. 6, pp. 679–698, Nov

1986.

[11] D. Hong et al., “A Threshold Strategy for Edge Tracking,” in IEEE International

Conference on Fuzzy Systems, New Orleans, USA, 1996, pp. 1886–1891.

[12] T. Qiu and Y. Yan, “An Autoadaptive Edge-Detection Algorithm for Flame and

Fire Image Processing,” IEEE Transactions on Instrumentation and Measurement,

vol. 61, pp. 1486–1493, May 2012.

[13] H. Chen and Q. Gao, “Integrating Color and Gradient into Real-time Curve Track-

ing,” in Canadian Conference on Computer and Robot Vision, Windsor, Canada,

2008, pp. 294–300.

[14] X. Zheng and Q. Gao, “Generic Edge Tokens: Representation, Segmentation and

Grouping,” in International Conference on Vision Interface, Halifax, Canada, 2003,

pp. 426–460.

[15] D. Casbeer et al., “Forest Fire Monitoring with Multiple Small UAVs,” in American

Control Conference, Portland, OR, 2005, pp. 3530–3535.

[16] P. Sujit et al., “Cooperative Forest Fire Monitoring Using Multiple UAVs,” in IEEE

Conference on Decision and Control, New Orleans, LA, 2007, pp. 4875–4880.

[17] T. Kawano et al., “Generating Target Path For Tracing a Line Before Mission the

Traced Line of Dead Angle of Camera,” in SICE-ICASE International Joint Con-

ference, Bexco, Korea, 2006, pp. 5286–5289.

[18] R. Holt, R. Beard et al., “Vision-Based Road-Following Using Proportional Naviga-

tion,” in Journal of Intelligent Robotic Systems, vol. 57, 2009, pp. 193–216.

[19] J. Clark and R. Fierro, “Cooperative Hybrid Control of Robotic Sensors for Perime-

ter Detection and Tracking,” in American Control Conference, Portland, OR, 2005,

pp. 3500–3505.

[20] M. Kemp et al., “Multi-UUV Perimeter Surveillance,” in IEEE Autonomous Under-

water Vehicles, 2004, pp. 102–107.

[21] S. Rangavajhala and A. Shirkhodaie, “Visual-Servoing and Navigation of Mobile

Robots for Target Tracking Applications,” in Southeastern Symposium on System

Theory, 2001, pp. 257–262.

References 80

[22] B. Thuilot et al., “Position Based Visual Servoing: Keeping the Object in the Field

of Vision,” in International Conference on Robotics and Automation, Washington,

DC, 2002, pp. 1624–1629.

[23] M. Bachiller et al., “Well Structured Robot Positioning Control Strategy for Position

Based Visual Servoing,” in International Conference on Robotics and Automation,

Seoul, Korea, 2001, pp. 2541–2546.

[24] A. Begum et al., “A Simple Visual Servoing and Navigation Algorithm for an Om-

nidirectional Robot,” in International Conference on Human-Centric Computing,

Cebu, Philippines, 2010, pp. 1–5.

[25] W. Fu et al., “Visual Servoing Based Mobile Robot Navigation Able to Deal with

Complete Target Loss,” in International Conference on Methods and Models in Au-

tomation and Robotics, Miedzyzdroje, Poland, 2013, pp. 502–507.

[26] 3DRobotics, “Connecting RC Outputs to the Motors,” http://copter.ardupilot.com/

wiki/connecting-your-rc-input-and-motors/, accessed: 2014-07-23.

[27] C. Tomasi and R. Manduchi, “Bilateral Filtering for Gray and Color Images,” in

International Conference on Computer Vision, Bombey, India, 1998, pp. 839–846.

[28] M. Koga, “Summer Project Report: Quadrotor Parameter Estimation,” Aug, 2014.

[29] CTVNews, “Wildfires Prompt Evacuations in BC, Alberta,” http://www.ctvnews.

ca/canada/wildfires-prompt-evacuations-in-b-c-alberta-1.1916251, accessed: 2014-

07-23.

Indoor Simulation of Wildfire Detection and Monitoring ... · Indoor Simulation of Wild re...

Documents

Transcript of Indoor Simulation of Wildfire Detection and Monitoring ... · Indoor Simulation of Wild re...