Visual Servoing of an Autonomous Helicopter in Urban Areas Using Feature Tracking presented by Wen Li

Luis Mejias, Srikanth Saripalli, Pascual Campoy and Gaurav Sukhatme

Outline

Introduction Related work Testbed Visual preprocessing Control Architectures Experiments Conclusion

Outline

Introduction Related work Testbed Visual preprocessing Control Architectures Experiments Conclusion

Introduction

Goal: vision-guided autonomous flying robots

Application: Law enforcement, search and rescue,

inspection and surveillance Technique:

Object detection, tracking, inertial navigation, GPS and nonlinear system modeling

Introduction

In this paper: Two UAVs – Avatar and COLIBRI Visual tracking => control

commands

Outline

IntroductionRelated work

Testbed Visual preprocessing Control Architectures Experiments Conclusion

Related Work

Hummingbird (A. Conway, 1995) Model-scale Use GPS only 4 GPS antennas Precisions: position 1cm

attitude 1 degree

Related Work

AVATAR (Jun, 1999) Onboard INS & GPS Kalman Filter for State Estimation Simulation

Related Work

Vision-guided Helicopter (Amidi, 1996, 1997) Onboard DSP-based vision processor Combine GPS and IMU data

Related Work

Vision-augmented navigation system (Bosse, 1997) Uses vision in-the-loop to control a helicopter

Visual odometer (Amidi, 1998) A notable vision-based technique used in

autonomous helicopter (Wu, et al, 2005)

Vision is used as additional sensor and fused with inertial and heading measurements for control

Outline

Introduction Related work

Testbed Visual preprocessing Control Architectures Experiments Conclusion

Autonomous Helicopter Testbed AVATAR

Gas-powered radio-controlled model helicopter RT-2 DGPS system provides positional accuracy of

2 cm ISIS-IMU provides rate information to onboard

computer, which is fused using a 16 state Kalman filter

Ground station: a laptop to send high-level control commands and differential GPS corrections

Autonomous flight is achieved using a behavior-based control architecture

Autonomous Helicopter Testbed COLIBRI

Gas powered model helicopter Fitted with a Xscale based flight computer

augmented with GPS, IMU, Magnetometer, fused with a Kalman filter

VIA mini-ITX 1.25 GHz computer onboard with 512 Mb RAM, wireless interface and a firewire color camera

Ground station: a laptop to send high-level control commands, and for visualization

Outline

Introduction Related work Testbed

Visual preprocessing Control Architectures Experiments Conclusion

Visual Preprocessing -- AVATAR Image segmentation and

thresholding Convert the image to grayscale Use the value of “target color” as

threshold Segment the image to binary image

where the object of interest is represented by 1’s and background with 0’s

Visual Preprocessing -- AVATAR Square Finding

Find contours (represented by polylines) from the binary image

Use an algorithm to reduce the points in polylines

Result: simplified squares

Visual Preprocessing -- AVATAR Template Matching

User selects a detected window (a target)from the GUI

A patch is selected around the location of the target

Use local search window to find best match between the target and the detected contours, deciding which window to track

Visual Preprocessing -- AVATAR Kalman Filter

Once a suitable match is found, a Kalman filter is used to track the feature positions

Input: x and y coordinates of the features

Output: estimates of these coordinates in the next frame

Visual Preprocessing -- COLIBRI The user selects the object of

interest from the GUI The location of the object is used to

generate visual reference

Visual Preprocessing -- COLIBRI Lateral visual reference

Visual Preprocessing -- COLIBRI Vertical Visual Reference

Outline

Introduction Related work Testbed Visual preprocessing

Control Architectures Experiments Conclusion

Control Architectures -- AVATAR A hierarchical

behavior based control architecture

Output of Kalman filter is compared with desired values to give an error signal to controller

Control Architectures -- COLIBRI Controller is

based on a decoupled PID control

Outline

Introduction Related work Testbed Visual preprocessing Control Architectures

Experiments Conclusion

Experimental results

At Del Valle Urban Search and Rescue Training site in Santa Clarita, California

AVATAR, four trials First, the helicopter is commanded to

fly autonomously to a given GPS waypoint

As soon as it detects the featured window, the controller switches from GPS-based to vision-based control

Location of the features in the image

Helicopter position in meters. (left figure) vertical axis– easting

(right figure) vertical axis – northing

Experimental Results

At ETSII Campus in Madrid, Spain COLIBRI Seven experimental trials on two

different days

Velocity references (vyr) with the helicopter velocity (vy)

Lateral displacement (east)

Velocity references (vzr) with the helicopter velocity (vz)

altitude displacement (down)

Helicopter displacements during the entire flight trial

Video demonstration

colibrivideoWeb.wmv

Outline

Introduction Related work Testbed Visual preprocessing Control Architectures Experiments

Conclusion

Conclusion -- Authors

Demonstrated an approach to visually control an autonomous helicopter: use visual algorithm to command UAV when GPS has dropouts

Experimentally demonstrated by performing vision-based window tracking tasks on two different platforms at different locations and different conditions

Conclusion -- Personal

The topic is interesting Visual algorithm is demonstrated

effective in the experiments

But… the writing is so ugly. Poor explanation▪ features, template and matching

Incomplete explanation of figures