VISION-BASED AUTONOMOUS APPROACH AND LANDING …

VISION-BASED AUTONOMOUS APPROACH AND LANDING FORAN AIRCRAFT USING A DIRECT VISUAL TRACKING METHOD

Tiago F. Goncalves, Jose R. AzinheiraIDMEC, IST/TULisbon, Av. Rovisco Pais N.1, 1049-001 Lisboa, Portugal

{t.goncalves, jraz}@dem.ist.utl.pt

Patrick RivesINRIA-Sophia Antipolis, 2004 Route des Lucioles, BP93, 06902 Sophia-Antipolis, France

[email protected]

Keywords: Aircraft autonomous approach and landing, Vision-based control, Linear optimal control, Dense visual track-ing.

Abstract: This paper presents a feasibility study of a vision-based autonomous approach and landing for an aircraft usinga direct visual tracking method. Auto-landing systems based on theInstrument Landing System(ILS) havealready proven their importance through decades but general aviation stills without cost-effective solutionsfor such conditions. However, vision-based systems have shown to have the adequate characteristics for thepositioning relatively to the landing runway. In the present paper, rather than points, lines or other featuressusceptible of extraction and matching errors, dense information is tracked in the sequence of captured imagesusing anEfficient Second-Order Minimization(ESM) method. Robust under arbitrary illumination changesand with real-time capability, the proposed visual trackersuits all conditions to use images from standardCCD/CMOS toInfrared (IR) and radar imagery sensors. An optimal control design isthen proposed using thehomography matrix as visual information in two distinct approaches: reconstructing the position and attitude(pose) of the aircraft from the visual signals and applying the visual signals directly into the control loop. Todemonstrate the proposed concept, simulation results under realistic atmospheric disturbances are presented.

1 INTRODUCTION

Approach and landing are known to be the most de-manding flight phase in fixed-wing flight operations.Due to the altitudes involved in flight and the con-sequent nonexisting depth perception, pilots must in-terpret position, attitude and distance to the runwayusing only two-dimensional cues like perspective, an-gular size and movement of the runway. At the sametime, all six degrees of freedom of the aircraft must becontrolled and coordinated in order to meet and trackthe correct glidepath till the touchdown.

In poor visibility conditions and degraded visualreferences, landing aids must be considered. TheIn-strument Landing System(ILS) is widely used in mostof the international airports around the world allow-ing pilots to establish on the approach and follow theILS, in autopilot or not, until the decision height isreached. At this point, the pilot must have visual con-tact with the runway to continue the approach andproceed to the flare manoeuvre or, if it is not the

case, to abort. This procedure has proven its relia-bility through decades but landing aids systems thatrequire onboard equipment are still not cost-effectivefor most of the general airports. However, in thelast years, theEnhanced Visual Systems(EVS) basedon Infrared (IR) allowed the capability to proceed tonon-precision approaches and obstacle detection forall weather conditions. The vision-based control sys-tem proposed in the present paper intends then to takeadvantage of these emergent vision sensors in order toallow precision approaches and autonomous landing.

The intention of using vision systems for au-tonomous landings or simply estimate the aircraft po-sition and attitude (pose) is not new. Flight tests ofa vision-based autonomous landing relying on fea-ture points on the runway were already referred by(Dickmanns and Schell, 1992) whilst (Chatterji et al.,1998) present a feasibility study on pose determina-tion for an aircraft night landing based on a modelof the Approach Lighting System(ALS). Many oth-ers have followed in using vision-based control on

94

fixed/rotary wings aircrafts, and even airships, in sev-eral goals since autonomous aerial refueling ((Kim-mett et al., 2002), (Mati et al., 2006)), stabiliza-tion with respect to a target ((Hamel and Mahony,2002), (Azinheira et al., 2002)), linear structuresfollowing ((Silveira et al., 2003), (Rives and Azin-heira, 2004), (Mahony and Hamel, 2005)) and, obvi-ously, automatic landing ((Sharp et al., 2002), (Rivesand Azinheira, 2002), (Proctor and Johnson, 2004),(Bourquardez and Chaumette, 2007a), (Bourquardezand Chaumette, 2007b)). In these problems, differ-ent types of visual features were considered includinggeometric model of the target, points, corners of therunway, binormalized Plucker coordinates, the threeparallel lines of the runway (at left, center and rightsides) and the two parallel lines of the runway alongwith the horizon line and the vanishing point. Due tothe standard geometry of the landing runway and thedecoupling capabilities, the last two approaches havebeen preferred in problems of autonomous landing.

In contrast with feature extraction methods, director dense methods are known by their accuracy be-cause all the information in the image is used with-out intermediate processes, reducing then the sourcesof errors. The usual disadvantage of such method isthe computational consuming of the sum-of-squared-differences minimization due to the computation ofthe Hessian matrix. TheEfficient Second-order Mini-mization(ESM) (Malis, 2004) (Behimane and Malis,2004) (Malis, 2007) method does not require thecomputation of the Hessian matrix maintaining how-ever the high convergence rate characteristic of thesecond-order minimizations as the Newton method.Robust under arbitrary illumination changes (Silveiraand Malis, 2007) and with real-time capability, theESM suits all the requirements to use images fromthe common CCD/CMOS to IR sensors.

In vision-based or visual servoing problems, a pla-nar scene plays an important role since it simplifiesthe computation of the projective transformation be-tween two images of the scene: the planar homogra-phy. The Euclidean homography, computed with theknowledge of the calibration matrix of the imagerysensor, is here considered as the visual signal to useinto the control loop in two distinct schemes. Theposition-based visual servoing(PBVS) uses the re-covered pose of the aircraft from the estimated projec-tive transformation whilst theimage-based visual ser-voing (IBVS) uses the visual signal directly into thecontrol loop by means of the interaction matrix. Thecontroller gains, from standard output error LQR de-sign with a PI structure, are common for both schemeswhose results will be then compared with a sensor-based scheme where precise measures are considered.

The present paper is organized as follows: In theSection 2, some useful notations in computer visionare presented, using as example the rigid-body motionequation, along with an introduction of the consid-ered frames and a description of the aircraft dynamicsand the pinhole camera models. In the same section,the two-view geometry is introduced as the basis forthe IBVS control law. The PBVS and IBVS controllaws are then presented in the Section 3 as well as theoptimal controller design. The results are shown inSection 4 while the final conclusions are presented inSection 5.

2 THEORETICAL BACKGROUND

2.1 Notations

The rigid-body motion of the frameb with respect toframea by aRb ∈ SO(3) andatb ∈ R3, the rotationmatrix and the translation vector respectively, can beexpressed as

aX = aRbbX + atb (1)

where,aX ∈ R3 denotes the coordinates of a 3D pointin the frameaor, in a similar manner, in homogeneouscoordinates as

aX = aTbbX =

[aRb

atb0 1

][bX1

](2)

where,aTb ∈ SE(3), 0 denotes a matrix of zeros withthe appropriate dimensions andaX ∈ R4 are the ho-mogeneous coordinates of the pointaX. In the sameway, the Coriolis theorem applied to 3D points canalso be expressed in homogenous coordinates, as a re-sult of the derivative of the rigid-body motion relationin Eq. (2),

aX = aTbbX = aTb

aT−1b

aX = (3)

=

[ω v0 0

]aX = aVab

aX , aVab ∈ R4×4

where, the angular velocity tensorω ∈ R3×3 is theskew-symmetric matrix of the angular velocity vec-tor ω such thatω×X = ωX and the vectoraVab =

[v,ω]⊤ ∈ R6 denotes the velocity screw and indicatesthe velocity of the frameb moving relative toa andviewed from the framea. Also important in thepresent paper is the definition of stacked matrix, de-noted by the superscript ”s ”, where each column isrearranged into a single column vector.

2.2 Aircraft Dynamic Model

Let F0 be the earth frame, also called NED for North-East-Down, whose origin coincides with the desirable

VISION-BASED AUTONOMOUS APPROACH AND LANDING FOR AN AIRCRAFT USING A DIRECT VISUALTRACKING METHOD

95

touchdown point in the runway. The latter, unless ex-plicitly indicated and without loss of generality, willbe considered aligned with North. The aircraft lin-ear velocityv = [u,v,w] ∈ R3, as well as its angularvelocity ω = [p,q, r] ∈ R3, is expressed in the air-craft body frameFc whose origin is at the center ofgravity whereu is defined towards the aircraft nose,v towards the right wing andw downwards. The at-titude, or orientation, of the aircraft with respect tothe earth frameF0 is stated in terms of Euler anglesΦ = [φ,θ,ψ] ⊂ R2, the roll, pitch and yaw angles re-spectively.

The aircraft motion in atmospheric flight is usu-ally deduced using Newton’s second law and con-sidering the motion of the aircraft in the earth frameF0, assumed as an inertial frame, under the influenceof forces and torques due to gravity, propulsion andaerodynamics. As mentioned above, both linear andangular velocities of the aircraft are expressed in thebody frameFb as well as for the considered forces andmoments. As a consequence, the Coriolis theoremmust be invoked and the kinematic equations appearnaturally relating the angular velocity rateω with thetime derivative of the Euler anglesΦ = R−1ω and theinstantaneous linear velocityv with the time deriva-

tive of the NED position[N, E,D

]⊤= S⊤v.

In order to simplify the controller design, it iscommon to linearize the non-linear model around angiven equilibrium flight condition, usually a func-tion of airspeedV0 and altitudeh0. This equilib-rium or trim flight is frequently chosen to be a steadywings-level flight, without presence of wind distur-bances, also justified here since non-straight landingapproaches are not considered in the present paper.The resultant linear model is then function of the per-turbation in the state vectorx and in the input vectoru as

[xvxh

]=

[Av 00 Ah

][xvxh

]+

[BvBh

][uvuh

]

(4)describing the dynamics of the two resultant decou-pled, lateral and longitudinal, motions. The longitu-dinal, or vertical, state vector isxv = [u,w,q,θ]⊤ ∈R4

and the respective input vectoruv = [δE,δT ]⊤ ∈ R2

(elevator and throttle) while, in the lateral case, thestate vector isxh = [v, p, r,φ]⊤ ∈ R4 and the respec-tive input vectoruh = [δA,δR]⊤ ∈ R2 (ailerons andrudder). Because the equilibrium flight condition isslowly varying for manoeuvres as the landing phase,the linearized model in Eq. (4) can then be consideredconstant along all the glidepath.

2.3 Pinhole Camera Model

The onboard camera frameFc, rigidly attached to theaircraft, has its origin at the center of projection ofthe camera, also called pinhole. The corresponding z-axis, perpendicular to the image plane, lies on the op-tical axis while the x- and y- axis are defined towardsright and down, respectively. Note that the cameraframeFc is not in agreement with those usually de-fined in flight mechanics.

Let us consider a 3D pointP whose coordinates inthe camera frameFc arecX = [X,Y,Z]⊤. This pointis perspectively projected onto the normalized imageplaneIm ∈ R2, distant one-meter from the center ofprojection, at the pointm = [x,y,1]⊤ ∈ R2 such that

m =1Z

cX. (5)

Note that, computing the projected pointm know-ing coordinatesX of the 3D point is a straightforwardproblems but the inverse is not true becauseZ is oneof the unknowns. As a consequence, the coordinatesof the pointX could only be computed up to a scalefactor, resulting on the so-called lost of depth percep-tion.

When a digital camera is considered, the samepoint P is projected onto the image planeI , whosedistance to the center of projection is defined by thefocal lengthf ∈ R+, at the pixelp = [px, py,1] ⊂ R3

asp = Km (6)

where,K ∈R3×3 is the camera intrinsical parameters,or calibration matrix, defined as follows

K =

fx f s px0

0 fy py0

0 0 1

(7)

The coordinatesp0 = [px0, py0,1]⊤ ∈ R3 define theprincipal point, corresponding to the intersection be-tween the image plane and the optical axis. The pa-rameters, zero for most of the cameras, is the skewfactor which characterizes the affine pixel distortionand, finally, fx and fy are the focal lengths in the bothdirections such that whenfx = fy the camera sensorpresents square pixels.

2.4 Two-views Geometry

Let us consider a 3D pointP whose coordinatescX inthe current camera frame are related with those0X inthe earth frame by the rigid-body motion in Eq. (1) ofFc with respect toF0 as

cX = cR00X + ct0. (8)

ICINCO 2009 - 6th International Conference on Informatics in Control, Automation and Robotics

96

Figure 1: Perspective projection induced by a plane.

Let us now consider a second camera frame denotedreference camera frameF∗ in which the coordinatesof the same pointP , in a similar manner as before,are

∗X = ∗R00X + ∗t0. (9)

By using Eq. (8) and Eq. (9), it is possible to relate thecoordinates of the same pointP between referenceF∗and currentFc camera frames as

cX = cR0∗R⊤

0∗X + ct0−

cR0∗R⊤

0∗t0 = (10)

= cR∗∗X + ct∗

However, considering thatP lies on a planeΠ, theplane equation applied to the coordinates of the samepoint in the reference camera frame gives us

∗n⊤∗X = d∗ ⇔1d∗

∗n⊤∗X = 1 (11)

where, ∗n⊤ = [n1,n2,n3]⊤ ∈ R3 is the unit normal

vector of the planeΠ with respect toF∗ andd∗ ∈ R+

the distance from the planeΠ to the optical center ofsame frame. Thus, substituting Eq. (11) into Eq. (10)results on

cX =

(cR∗ +

1d∗

ct∗∗n⊤

)∗X = cH∗

∗X (12)

where,cH∗ ∈ R3×3 is the so-called Euclidean homog-raphy matrix. Applying the perspective projectionfrom Eq. (5) along with the Eq. (6) into the planarhomography mapping defined in Eq. (12), the rela-tion between pixels coordinatesp and ∗p illustratedin Figure 1 is obtained as follows

cp ∝ K cH∗K−1∗p ∝ cG∗∗p (13)

where,G∈R3×3 is the projective homography matrixand ”∝ ” denotes proportionality.

3 VISION-BASED AUTONOMOUSAPPROACH AND LANDING

3.1 Visual Tracking

The visual tracking is achieved by directly estimat-ing the projective transformation between the image

taken from the airborne camera and a given referenceimage. The reference images are then the key to re-late the motion of the aircraftFb, through its airbornecameraFc, with respect to the earth frameF0. Forthe PBVS scheme, it is the known pose of the refer-ence camera with respect to the earth frame that willallows us to reconstruct the aircraft position with re-spect to the same frame. What concerns the IBVS,where the aim is to reach a certain configuration ex-pressed in terms of the considered feature, the pathplanning is then an implicity need of such scheme.For example, if lines are considered as features, thepath planning is defined as a function of the parame-ters which define those lines. In the present case, thepath planning shall be defined by images because it isthe dense information that is used in order to estimatethe projective homographycG∗.

3.2 Visual Servoing

3.2.1 Linear Controller

The standard LQR optimal control technique waschosen for the controller design, based on the lin-earized models of both longitudinal and lateral mo-tions in Eq. (4). Since not all the states are expectedto be driven to zero but to a given reference, the con-trol law is more conveniently expressed as an opti-mal output error feedback. The objective of the fol-lowing vision-based control approaches is then to ex-press the respective control laws as a function of thevisual information, which is directly or indirectly re-lated with the pose of the aircraft. As a consequence,the pose state vectorP = [n,e,d,φ,θ,ψ]⊤ ∈ R6, inagreement to the type of vision-based control ap-proach, is given differently from the velocity screwV = [u,v,w, p,q, r]⊤ ∈ R6, which could be providedfrom an existentInertial Navigation System(INS) orfrom some filtering method based on the estimatedpose. Thus, the following vision-based control lawsare more correctly expressed as

u = −kP(P−P∗)−kV(V −V∗) (14)

where,kP andkV are the controller gains relative tothe pose and velocity states, respectively.

3.2.2 Position-based Visual Servoing

In the position-based, or 3D, visual servoing (PBVS)the control law is expressed in the Cartesian spaceand, as a consequence, the visual information com-puted into the form of planar homography is used toreconstruct explicitly the pose (position and attitude).The airborne camera will be then considered as only


97

another sensor that provides a measure of the aircraftpose.

In the same way that, knowing the relative posebetween the two cameras,R and t, and the planarscene parameters,n andd, it is possible to computethe planar homography matrixH it is also possible torecover the pose from the decomposition of the esti-mated projective homographyG, with the additionalknowledge of the calibration matrixK . The decom-position ofH can be performed by singular value de-composition (Faugeras, 1993) or, more recently, by ananalytical method (Vargas and Malis, 2007). Thesemethods result into four different solutions but onlytwo are physically admissible. The knowledge of thenormal vectorn, which defines the planar sceneΠ,allows us then to choose the correct solution.

Therefore, from the decomposition of the esti-mated Euclidean homography

cH∗ = K−1cG∗K , (15)

bothcR∗ andct∗/d∗ are recovered being respectively,the rotation matrix and normalized translation vector.With the knowledge of the distanced∗, it is then pos-sible to compute the estimated rigid-body relation ofthe aircraft frameFb with respect to the inertial oneF0 as

0Tb = 0T∗

(bTc

cT∗

)−1=

[0Rb

0tb0 1

](16)

where,bTc corresponds to the pose of the airbornecamera frameFc with respect to the aircraft bodyframeFb and0T∗ to the pose of the reference cameraframeF∗ with respect to the earth frameF0. Finally,without further considerations, the estimated posePobtained from0Rb and 0tb could then be applied tothe control law in Eq. (17) as

u = −kP(P−P∗)−kV(V −V∗) (17)

3.2.3 Image-based Visual Servoing

In the image-based, or 2D, visual servoing (IBVS)the control law is expressed directly in the imagespace. Then, in contrast with the previous approach,the IBVS does not need the explicit aircraft pose rel-ative to the earth frame. Instead, the estimated planarhomographyH is used directly into the control lawas some kind of pose information such that reach-ing a certain reference configurationH∗ the aircraftpresents the intended pose. This is the reason why anIBVS scheme needs implicitly for path planning ex-pressed in terms of the considered features.

In IBVS schemes, an important definition is thatof interaction matrix which is the responsible to relate

the time derivative of the visual signal vectors∈ Rk

with the camera velocity screwcVc∗ ∈ R6 as

s= LscVc∗ (18)

where,Ls ∈ Rk×6 is the interaction matrix, or the fea-ture jacobian. Let us consider, for a moment, thatthe visual signal vectors is a matrix and equal to theEuclidean homography matrixcH∗, the visual featureconsidered in the present paper. Thus, the time deriva-tive of s, admitting the vector∗n/d∗ as slowly vary-ing, is

s= cH∗ = cR∗ +1d∗

ct∗∗n⊤ (19)

Now, it is known that bothcR∗ andct∗ are related withthe velocity screwcVc∗, which could be determinedusing Eq. (3), as follows

cVc∗ = cT∗cT−1

∗ = (20)

=

[cR∗

cR⊤∗

ct∗− cR∗cR⊤

∗ct∗

0 1

]

from where,cR∗ = cω cR∗ andct∗ = cv+ cω ct∗. Byusing such results back in Eq. (19) results on

cH∗ = cω(

cR∗ +1d∗

ct∗∗n⊤

)+

1d∗

cv∗n⊤ =

= cω cH∗ +1d∗

cv∗n⊤ (21)

Hereafter, in order to obtain the visual signal vector,the stacked version of the homography matrixcHs

∗must be considered and, as a result, the interactionmatrix is given by

s= cHs∗ =

I(3)∗n1/d∗ −cH∗1

I(3)∗n2/d∗ −cH∗2

I(3)∗n3/d∗ −cH∗3

cv

cω

(22)

where, I(3) is the 3× 3 identity matrix andH i isthe ith column of the matrix as well asni is theith element of the vector. Note that,ωH is the ex-ternal product ofω with all the columns ofH andω×H1 = −H1×ω = −H1ω.

However, the velocity screw in Eq. (18), as wellas in Eq. (22), denotes the velocity of the referenceframeF∗ with respect to the airborne camera frameFc and viewed fromFc which is not in agreementwith the aircraft velocity screw that must be appliedinto the control law in Eq. (17). Instead, the veloc-ity screw shall be expressed with respect to the refer-ence camera frameF∗ and viewed from aircraft bodyframeFb, where the control law is effectively applied.In this manner, and knowing that the velocity tensorcVc∗ is a skew-symmetric matrix, then

cVc∗ = −cV⊤c∗ = −cV∗c (23)


98

Now, assuming the airborne camera frameFc rigidlyattached to the aircraft body frameFb, to change thevelocity screw from the aircraft body to the airbornecamera frame, the adjoint map must be applied as

cV = bT−1c

bVbTc = (24)

=

[bR⊤

cbωbRc

bR⊤c

bv+ bR⊤c

bωbtc0 0

]

from where,bR⊤c

bωbRc = bR⊤c

bω and bR⊤c

bωbtc =

−bR⊤c

btccω and, as a result, the following velocity

transformationcWb ∈ R6×6 is obtained

cV = cWbbV =

[bR⊤

c −bR⊤c

btc

0 bR⊤c

][bvbω

](25)

Using the Eq. (25) into the Eq. (22) along with theresult from Eq. (23) results as follows

s= cHs∗ = −Ls

cWbbV∗c (26)

Finally, let us consider the linearized version of theprevious result as

s−s∗ = cHs∗−H∗s = −Ls

cWbbW0 (P−P∗) (27)

where,bW0 =

[S0 00 R0

](28)

are the kinematic and navigation equations, respec-tively, linearized for the same trim point as for the air-craft linear model[φ,θ,ψ]⊤0 = [0,θ0,0]⊤. It is thenpossible to relate the pose errorP−P∗ of the air-craft with the Euclidean homography errorcHs

∗−H∗s.For the present purpose, the reference configuration isH∗ = I(3) which corresponds to match exactly bothcurrent I and referenceI ∗ images. The proposedhomography-based IBVS visual control law is thenexpressed as

u = −kP (LscW0)

†(

cHs∗−H∗s

)−kV(V −V∗)

(29)where, A† =

(A⊤A

)−1A⊤ is the Moore-Penrose

pseudo-inverse matrix.

4 RESULTS

The vision-based control schemes proposed abovehave been developed and tested in an simulationframework where the non-linear aircraft model is im-plemented in Matlab/Simulink along with the controlaspects, the image processing algorithms in C/C++and the simulated image is generated by the Flight-Gear flight simulator. The aircraft model considered

Figure 2: Screenshot from the dense visual tracking soft-ware. The delimited zone (left) corresponds to the bottom-right image, warped to match with the top-right image. Thewarp transformation corresponds to the estimated homogra-phy matrix.

corresponds to a generic category B business jet air-craft with 50m/sof stall speed, 265m/sof maximumspeed and 20mwing span. This simulation frameworkhas also the capability to generate atmospheric condi-tion effects like fog and rain as well as vision sensorseffects like pixels spread function, noise, colorimetry,distortion and vibration of different types and levels.

The chosen airport scenario was the Marseille-Marignane Airport with an nominal initial positiondefined by an altitude of 450m and a longitudinaldistance to the runway of 9500m, resulting into a3 degreesdescent for an airspeed of 60m/s. In orderto have illustrative results and to verify the robustnessof the proposed control schemes, two sets of simula-tions results are presented. The first with an initiallateral error of 50m, an altitude error of 30m and asteady wind composed by 10m/sheadwind and 1m/sof crosswind. The latter, with a different initial lateralerror of 75m, considers in addition the presence of tur-bulence. What concerns the visual tracking aspects, adatabase of 200m equidistant images along the run-way axis till the 100m height, and 50m after that,was considered and the following atmospheric con-ditions imposed: fog and rain densities of 0.4 and 0.8([0,1]). The airborne camera is considered rigidly at-tached to the aircraft and presents the following posebPc = [4m,0m,0.1m,0,−8 degrees,0]⊤. The simula-tion framework operates with a 50ms, or 20Hz, sam-pling rate.

For all the following figures, the results of the twosimulations are presented simultaneously and iden-tified in agreement with the legend in Figure 3(a).When available, the corresponding references are pre-sented in black dashed lines. Unless specified, thedetailed discussion of results is referred to the casewithout turbulence.

Let us start with the longitudinal trajectory illustratedin Figure 3(a) where it is possible to verify immedi-ately that the PBVS results are almost coincident withthe ones where the sensor measurements were consid-ered ideal (Sensors). Indeed, because the same con-


99

trol law is used for these two approaches, the resultsdiffer only due to the pose estimation errors from thevisual tracking software. For the IBVS approach, thefirst observation goes to the convergence of the air-craft trajectory with respect to the reference descentthat occurs later than for the other approaches. Thisfact is a consequence not only of the limited validityof the interaction matrix in Eq (27), computed for astabilized descent flight, but also of the importance ofthe camera orientation over the position, for high al-titudes, when the objective is to match two images.In more detail, the altitude error correction in Fig-ure 3(b) shows then the IBVS with the slowest re-sponse and, in addition, a static error not greater than2m as a cause of the wind disturbance. In fact, thepath planning does not contemplates the presence ofthe wind, from which the aircraft attitude is depen-dent, leading to the presence of static errors. Thesesame aspects are verified in the presence of turbulencebut now with a global altitude error not greater than8m, after stabilization. The increasing altitude errorat the distance of 650m before the touchdown corre-sponds to the natural loss of altitude when proceedingto the pitch-up, or flare, manoeuvre (see Figure 3(c))in order to reduce the vertical speed and correctly landthe aircraft. What concerns the touchdown distances,both Sensors and PBVS results are very close and ata distance around 330mafter the threshold line while,for the IBVS, this distance is of approximately 100m.In the presence of turbulence, these distances becameshorter mostly due to the oscillations in the altitudecontrol during the flare manoeuvre. Again, Sensorsand PBVS touchdown points are very close and about180m from the runway threshold line while, for theIBVS, this distance is about 70m.

The lateral trajectory illustrated in Figure 3(e) shows asmooth lateral error correction for all the three controlschemes, where both visual control laws maintain anerror below the 2mafter convergence. Once more, theoscillations around the reference are a consequence ofpose estimation errors from visual tracking software,which become more important near the Earth surfacedue to the high displacement of the pixels in the im-age and the violation of the planar assumption of theregion around the runway. The consequence of theseeffects are perceptible in the final part not only in thelateral error correction but also in the yaw angle of theaircraft in Figure 3(f). For the latter, the static error isalso an influence of the wind disturbance which im-poses an error of 1degreewith respect to the runwayorientation of exactly 134.8 degreesNorth.

In the presence of turbulence, Sensors and PBVS con-trol schemes present a different behavior during the

−10000 −8000 −6000 −4000 −2000 0 2000−100

0

100

200

300

400

500

Distance to Touchdown − [m]

Alti

tude

− [m

]

ReferenceSensorsPBVSIBVSSensors w/ turb.PBVS w/ turb.IBVS w/ turb.

(a) Longitudinal trajectory

−10000 −8000 −6000 −4000 −2000 0 2000−35

−30

−25

−20

−15

−10

−5

0

5

10

15


Alti

tude

Err

or −

[m]

(b) Altitude error

−10000 −8000 −6000 −4000 −2000 0 2000−2

0

2

4

6

8

10

12


Pitc

h an

gle

− [d

eg]

(c) Pitch angle

−10000 −8000 −6000 −4000 −2000 0 200040

45

50

55

60

65

70


Tru

e ai

rspe

ed −

[m/s

]

(d) Airspeed

−10000 −8000 −6000 −4000 −2000 0 2000−60

−40

−20

0

20

40

60

80


Late

ral E

rror

− [m

]

(e) Lateral trajectory

−10000 −8000 −6000 −4000 −2000 0 2000120

122

124

126

128

130

132

134

136

138

140


Yaw

ang

le −

[deg

]

(f) Yaw angle

Figure 3: Results from the vision-based control schemes(PBVS and IBVS) in comparison with the ideal situation ofprecise measurements (Sensors).

lateral error correction manoeuvre. Indeed, due to theimportant bank angle and the high pitch induced bythe simultaneous altitude error correction manoeuvre,the visual tracking algorithm lost information on thenear-field of the camera essential for the precision ofthe estimated translation. The resultant lateral errorestimative, greater than it really is, forces the lateralcontroller to react earlier in order to minimize sucherror. As for the longitudinal case, the IBVS presentsa slower response on position error corrections result-ing into a lateral error not greater than 8mwhich con-trasts with the 4m from the other two approaches.

It should be noted the precision of the dense visualtracking software. Indeed, the attitude estimation er-rors are often below 1degreefor transient responsesand below 0.1 degreesin steady state. Dependingon the quantity of information available in the nearfield of the camera, the translation error could varybetween the 1m and 4m for both lateral and altitudeerrors and between 10m and 70m for the longitudinaldistance. The latter is usually less precise due to itsalignment with the optical axis of the camera.


100

5 CONCLUSIONS

In the present paper, two vison-based control schemesfor an autonomous approach and landing of an aircraftusing a direct visual tracking method are proposed.For the PBVS solution, where the vision system isnothing more than a sensor providing position and at-titude measures, the results are naturally very similarwith the ideal case. The IBVS approach based on apath planning defined by a sequence of images shownclearly to be able to correct an initial pose error andland the aircraft under windy conditions. Despite theinherent sensitivity of the vision tracking algorithm tothe non-planarity of the scene and the high pixels dis-placement in the image for low altitudes, a shorter dis-tance between the images of reference was enough todeal with potential problems. The inexistence of a fil-tering method, as the Kalman filter, is the proof of therobustness of the proposed control schemes and thereliability of the dense visual tracking. This clearlyjustify further studies to complete the validation andthe eventual implementation of this system on a realaircraft.

ACKNOWLEDGEMENTS

This work is funded by the FP6 3rd Call EuropeanCommission Research Program under grant ProjectN.30839 - PEGASE.

REFERENCES

Azinheira, J., Rives, P., Carvalho, J., Silveira, G., de Paiva,E.C., and Bueno, S. (2002). Visual servo control forthe hovering of an outdoor robotic airship. InIEEE In-ternational Conference on Robotics and Automation,volume 3, pages 2787–2792.

Behimane, S. and Malis, E. (2004). Real-time image-basedtracking of planes using efficient second-order mini-mization. InIEEE International Conference on Intel-ligent Robot and Systems, volume 1, pages 943–948.

Bourquardez, O. and Chaumette, F. (2007a). Visual servo-ing of an airplane for alignment with respect to a run-way. In IEEE International Conference on Roboticsand Automation, pages 1330–1355.

Bourquardez, O. and Chaumette, F. (2007b). Visual servo-ing of an airplane for auto-landing. InIEEE Interna-tional Conference on Intelligent Robots and Systems,pages 1314–1319.

Chatterji, G., Menon, P., K., and Sridhar, B. (1998). Vision-based position and attitude determination for aircraftnight landing.AIAA Journal of Guidance, Control andDynamics, 21(1).

Dickmanns, E. and Schell, F. (1992). Autonomous land-ing of airplanes by dynamic machine vision.IEEEWorkshop on Application on Computer Vision, pages172–179.

Faugeras, O. (1993).Three-dimensional computer vision: ageometric view point. MIT Press.

Hamel, T. and Mahony, R. (2002). Visual servoing ofan under-actuated dynamics rigid-body system: animage-based approach. InIEEE Transactions onRobotics and Automation, volume 18, pages 187–198.

Kimmett, J., Valasek, J., and Junkins, J. L. (2002). Vi-sion based controller for autonomous aerial refueling.In Conference on Control Applications, pages 1138–1143.

Mahony, R. and Hamel, T. (2005). Image-based visualservo control of aerial robotic systems using linear im-ages features. InIEEE Transaction on Robotics, vol-ume 21, pages 227–239.

Malis, E. (2004). Improving vision-based control using ef-ficient second-order minimization technique. InIEEEInternational Conference on Robotics and Automa-tion, pages 1843–1848.

Malis, E. (2007). An eficient unified approach to directvisual tracking of rigid and deformable surfaces. InIEEE International Conference on Robotics and Au-tomation, pages 2729–2734.

Mati, R., Pollini, L., Lunghi, A., and Innocenti, M. (2006).Vision-based autonomous probe and drogue aerial re-fueling. In Conference on Control and Automation,pages 1–6.

Proctor, A. and Johnson, E. (2004). Vision-only aircraftflight control methods and test results. InAIAA Guid-ance, Navigation, and Control Conference and Ex-hibit.

Rives, P. and Azinheira, J. (2002). Visual auto-landing ofan autonomous aircraft. Reearch Report 4606, INRIASophia-Antilopis.

Rives, P. and Azinheira, J. (2004). Linear structure follow-ing by an airship using vanishing point and horizonline in visual servoing schemes. InIEEE InternationalConference on Robotics and Automation, volume 1,pages 255–260.

Sharp, C., Shakernia, O., and Sastry, S. (2002). A visionsystem for landing an unmanned aerial vehicle. InIEEE International Conference on Robotics and Au-tomation, volume 2, pages 1720– 1727.

Silveira, G., Azinheira, J., Rives, P., and Bueno, S. (2003).Line following visual servoing for aerial robots com-bined with complementary sensors. InIEEE Interna-tional Conference on Robotics and Automation, pages1160–1165.

Silveira, G. and Malis, E. (2007). Real-time visual trackingunder arbitrary illumination changes. InIEEE Con-ference on Computer Vision and Pattern Recognition,pages 1–6.

Vargas, M. and Malis, E. (2007). Deeper understanding ofthe homography decomposition for vision-base con-trol. Research Report 6303, INRIA Sophia-Antipolis.


101

VISION-BASED AUTONOMOUS APPROACH AND LANDING …

Documents

Transcript of VISION-BASED AUTONOMOUS APPROACH AND LANDING …