High Performance Control of an Intelligent Wheelchair Using

Visual Information

Shilpa Gulati3-8-2007

Motivation

Safe navigation without constant control from driver

Reduce stress.Increase mobility.

Gracefully perform tasks such asPass through a door. Go up and down ramps.Turn sharp corners.

Problem Definition

Pose isLocation + Orientation.

Start from a pose such that:Location within circle of radius R from door.Both door edges clearly visible.

Move gracefully and pass-through-door.Stop on other side of door

Approximately at distance R from door.Facing away from door.

Pass-Through-Door

t

v

One possible velocity profile

Graceful Motion

Velocity and acceleration are within comfortable limits. Change smoothly.No backing-up maneuvers.Passenger always facing forward in direction of motion. Sufficient clearance from door edges and walls for passenger comfort.No collisions.

One Approach: Motion Planning then Feedback Control

Most common approach Plan desired path with motion planner: open-loop control.Make robot follow desired path with feedback controller: closed loop control.

Motion Planner1) Optimal Control2) Vector Fields3) Many more

Feedback ControllerKinematics

and Dynamics

of Wheelchair

DesiredPath/Trajectoryqd

Control Commandsu, v

Error = Actual - Desired

Let’s Jump into Feedback Control Now: Kinematic Model of Wheelchair

⎛⎝ xy

θ

⎞⎠ =⎛⎝ v cos θv sin θω

⎞⎠

v is linear velocity along ~CPω is angular velocity about C

Nonlinear SystemUnicycle Model

Nonholonomic Constraints

Constraint among velocities that cannot be integrated.Physically means that wheelchair cannot move sideways.Reduces feasible paths: makes motion planning difficult.2 inputs , 3 outputs

£sin θ − cos θ 0

¤⎛⎝ xy

θ

⎞⎠ = 0

(x, y, θ)(v,ω)

Controllability and Stabilizability

Does there exist piecewise smooth input that takes the system from a point to another in its neighbourhood?

Yes

Can the system be exponentially stabilized?Cannot be stabilized to a pose with smooth time invarying feedback.Can be stabilized to a trajectory. (Good for us!)

Types of Control Problems: Path Following

Follow a cartesian path.Linear velocity v assumed constant.Only one parameter to be controlled: e.Use ω to control e.

Easy!!

Path Following Control Law: Follow a Wall

Follow wall so that y = yd = constant.e = y – yd.

We want the system to behave like a “damped spring”.

e+ k1e+ k2e = 0

v cos θ θ + k1v sin θ + k2e = 0

ω = −k1 tan θ − k2v cos θ e

Damped Spring: System converges to y = yd

Types of Control Problems: Trajectory Tracking

Follow a time varying trajectoryqd(t) = [xd(t), yd(t), θd(t)]T

Two parameters to be controlled explicitly

ex(t), ey(t)eθ(t) controlled implicitly due to nonholonomic constraints

Two control inputs.Possible!!

Two Major Approaches for Trajectory Tracking

Feedback LinearizationNovel et. al. [1], Oriolo et. al. [8], De Luca et. al. [6 ] and many more.

Sliding Mode ControlYang and Kim [ 9]

Is path following sufficient? Why should we look at trajectory tracking?

We want the wheelchair to start from rest, move and then come to rest again.

Velocity is time varying.Just following a path may not be enough.Or maybe it will be…Let’s look at trajectory tracking.

Trajectory Tracking: Feedback Linearization - I

Exact linearization based on state transformations.Different from approximate linearization around an operating point.We want to control

z = h(q) = [x, y]T

We want find a relationship between input u = [u, ω]T and output z = [x, y]T

Differentiate output to getz = E(θ)u

Trajectory Tracking: Feedback Linearization - II

We got

If E(θ) is invertible, we can find new inputs ν = [ν1, ν2 ]T such that

u = E(θ)−1ν

So we get one input per output

z = E(θ)u

µz1z2

¶=

µν1ν2

¶ Now write control laws for these two decoupled linear systems

Trajectory Tracking: Sliding Control-I

Convert problem of tracking qd(t) to that of staying on a surface s(t) for all t > 0.

s = 0 is solution of differential equation with q(t) = qd(t).Design surface so all trajectories point towards it.Once on surface, trajectory stays on surface.

Sliding Surface

Control LawChange u based on which side of the surface system is on.

Which is better? Feedback Linearization or Sliding Control? - I

Sliding Control suffers from “chatter”System is constantly pushed from one side to the other.Can overcome this by specifying a boundary-layer around surface to get smooth control laws.Many other methods to remove chatter.

Sliding Surface

Boundary Layer

Which is better? Feedback Linearization or Sliding Control? - II

Sliding Control does not need exact knowledge of system parameters.

Qualitative description in the form of bounded functions is sufficient.Let control estimate based on qualitative model be û. Design control u to be

u = u− sign(s)sign(s) = +1 s > 0sign(s) = −1 s < 0

Feedback linearization lets us use a familiar, linear control method.

Can we do without motion planning for specialized tasks? - I

Possibly.Alter system dynamics using

External forces.Control inputs.

So that system naturally follows the trajectories we want.

Can we do without motion planning for specialized tasks?

Divide into regions separated by sliding surfaces

Different control law in each region.Blend the control laws for smoothness.

Determine sliding surfaces by gradient lines in a vector field. Still figuring out the details.Suggestions? Ideas?

Now let’s talk about using visual information for control

We need to estimate error e between actual and desired trajectory.Use sensory information

Vision is good: depth, color, texture.Challenge

Identify landmarks that are clearly visible despite motion.Will a fixed camera do? Or will we need to actively track landmarks with a moving camera?

Vision Based Control

Pose-Based approachConvert image data to robot’s state space to compute control inputs.Eberst et. al. [2 ]: door navigation.

Image-Based approachCompute control inputs directly from sensor inputs.Ma, Kôsecká and Sastry [7]: curve properties.Patel et. al. [10 ]: multiple sensors for door navigation.

So what’ new?-I

To the best of my knowledge Most research has focused on

Tracking specific trajectories such as curves without regard for the extent of the robot and comfort of passenger.Vision based navigation: navigating safely but not necessarily comfortably.

No one has used information from a body mounted camera alone for high performance feedback control.

So what’s new? -II

Results in all the research show either of theseBack-up maneuvers, chattering, spikes in angular velocity.Will not do for our application.

Research Plan

On a straight line trajectoryEvaluate feedback linearization and sliding mode control.Assume point mass and perfect odometry.Eliminate back-up maneuvers.

Use visual informationAssume noiseless sensors.

Introduce noise and uncertainty.Evaluate controller on more complex paths.

References[1] B. dAndrea Novel, G. Campion, and G. Bastin. Control of nonholonomic

wheeled mobile robots by state feedback linearization. The International Journal of Robotics Research, 14(6):543–559, 1995.

[2] C. Eberst, M. Andersson, and H. Christensen. Vision-based door-traversal for autonomous mobile robots. Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems, 1:620–625, 2000.

[3] R. Fierro and F. L. Lewis. Control of a nonholonomic mobile robot: Backstepping kinematics into dynamics. Journal of Robotic Systems, 14(3):149–163, 1997.

[4] B. Kuipers. Control tutorial. 2004.

[5] J.P. Laumond, S. Sekhavat, and F. Lamiraux. Robot Motion Planning and Control, chapter Guidelines in Nonholonomic Motion Planning for Mobile Robots, pages 1–54. Springer-Verlag,1998.

[6] A. De Luca and M. D. Di Benedetto. Control of nonholonomic systems via dynamic compensation. Kybernetica, 29:593–605, 1993.

References[ 7] Y. Ma, Kôsecká J, and S. Sastry. Vision-based door-traversal for autonomous

mobile robots. Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems, 1:620–625, 2000.

[8] G. Oriolo, A. De Luca, and M. Vendittelli. WMR control via dynamic feedback linearization: Design, implementation, and experimental validation. IEEE Trans. on Control Systems Technology, 10(6):835–852, 2002.

[9] J.M. Yang and J. H. Kim. Feedback control of a nonholonomic wheeled cart in cartesian space. Proc. IEEE International Conference on Robotics and Automation, 15:578–587, 1999.

[10] S. Patel, S. H. Jung, J. P. Ostrowski, R. Rao, and C. J. Taylor. Sensor based door navigation for a nonholonomic vehicle. Proc. IEEE International Conference on Robotics and Automation, 3:3081–3086, 2002.