An Edge detection and HPF-based Intelligent Space – A Network based Integrated Navigation System...

An Edge detection and HPF-An Edge detection and HPF-based Intelligent Space – A based Intelligent Space – A Network based Integrated Network based Integrated

Navigation SystemNavigation System

ByBy, , Rachana Ashok GuptaRachana Ashok Gupta

Under the direction ofUnder the direction ofDr. Mo-Yuen ChowDr. Mo-Yuen Chow

NC STATE UNIVERSITY

ADAC, NC State University 2

Overview of presentation

What are network based integrated navigation system

(NBINS) in brief followed by Abstract for this research.

Introduction to iSpace

– iSpace components and modules

– Limitations and Scope of improvement as a NBINS

The new structure of iSpace

– New modules (edge detection and HPF planner)

– Advantages and improvements achieved over the old structure

Results and Discussions

NC STATE UNIVERSITY


Network based integrated navigation system

Advantages– Remotely control over a long-distance.– Efficient to fuse global information.– Scalability - Easy to add more sensors and UGVs with very

little cost and without heavy structural changes. Applications

– Manufacturing plant monitoring– Nursing homes or hospitals – Tele-robotics & Tele-operation etc

Issues to be considered– Network and processing delay – Data sharing and – Interfacing

Network based Integrated Navigation systems – Different modules combined together to guide a UGV (Unmanned Ground Vehicle) from one point to another in the space of interest (2), where the navigational intelligence lies on a main controller away from the UGV.

NC STATE UNIVERSITY


What is Intelligent Space

A new concept to effectively use distributed sensors, actuators, robots, computing processors, and information technology over a physically and/or virtually connected space. For examples, a room, a corridor, a hospital, an office, or a planet.

It fuses global information within the space of interest to make intelligent operation decision such as how to move a mobile robot effectively from one location to another.

Human

Webcam

Webcam

Mobile Robot

MicrophoneMicrophone

Human-machine interaction in iSpace

NC STATE UNIVERSITY


iSpace as a NBINS

Components– Overhead network camera– Network controller with graphical

user interface – A Differential drive UGV as the

navigator– Computer Network (IP)

On the main network controller– Image acquisition– Image processing– Path generation– Path tracking controller

Graphic User Interface (GUI)– Any remote Computing interface in

the world (Internet)

NC STATE UNIVERSITY


Template Matching

Image acquisition– Top view

Image processing– Image Thresholding – Black and

white– Template matching– Circles used for rotation invariance– Draws the circle around obstacle

with safety margin radius of rsafe.

2

( , ) ( , )( , ) 1i j i jA B A B

j i

h f f f f

1tanb f

b f

y y

x x

fB – Template image

NC STATE UNIVERSITY


Path generation

Find a path from the starting point A to the end point B for the UGV

– The path of the UGV should be as short as possible (minimize time)

– The path of the UGV should not collide with any obstructions

Fast Marching Method (by J.A. Sethian, Dept. of Mathematics, UC Berkeley)

– A numerical technique that counts the shortest distance from a point to the original point with a shortest distance update algorithm

NC STATE UNIVERSITY


Path tracking

The path tracking algorithm runs in every control loop and adjusts the speed and turn rate of the UGV to track the generated path.

1. Calculate the closest point on the reference path from the current UGV position (xc, yc,c) .

2. Pick a reference point (xref, yref)

on the generated path that is a set distance (d0)

3. speed and turn rate for the UGV to reach the reference point given its current position and orientation.

LloadRloadLR

L

R

Wr

Wr

rr

carLR

carLR

LR

FrTFrTvv

v

w

vwith

JW

FF

JFFW

vMFF

LR

,,2

)3(2

)(2 22

R

v

Wr

LF RF

12

12

Wr r

Wr r

r

l

max0 1

dd

An

NC STATE UNIVERSITY


Time delay issue

Network delay component

– Sensor to controller delay (Image Acquisition)

– Controller to actuator delay (Commands from controller to the UGV)

Processing or computational delay component

– Non-real time Computational delay (Initial image processing, path

planning)

– Real time Computational delay (continuous image processing, motion

control)

NC STATE UNIVERSITY


Limitations - Template matching

Insufficient safety margin

Conservative safety margin

Template matching output– Location co-ordinates of

» UGV» Obstacles

Limitations– Obstacles are from a

priori set– No knowledge about

shape and size– Either Insufficient or

conservative rsafe – Leading to inefficient or

non-optimal path planning

All these make the system restricted to operate in only a few environmental patterns.

rsafe

NC STATE UNIVERSITY


Limitations with Fast Marching

Implements and maintains a binary tree through out the path generation.

O(N(LogN)) problem and memory intensive. The time and the number of loop iterations are dependent on the respective position of the destination from the source.

Operations involved are – search, distance calculations (squares and sqaure root functions)

Every iteration – need of check whether the TRIAL point is outside the rsafe margin of recognized obstacle.

Alive

Trial

Close

Destination

Far

Obstacles Far

Obstacles

NC STATE UNIVERSITY


Limitations with path planning

Navigation problem is looked upon as a path tracking problem and therefore the reference path generation is mandatory.

The reference point generation on the path for the UGV (off the path) does not consider the obstacle avoidance.

Quadratic curve controller needs current position and the reference point to calculate speed. Complete reference path is not needed.

Real time computation to find the closest point on the reference path

Point to point guidance function

NC STATE UNIVERSITY


Important points

Delay tolerance, efficiency, accuracy, generality and optimality of iSpace depend upon the following factors.

Fast, efficient and generic enough Image processing algorithm with different category of obstacle maps for the navigation system.

The path planning algorithm will decide – Optimality and length of the path generated

Path tracking algorithm has to continuously consider the obstacle avoidance for a navigation system

– Probability to hit an obstacle. – The time required to track that path

Compatibility and data flow between different modules is also a key factor for an integrated system.

NC STATE UNIVERSITY


New Structure for iSpace

Points of emphasis –

1. Processing and computation delay, increase in the efficiency, generality, optimality with each added new module to create a suitable platform for NBINS.

2. Creating a homogenous structure by putting edge detection, HPF planner and quadratic curve fitting path tracking controller - three heterogeneous systems together for the first time to create a network based system is the novel contribution to the network based integrated navigation system area.

NC STATE UNIVERSITY


Edge Detection

Image

I

Laplacian of Gaussian 2G

Gradient of Gaussian G

Zero Crossing

Magnitude Threshold(C)

Edgemap

E

E(xi, yj) = 1 if (xi, yj) B

= 0 if (xi, yj) B for all (i, j)

Where E(x, y) is the image representing the edge map and B is the set of edge points including the

boundary points for all obstacles in workspace.

2 22 ( , )

2 2G G

G x yx y

2 2

22

1 ( )( , , ) exp 22

x yG x y

( , )T

G GG x y

x y

NC STATE UNIVERSITY


2-D Harmonic Potential Field

2 is the Laplace operator, is the workspace of the UGV ( 2), is the boundary of the obstacles (output of the edge detection stage), and is (xT, yT) the target point. The obstacles were represented by the repelling force and the point of destination was represented by the attractive force.

– The potential function in closed contour of will converge to a constant potential

– The obstacle free path to the target is generated by traversing the negative gradient() i.e. . The normalized gradient at each point represents the directional guidance at that point in the workspace.

2

T T

( , ) 0 , Ωsubject to ( , ) 1 , Γand ( , ) 0 ( , ) ( , )

x y x y

x y x yx y x y x y

02

2

2

22

yxh

Laplace equation for 2

is a Harmonic function.

NC STATE UNIVERSITY


Solving Laplace Equation

1 12

1 12

( , ) 2 ( , ) ( , )( , )

( , ) 2 ( , ) ( , )( , ) ( 1)

i j i j i jxx i j

i j i j i jyy i j

u x y u x y u x yx y

hu x y u x y u x y

x y h kk

21 1 1 1( , ) ( , ) ( , ) ( , ) ( , ) 4 ( , )i j i j i j i j i j i jh x y u x y u x y u x y u x y u x y

1 1 1 12 ( , ) ( , ) ( , ) ( , )( , ) 0 ( , )

4i j i j i j i j

i j i ju x y u x y u x y u x y

h x y u x y

Thus with Laplace equation, this method simply consists of repeatedly replacing each grid points with the average of its neighbors using successive relaxation. Terminate when the array u contains a sampling of where every non-boundary condition node has a neighbor with a smaller value representing a negative gradient except the destination point.

Using finite difference method, Taylor series approximation,

NC STATE UNIVERSITY


HPF with synthetic data

The destination point is represented by the lowest potential ( =

-1)

NC STATE UNIVERSITY


HPF and Edge detection2

T T

( , ) 0 , Ωsubject to ( , ) 1 , Γand ( , ) 0 ( , ) ( , )

x y x y

x y x yx y x y x y

E(xi, yj) = 1 if (xi, yj) B

= 0 if (xi, yj) B for all (i, j)

is the boundary of the obstacles

Where E(x, y) is the edge map and B is the set of edge points including the boundary points for all obstacles in workspace

B nothing but boundaries of the obstacle, , raised to a high potential. Thus it provides the exact raw data required for HPF in Dirichlet’s setting to create the gradient direction matrix.

The destination point is then represented by the lowest potential ( = -1)

NC STATE UNIVERSITY


HPF, a region to point guidance function

We observe that the HPF plan of the workspace is the function of obstacle boundaries and the destination point. Thus HPF converts the edge map into a “Region to Point Guidance Function”

NC STATE UNIVERSITY


Goal Seeking with HPF

The important feature of HPF planner is to convert the edge map into a region to point guidance function. the problem is converted to a “goal seeking” problem from a “path tracking” problem.

22

yx

xVx

22

yx

yVy

2

),1(),1( yxyxx

2

)1,()1,(

yxyxy

NC STATE UNIVERSITY


HPF with Motion Controller

The reference position for each current position is calculated from the gradient array of the HPF ().

timesL

),(

),(

000

000

yxVyy

yxVxx

yR

xR0 0( , ) ( , )R RR x y x y

L – discretized look-ahead distance. (ex, ey, e) the error vector is calculated for (x0, y0, 0) and (xR, yR)

( ) 21 | |n x n nK sign e v K An K

An

2( ) y

xx

eAn sign e

ey = An x2

NC STATE UNIVERSITY


Effect of look-ahead distance

L = 1, Network delay = 0.1sT = 27 seconds

L = 8, Network delay = 0.1sT = 16 seconds

L = 8, Network delay = 0.6 s

The magnitude of the speed v and the turn-rate is proportional to the distance d0 between (x0, y0) and (xref, yref)Small L – UGV close to the path – Small distance error – more time Large L – Less time – large distance error – probability to hit the obstacle

NC STATE UNIVERSITY


Dynamic look-ahead distance

0 D

dL

G

0.0125a aD

x yG m

m n

GD – grid size distanceWorkspace image resolution – (m x n) = 320 x 240 Workspace size – (xa x ya) = (4m x 3m)High curvature point small d0, small L The UGV runs slowly on the turn. Path is a straight line (low curvature point) large d0, large L. UGV moves faster. Optimality between the path tracking accuracy and the time required to reach the goal.

Look-ahead distance (d0) is

function of curvature.max

0 1

dd

An

From the Quadratic curve controller

y = An x2

Network delay = 0.3 s

T = 24.3 s

NC STATE UNIVERSITY


Edge detection Vs Template matching

Template matching Need of templates for the obstacles Shape and size restriction on these obstacles. Knowledge about only the location of the obstacles No knowledge about the actual boundaries of the obstacles Many restrictive assumptions about the environment

Edge Detection No need of templates No Shape and size restrictions on the obstacles Capture a full representation of the environment Boundaries of the obstacles are marked distinctively. No restrictive assumptions regarding its contents.

NC STATE UNIVERSITY


Fast Marching Vs HPF

Fast Marching with Motion controller

Source to destination Reference path Path tracking Reference point calculation does not have obstacle vicinity knowledge. Computationally and Memory intensive algorithm.

HPF with Motion controller Region to point Reference array for the whole workspace. Goal seeking Reference point is calculated using the gradient of HPF planner, which always drive UGV away from obstacles. Less computationally intensive (simple averaging operation)

NC STATE UNIVERSITY


Real time computation

.

.

Np n Np

L n L

1 10 0( , ), ( , ), ..., ( , )Path - Np Npx y x y x y

Np – Number of points on reference path

At each sampling instance ti –

Np real time checks on the path to find out the closest point and (xref, yref)

say n.ti – time required to reach the goal (n – number of loop iterations)

Total (Np.n) real time computations

Np L p 1

Each reference point (xref, yref) computation in real time will take L addition operations.

Total (L .n) real time computations.

Performace improvement factor

p =

timesL

),(

),(

000

000

yxVyy

yxVxx

yR

xR

NC STATE UNIVERSITY


Key Interface points

Old structure of iSpace1. After template matching, safety radius has to be decided before the points can be passed on to the fast marching method.

2. Fast marching method has to keep track of all the points using a binary tree, whether the point lies inside or outside the safety circle.

3. Reference point calculation in real time is mandatory due to path tracking problem.

New structure of iSpace1. The output from edge detection can be directly fed to HPF planner without preprocessing

2. Knowledge of boundaries (high potential) make it easy with successive relaxation to get the gradient array

3. Reference array calculation from the HPF planner makes it quick for the quadratic curve fitting controller.

The new structure has efficient interfacing

NC STATE UNIVERSITY


Results - 1

For Comparison purpose, ideal path from source to destination was generated

0 0.2 0.4 0.6 0.8 1 1.2 1.40

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

Network delay (second)

Max

imum

Dis

tanc

e E

rror

(m

eter

)

Maximum Distance Error Comparison

HPF

Fast Marching

0 0.2 0.4 0.6 0.8 1 1.2 1.40

0.02

0.04

0.06

0.08

0.1

0.12

0.14

Network Delay (second)

Mea

n D

ista

nce

Err

or (

met

er)

Mean Distance Error Comparison

HPF

Fast Marching

NC STATE UNIVERSITY


Results - 2

Left – HPF planner and Right – fast marching planner for same grid size

0 0.2 0.4 0.6 0.8 1 1.2

12

14

16

18

20

22

24

Tim

e t

o r

each

th

e g

oal

(seco

nd

)

Network delay (second)

Total time comparison

HPF

Fast marching

Image with a barrier separating two regions. (Blue dot is the source and red dot is the chosen goal.)

NC STATE UNIVERSITY


Conclusion

Edge detection, a model based HPF planner and network based quadratic Controller go hand in hand to create an efficient and delay-tolerant integrated navigation system.

More generality and flexibility to the UGV workspace environment. A good edge map helps to build the correct HPF planner. The gradient array calculation from HPF planner decreases the

computational burden in real time making - more suitable for network-based control.

The Dirichlet’s setting keeps the robot path, as much as possible, away from the obstacles making it efficient even in heavily cluttered environments.

The combined effect of HPF and the quadratic curve controller display intelligent behavior such as no movement in case of goal unreachable problems.

Thus the new iSpace structure suggested satisfies many requirement which are key for a network

based integrated navigation system.

NC STATE UNIVERSITY


Future Research

Improvement on edge detection as reliable edge detection is the backbone of the new structure of iSpace.

Considering the dimensions of the robot before creating the HPF planner to take care of the safety margin around the wall. (Possible solutions - Dilation of the edge map)

Using HPF for velocity control for the UGV. Dynamic obstacle avoidance for NBINS.

NC STATE UNIVERSITY


An Edge detection and HPF-based Intelligent Space – A Network based Integrated Navigation System...

Documents

Transcript of An Edge detection and HPF-based Intelligent Space – A Network based Integrated Navigation System...