RAVON - Platform for Outdoor Field Robotics -...
Transcript of RAVON - Platform for Outdoor Field Robotics -...
![Page 1: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/1.jpg)
RAVON - Platform for Outdoor
Field Robotics
Prof. Dr. Karsten Berns
Robotics Research Lab
Department of Computer Sciences
University of Kaiserslautern
![Page 2: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/2.jpg)
Application Areas for Off-road Robots
• Service tasks
– agriculture, forestry, logistics,
environmental monitoring
• Assistance in disaster areas
– equipment transport, scouting
• Security tasks
– site / border patrol, demining
![Page 3: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/3.jpg)
Off-road Robots for Different Applications
Timberjack
© John Deere
Workpartner
© Helsinki University of Technology
Telerob tEODor
© www.telerob.de
Remotec Wheelbarrow Revolution
![Page 4: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/4.jpg)
Off-road Vehicles
Stanley (Stanford) RTS-Dora (Hannover)
Demo III (U.S. Army)
Navlab (CMU)
RoboScout (Base 10)
Sojourner (JPL) Amor
(Uni Siegen) MuCar
(UniBW München)
![Page 5: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/5.jpg)
RAVON Project
Goal: Development of vehicle that can fully autonomously operate in
highly vegetated, rough terrain
![Page 6: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/6.jpg)
Operational Environment
![Page 7: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/7.jpg)
RAVON
• 4WD with separate motors
• Front and rear axes can be steered
independently
• Max. velocity: 10 km/h
• Max. slope: 100% at 7 km/h
• Energy source: 8 Lead batteries
• Runtime: about 4 hours
• LWH: 2.35x1.4x1.8 m
(highest point: GPS antenna)
• Weight: 750 kg
• Ground clearance: 0.3 m
![Page 8: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/8.jpg)
Control System Architecture
Drive
Commands
Hardware Interface
Sensor Processing
Sector
Map
Creation
Mediation
Global Navigation
Short-term
Memory
Raw
Sensor Data
Se
cto
r M
ap
s
Status
Status Target
Poses
Target
Poses
Safety System
Point Access
Local Path Planning
Local Navigation
![Page 9: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/9.jpg)
External Sensor Systems
Long Range Color Stereo System Large Scale Terrain Traversability Estimation
Short Range Color Stereo System Obstacle Detection
Rotating 2D Laser Scanner Obstacle Detection / 3D Local Memory
Planar 2D Laser Scanner Obstacle Detection / Safety System
Spring-Mounted Bumper Tactile Vegetation Discrimination
![Page 10: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/10.jpg)
Instrumented Safety Bumper
Design and construction of the bumper
Tactile Mapping of the environment
![Page 11: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/11.jpg)
Titel des Vortrags
3D Laser Scanner Concept
![Page 12: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/12.jpg)
Sensor Processing
Main sensor: panning laser range finder
• Yields position, shape, and type of hazards in 3D space, e.g.:
Solid Object Penetrable Object
• Penetration technique for evaluating an obstacle’s rigidity
( Gras or rock?)
h
d h h
d
Positive Obstacle Negative Obstacle Overhanging Obstacle Water
![Page 13: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/13.jpg)
Titel des Vortrags
beneath ground
ground
near ground
far ground
sky
shortest ground
distance
Y
X
• Extract ground representatives
(blue)
• From ground level classify:
• Pts beneath ground
• Associated ground pts (purple)
represent the load-bearing
surface
• Near ground pts
(surmountable)
• Far ground pts (positive or
overhanging obstacles)
• Sky points
Evaluation of Individual Scans
![Page 14: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/14.jpg)
Titel des Vortrags
Solid Entities
Vegetation
Classification – Examples
![Page 15: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/15.jpg)
Example of Content of Grid Map Cells
• General information: Properties
• Can also store height or organize cell space in 3D
![Page 16: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/16.jpg)
Titel des Vortrags
Vision-Based Obstacle Detection
• Stereo-vision-based 3D scene analysis
• Tracking of obstacles over time
-> 4th Dimension
• Filter artefacts
• Building local maps
• Determination of colour
and texture
![Page 17: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/17.jpg)
Water Detection based on Polarisation
Detection Process
AquaVisor C
(H&S Robotic Solutions, KL)
Compact design
90mm x 90mm x 130mm
Weight: 600g
Onboard sensor data
evaluation
Extension modules for
Vegetation detection
3D-enhanced Vision
![Page 18: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/18.jpg)
Grid Map as Local Obstacle Memory
• Robot centric, orientation-fixed local grid map
• Scrolls in x- and y-direction
• Serves as local obstacle memory
– Robot “remembers” obstacles
– Collisions with obstacles not in direct sensor range can be
avoided
![Page 19: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/19.jpg)
Sensor Data Representation
• Data of different sensor systems is entered into
grid maps
• Grid maps used as “local obstacle memory”
Important as sensor systems cannot
cover complete area around robot
• Sector maps as abstract and uniform
data representation
• Extracted from grid map
• Each sector contains
information about area
it covers Cartesian Sector Maps Polar Sector Map
Grid Maps
Extraction
![Page 20: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/20.jpg)
Sector Maps around RAVON
![Page 21: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/21.jpg)
Local Navigation (Pilot) and Drive Modes
• Task: drive robot to next target location while avoiding collisions with
obstacles
• Fast driving: requires long-range obstacle detection and the
evaluation of all detected obstacles
• Moderate driving:
– Uses penetration evaluation for vegetation density determination
– Maximum velocity reduced as only close-range area can be
monitored
• Tactile creep:
– Evaluates deflection of the bumper system
– Maximum velocity is limited to very low value
Pilot can adopt robot’s basic driving behaviour to situation
![Page 22: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/22.jpg)
Behavior-based Control Approach
Approach: Decompose task into relatively simple independent
components
• Incremental implementation with subsequent evaluation
• Distributed representation of environment, no central world model
Problems to be solved:
• Find suitable decomposition
• Coordination of contradictory commands
• Validation of system behavior
![Page 23: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/23.jpg)
Behavior-Based Architecture iB2C
• Uniform behavior modules
• Behavior B defined as B=(r,a,F) with
– Input vector e and output vector u sensor/control data
– Stimulation s in [0,1] gradually activating the behavior
– Inhibition i in [0,1] gradually deactivating the behavior
– Activation ι=s∙(1-i)
– Activity a in [0,1] indicating how much B is doing
– Target rating r in [0,1] assessing current situation
– Transfer function F(e,s,i)=u implementing the functionality
Basic behavior module
![Page 24: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/24.jpg)
Behavioral Groups
![Page 25: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/25.jpg)
Behavioural Group for
Fast Driving
Structure of the Pilot
Emergency Stop
Drive Commands
Point Access
Behavioural Group for
Moderate Driving
Behavioural Group for
Tactile Creep
State Switching and
Behavioural Group Stimulation
Stimulation
Drive Commands
Drive Commands
Fusion
Drive Commands
Human
Operator
![Page 26: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/26.jpg)
Control Chains
![Page 27: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/27.jpg)
Slow Down Behaviors
![Page 28: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/28.jpg)
Safety Behaviors
![Page 29: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/29.jpg)
Keep Distance Behaviors
![Page 30: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/30.jpg)
Evasion Behaviors
![Page 31: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/31.jpg)
Structure of the Guardian
![Page 32: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/32.jpg)
Intermediate Layer - Passage Driving
• Use of passage realised by behaviour-based network
• Robot is drawn towards PEMP
=> does not drive into indentation
• Robot reaches PEMP with orientation of passage
=> is driving smoothly into the passage
Start
Passage Entry
Draw towards Passage
Target
Passage Orientation
![Page 33: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/33.jpg)
Global Robot Navigation
• How can a robot navigate efficiently to
a distant location?
• Requires a world model (a map)
– quantify terrain traversability
→map cost measure
• Traversability estimation is hard
– ground properties
– flexible/rigid vegetation
?
![Page 34: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/34.jpg)
Proposed Combined Approach
• Formulate a global (off-road) navigation strategy which
– scales to very large environments, but
– allows to reason about terrain traversability
• Solution strategy:
– use hybrid world representation
• global topological navigation layer
• local metrical pilot layer
– extend topological world model to handle traversability costs
– transfer local cost experience to global level
![Page 35: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/35.jpg)
Abstract Views - Example
• Sensors
Physical
Short-Term Memory
Slowly Traversable Terrain Quickly Traversable Terrain
Sensors
Virtual
Sensors
![Page 36: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/36.jpg)
Pilot
• Receives target from topological navigator
• Uses sensors to build metrical (grid) short-term memory
• Relevant memory content is transformed into standardized ‘abstract views’
• Behavior based control system uses abstract views for steering and collision avoidance
![Page 37: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/37.jpg)
Local Traversability Maps (LTMs)
• Obstacle information is
relevant for later
topological map
extension
• Data is transferred from
abstract views into local
traversability maps
(LTMs)
• LTMs store abstracted
cost modifiers around a
topological node
![Page 38: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/38.jpg)
Local Traversability Maps
![Page 39: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/39.jpg)
A posteriori Cost Learning
• Observation of
behaviors yields cost
information after each
edge traversal
• Spatio-temporal
integration
• Transfers real-world
cost experience from
local to global scope
![Page 40: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/40.jpg)
Topological Exploration
• Generate multiple new
path hypotheses
between two
unconnected nodes
• Predict likely costs
• Add cheapest route
• How to predict costs for
unknown edges ?
![Page 41: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/41.jpg)
Edge Cost Calculation
• Three types of edge costs: risk, effort, and familiarity
• Risk and effort can be learnt during edge traversing by monitoring
distinct behaviours of pilot
No need to analyse the terrain extensively and construct a highly
detailed world model
• Different methods to predict costs of edges that have not been
traversed yet
Costs for driving in partly unknown terrain can be estimated
![Page 42: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/42.jpg)
Vision Based Terrain Analysis
![Page 43: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/43.jpg)
Image Feature Extraction
stereo camera system
color image pairs
texture area descriptors
HSV
VAR
• Highly parallelizable operators extract independent information
– Local Binary Pattern, VARiance, Hue Saturation Value
• Descriptors support efficient area union: →
contrast
color
LBP
BA
)( Adi
)()()( BdAdBAdiii
A
![Page 44: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/44.jpg)
Unsupervised Segmentation
• Construct image regions with similar apperance → Fewer, more stable objects to classify
• Unsupervised Split & Merge segmentation – Jenson-Shannon divergence as pseudo-metric
– Threaded neighborhood lists for efficient merging
![Page 45: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/45.jpg)
Self-Supervised Learning Procedure
1. Stereo image capture
2. Edge traversal
3. Experience projection
4. Region correlation
5. Prototype addition
![Page 46: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/46.jpg)
Experiment: Self-Supervised Learning
![Page 47: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/47.jpg)
Offroad Path Detection
• Not only look for free traversable space
• Find structures that help reaching the horizon
![Page 48: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/48.jpg)
Long Range Stereo Reconstruction
![Page 49: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/49.jpg)
Finding Ground and Horizon
• “Side-view“ Analysis
• Filter by density
• Apply RANSAC and least squares fitting
• Classify horizon and ground region
![Page 50: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/50.jpg)
Obstacle Analysis
• Front-to-back Analysis
• Adopt obstruction to back
• Extract upper ground edge
• Fit polynomial to ground
• Use derivative for traversability
classification
• Take declination of vehicle into
account
.
.
.
![Page 51: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/51.jpg)
Traversability Map
• Apply perspective back-transformation
• Extraction and filtering of traversable regions
![Page 52: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/52.jpg)
Path Classification via Particle Filter
• Apply path primitives (polynomials of deg. 2)
• Primitives are managed as particles
• Overlapping of traversability map and primitive
form confidence
+ =
![Page 54: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/54.jpg)
Place Recognition
• Machine-readable description needed
• Descriptors must be comparable
• Common known fact: Single features do
no generally work on their own
• Redundancy compensates for single
feature failure
• What is a place?
– Relevance for navigation decisions
– Descriptor is prominent in local area
![Page 55: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/55.jpg)
Point Features (SURF)
• Salient points in scale-space (similar to SIFT)
• Gradient in these point forms feature descriptor
• Set of features forms place descriptor
![Page 56: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/56.jpg)
Comparison of Point Features
• Use RANSAC filter for affine transformed correspondence sets
• 500 random sets are feasible for shifted and scaled images
![Page 57: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/57.jpg)
Area Features (LBP)
• Partition image and compute LBP histgram for each tile
• Image descriptor is the list of histograms
![Page 58: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/58.jpg)
Comparison of Area Features
• Shift images and compare histograms
• Calculate normalized sum of absolute differences
• Example for not matching images
![Page 59: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/59.jpg)
Comparison of Area Features (Matching)
• Minimal value for each pair relates to best shift
• Low value means high similarity
• Not robust against roll rotation
![Page 60: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/60.jpg)
Results: Correct Decision due to SURF
• SURF (blue) outweighs uncertain and wrong LBP (orange)
• Image pairs showing the same place usually have a great number
(>>10) of SURF matches after RANSAC filtering
![Page 61: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/61.jpg)
Results: Correct Decision due to LBP
• In some situations SURF failes
• LBP does not dominate in one place but can compensate
• Still correct with wron SURF (blue) and correct LBP (orange)
![Page 62: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/62.jpg)
Control System Architecture
![Page 63: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/63.jpg)
Travel through Palantine Forest
![Page 64: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/64.jpg)
Thank you for your attention!
http://agrosy.informatik.uni-kl.de
![Page 65: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/65.jpg)
References
• [Wünsche 07] LIDAR-based Perception for Offroad Navigation M.
Himmelsbach, F. Hundelshausen, H-J. Wünsche
• [Siegwart 07] Path Planning, Replanning, and Execution for
Autonomous Driving in Urban and Offroad Environments R.
Philippsen, S. Kolski, K. Macek, R. Siegwart
• [Peterson 05] Red Team Too, Darpa Grand Challenge 05, Technical
Paper, K. Peterson et. al.
• [Urmson 05] High Speed Navigation of Unrehearsed Terrain, C. Urmson
et. al.
• [Pfaff 08] Probabilistic Models for Autonomous Systems, P. Pfaff
![Page 66: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/66.jpg)
References 2
• [Thrun 95] Map Learning and High Speed Navigation in RHINO,
S. Thrun et. al.
• [Vale 95] Mobile Robot Navigation in Outdoor Environments: A
Topological Approach, A. Vale
• [Tapus 05] A Cognitive Modeling of Space using Fingerprints of
Places for Mobile Robot Navigation, A. Tapus et. al.
• [Madrigal 99] Mobile robot path planning: a multicriteria
approach, J. Fernandeza-Madrigal et. al.
![Page 67: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/67.jpg)
Related Work
• Most offroad robots use metrical grid maps
• SmartTer (ETH Zuerich) [Siegwart 07]
– global planning: multi-level-surface map, field D* algorithm
– local planning: grid map, driving arcs
• Red Team (Sandstorm, CMU) [Urmson 05]
– global: A* on high-resolution elevation map
– local: laser scan evaluation, path adjustment
• Red Team Too (CMU) [Peterson 05]
– global: A* on high-resolution elevation map
– local: obstacle + traversability cost map, A*
• MuCar (Uni BW München) [Wünsche 07]
– local: traversability cost map, driving ‚tentacles‘
![Page 68: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/68.jpg)
Related Work
• Topological Maps are mostly used indoors or in urban areas
• RHINO (Uni Bonn) [Thrun 95]
– focus on fast path planning (dijkstra), abstraction (voronoi)
– distance based cost measure
• ATRV-Jr (Uni Lisbon) [Vale 05]
– Focus on localization (POMDP), map building (EM)
– constant cost measure
• Tapus (ETH Zürich) [Tapus 05]
– focus on localization (POMDP), map building (fingerprints)
– constant cost measure
• Ram-2 (Uni Malaga) [Madrigal 99]
– focus on multicriteria path planning
– hierarchical constraint satisfaction, pre-learned costs
![Page 69: RAVON - Platform for Outdoor Field Robotics - …cyphynets.lums.edu.pk/images/Berns_ravon_wfr2011_slides.pdf · 2013-10-24 · RAVON - Platform for Outdoor Field Robotics Prof. Dr.](https://reader031.fdocuments.in/reader031/viewer/2022022603/5b5d09367f8b9a68368dfb0b/html5/thumbnails/69.jpg)
LADAR based Classification