Vision-based World Modeling Using A Piecewise Linear Representation of The Occupancy Function
-
Upload
kaseem-hawkins -
Category
Documents
-
view
12 -
download
0
description
Transcript of Vision-based World Modeling Using A Piecewise Linear Representation of The Occupancy Function
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Vision-based World ModelingVision-based World Modeling
Using A Piecewise Linear Representation
of The Occupancy Function
Dmitry O. GorodnichyDepartment of Computing Science
University of Alberta http://www.cs.ualberta.ca/~dmitri
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Outline
World Exploration Problem: Tasks and Sensors Why Occupancy Approach? Problems:
1. Designing fast and affordable 3D sensor:
Visual sensor implementation & Calculating reliability values
2. Combining dependent range data
3. Representing the occupancy:
Memory consumption & Utility to application Solutions Putting into a framework. Conclusions.
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
World Exploration Task
An agent equipped with range sensors is put in unknown environment. The agent tries to understand what is around it, by building a model of the environment, in order to accomplish a task.
Quality of the model is governed by:
1) the task the agent has to fulfill
2) the quality of the sensor
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Types of World Modeling
A test-bed problem:
“Find a hidden target (green triangle) in an unknown room by navigating and exploring”
Slow, but precise (NRC,ULaval):
For virtual environments
Cyber-cities, surgeries
Crude, but fast:
In mobile robotics
Exploring, finding objects
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Range sensors (Why Vision?)
Sonars (CMU,UMich,NCAR, …): slow, not for 3D acquisition
Laser-based (NRC,Laval, …, UofA): expensive and/or slow
Other: radars, lidars and infrared camera
Vision-based sensors (UBC,McGill,UBonn,CMU,UPenn, …):
1) Good for building 3D models and large scale models.
2) Good for surveillance and recognition
3) Fast* and affordable*
* Depends on the quality of the stereo system(The more robust the stereo, the slower and/or more expensive it is)
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Occupancy Approach
When sensor data are not reliable:
Calculate the evidence that a point in space is occupied by building the Occupancy Function from excessive amount of data:
Range data are pairs: depth & confidence values
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Occupancy Approach: background
To be used when
1. Sensors are not reliable
2. Different sensors are used
3. No geometry constraints can be implied
4. Environment is changing
5. Time is critical (e.g. in mobile robotics)
Previous work (1982-1998)
• Moravec, Elfes, Thrun, Borenstein, Pagas, Payeur, Elsner
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Occupancy approach: questions
Questions:
• How to assign m to r ? (to observed point & other points)
• How to build ? (to combine & represent)
• How to use it ?
Vision-based occupancy modeling:Vision-based occupancy modeling:
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Visual Sensor DesignVisual Sensor Design Previous work:
• For occupancy modeling: UBC, CMU,UBonn, Shirai Lab, …
• For geometry-based modeling: McGill, Bochum, INRIA, …
What do we want ?
• Low-cost and fast 3D range sensor
• To acquire data all around (but not too much of data)
• Easy assignment of confidence values
What does it mean?
• Using one off-the-shelf camera only
• Avoiding long computations
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Camera and Images What would you tell about the quality of this image?
(Those are green rectangles on white wall as observes by a camera)
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Camera and Images (cntd)
What would you tell about the distinctiveness of features?
(Those are features selected for tracking in a 160 x 120 image)
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Depth Calculation Details
Projective geometry: focal length F=120
Features:
• Are tracked along vertical epipolar line (+/– a pixel)
• Are those which are easy to track: Iy
| > Ithresh
• Marginal features discarded
RGB, 160 x 120 images, with no rectification
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Sensor model
Sensor model defines the evidence/probability that a point in space is occupied given a range sensor measurement :
Questions:
• How to calculate this value for the observed point ?
• How to calculate this value for neighbouring points ?
Using evidential approach allows one to calculate these values using the reliability parameters of the sensor.
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
More on Evidential Theory
Operates with both occupancy and emptiness evidence: m(o) and m(e), which are initially set to zero.
Accumulates them independently.
Provides Belief and Plausibility measures:
Bel = m(o) Pl =1 – m(e)
Advantages:
• Allows to use sensor error analysis in calculating the evidence of range data.
• Resolves "contradictory vs unknown" ambiguity (when m=0.5), by providing an interval of uncertainty [Bel;Pl]
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Sonar Sensor model
Probabilistic approach:
P(occ) = 1 – P(emp)
Evidential approach:
On the arc: m(occ)=1/N, m(emp)=0
In sector: m(occ)=0, m(emp)=1/S
Elsewhere: m(occ)=0, m(emp)=0
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Visual Sensor model
Where uncertainty comes from ?
• From the limitations of the camera
• From the complexity of the environment
What are parameters that determine the reliability of data ?
• Feature match error
• Depth calculation error
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Single Camera Sensor Model
Real camera:Ideal camera:
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Stereo Error Analysis
Quantization error gives:
Match error E gives:
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Image,Features & Range Data
(The brighter a range point, the higher its evidence)
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Image,Features &Range Data
From another robot location:
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Complete Scan of a Room
• It takes ~1 min :
two pairs of grabbed images in each of 15 camera pan positions.
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Combining EvidenceCombining Evidence
Fusion Problem:
- According to one measurement “point A is non-occupied”:
- According to another “point A is most likely occupied”:
• So, is the point occupied or not ?
• How sure are we in the answer ?
General situation:
Measurements describe not the same point, but neighboring points.
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Conventional Approaches
Probabilistic approach:
Bayesian rule (requires knowledge of conditional probability tables)
Evidential approach:
Dempster-Shafer rule (assumes DS-independence)
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
In practice simplified rules are used (ULaval):
These rules however cannot be applied for dependent range data!
Example (from real experiment):
The same unreliable feature is observed twice by the cameram1 = 0.6 & m2 = 0.6 result in m1+2 = 0.7
Other combination rules need to be investigated!
Assumption of Independence
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Fusion as Regression
Given a set of sample points from a sensor,
find a function such that minimizes the error
Offers a rule for combining dependent and adjacent data Requires evidential approach (probability axioms no longer hold)
Questions:
• How to incorporate sensor model?
• Which regression technique to use?
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Incorporating Sensor Model
Choose constraints - To suit well 1) sensor model, and 2) the application
Each feature induces a set of features according to the sensor model
Use it! Choose coordinate system - To be
convenient for 1) the application, and 2) sensor model
Generate sample points just enough.
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Rule-based vs Regression-based
Conventional approach:
Regression based approach approach:
3-5 points are enough (scale invariant)
Number of points depends on the scale
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Choosing Regression Technique
Two factors to take into account:
Factor 1: The quality of range data
• Robust regression (LMS, WLS) - deal with outliers
• Least Squares estimators – fast, but not good on outliers
Factor 2: The use of the constructed model
• Linear functions are easy to invert.
• Linear functions are easy to constrain.
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Adaptive Logic Network
ALN – Least Squares multiple Linear regression technique
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Experiments
Terms of experiments:
• Environment – large-scale, full of visual features.
• Range data – obtained by a single camera sensor.
• Regression – ALN-based.
Goals of experiments – to observe the problems:
• How are planar objects modeled in a radial-based system?
• How much time and memory does it takes to model?
• Is the quality good enough for planning navigation?
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Experiments (Simulated Data)
Cylindrical room with object:
L=150
(Error=0.15)
Empty rectangular room:
L=46
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Experiments (Real Data)
Problems observed:
• LS and outliers
• Discontinuity of the models
The precision of modeling is determined by the number of linear pieces used in regression: L=8 L=16
• Lack of features
• Visualization problem
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Representing the Occupancy
Conventional grid representation of m=F(x,y,z) requires storing and processing huge 3D arrays of data
Solution: Parametric representation.
Any function can be represented using linear equations:
Questions:
• How to calculate surface equations ? - Use regression.
• How to extract navigation maps from them?
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Extracting maps
1. Obtain the inverse (which is possible due to the monotonicity constraint):
2. Apply at different heights h, shrink and take the intersection
The obtained polygon can be used in navigation planning.
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Planning Navigation
Use Reinforcement Learning or Potential Fields approach.
Reinforcement of action aa (where to go) taken at state SS (of the
robot and environment) is r(S,a) = rgoal + rexploration + robstacles• Obstacle points repel: robstacles < 0• Goal and exploration points attract: rgoal > 0, rexploration > 0
Optimal navigation policy maximizes the total discounted reinforcement defined by the Value Function:
Value Function of the optimal policy satisfies Bellman’s equation:
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Planning Navigation (cntd)
Obstacle and exploration points are extracted from the maps:
Value function is calculated using ALN-based reinforcement learning.
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Putting it together
``Explore an environment with a video-camera''``Explore an environment with a video-camera''
Explore environment (e.g. to locate an object) by
planning intelligently navigation using
compact and convenient models constructed by
combining uncertain dependent range data
registered by an affordable and fast 3D range sensor.
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
From Image Processing to Navigation planning
Movie 1: From 2D features to 3D depth map to 3D occupancy models to 2D navigation polygons.
Movie 2: Looking for the target in unknown environment.
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Conclusions
The problems of occupancy modeling identified. Evidence-based visual sensor model is developedEvidence-based visual sensor model is developed
(allows one to use an off-the-shelf video-camera). Regression-based range data fusion technique is developedRegression-based range data fusion technique is developed
(allows one to combine dependent data). Methods for building and using parametric occupancy modelsMethods for building and using parametric occupancy models
(require little memory and yield efficient map extracting).
Framework established. Major problems identified. Further improvements shown.
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Directions for Further Improvement
Better range data:
Better cameras, better calibration, image rectification;
Better feature selections, robust tracking
Other range sensors (e.g. laser range, infrared scanners)
Other regression and neuro-computational techniques
Belief Occupancy in addition to Plausibility Occupancy
Combine several models into one model
Work on improving the accuracy and apply to other exploration tasks (e.g. building virtual environments, robot localization, tele-robotics, texture mapping)
Dmitry Gorodnichy. Vision-based Occupancy Modeling.
Acknowledgements
This research was partially supported by the Defense Research Establishment Suffield.
The ideas of using piecewise linear surfaces for occupancy function approximation and using only one camera for 3D data acquisition were initially suggested by W.W. Armstrong.
Thanks to employees of Dendronic Decision Limited, Edmonton, Canada: Kyle Palmer, Monroe Thomas, Brant Coghlan and Ron Kube.
The influence of Sandro Botticelli and other artists of the Renaissance, the cultural movement concerned with the multifaceted development of a human being, is acknowledged as well.