Obstacle avoidance system for an ROV
Master thesis by: Lars BruslettoSupervisor: Professor Mart in Ludvigsen
Problem
Autonomous underwater operations need to be able to avoid obstacles.
Is it possible to do this with computer vision in underwater situations?
This thesis work is about obstacle avoidance methods that work in low light, scattering, blurring, and dif f icult camera calibration.
Disparity map obstacle avoidance
We don't want to crash into unknown objects Stereo rig
If (obstacle)
Calculate center of given object.
Calculate possible path away from object
Send calculated path directions using UDP to path program
A BStereo block
match algorithm
Rectify image Aquire images using VimbaSDK
SLIC Superpixel LBP method -->Train (Machine Learning) model
Uniform Local Binary Pat tern
-low computat ional
-high qual ity descript ion
-rotat ion invariant texture
classif icat ion
- i l luminat ion invariant
Machine Learning
Linear Support Vector
Classif ication
Feature extract ionClassif ier t raining
Image database Segmentation
Simple l inear i terat ive
clustering (SLIC)
Fit model
Compute the histogram of the LBP
Histogram computat ion
To be able to calculate disparity:
- need a good camera calibration
Underwater this is tricky:
Figure: Chessboard placed at 80 m depth
Because: Calculate intrinsic and extrinsic camera parameters.
Co-Supervisors: Phd. candidate Trygve Fossum, Phd. candidate Stein M Nornes Phd. candidate Mauro Candeloro
--> Development of the SLIC Superpixel LBP method
--> Development of the Disparity method
--> Development of a way to acquire images from allied vision cameras with python
---> Development of simulation environment module to test computer vision obstacle avoidance algorithms .
--> Development of SLIC Superpixel LBP classif ication algorithm that might be used in dif ferent kinds of applications, i,e autonomous pipe following , rust detection , shell f ish recognition ,star f ish recognition and more.
Main Contribut ions
The obstacle avoidance system takes control of the path choice
when it detects obstacles.
Sends calculated path to path program
Part of autonomy program
Disparity image with a f itted ell ipse
Resul tsUsing model to Predict and avoid obstacle
Conclusion
The disparity method and the SLIC Superpixel LBP method is successfully implemented and tested. The disparity method is proven to work in sea trials in the Trondheimsfjord.
Under simulation experiments the two methods have been compared and they both give very satisfying results in obstacle avoidance.
The disparity method is faster, and therefore better suited for real t ime applications.
The SLIC Superpixel LBP method has great potential also for other underwater applications as it is rotation and il lumination invariant. This means it is robust under dif ferent rotation and light conditions.
Future work
One could experiment with training the classif ier for more objects as the model has capacity to classify more than two classes. It could be interesting to train the classif ier for "reef", "sand", "subsea installation" and try to preform autonomous scene interpretation based on such a classif ier.
The disparity image contains noise that makes it hard to reconstruct 3D models form the image, one could investigate further how one would f ix the noise problem.
Implement underwater computer vision SLAM, if one is able to get disparity images with litt le noise.
One could create and test a DP anchor module based on the SLIC Superpixel LBP method, as it is robust to light and rotation. And the user could specify the object it wants to anchor to.
Make the SLIC Superpixel LBP method run faster using GPU multi-threading and CPU threading. And also vectorize more of the for loops in python.
Using disparity method to avoid obstacle
Achanta R, Shaji A, Smith K, et al . (2012) SLIC superpixels compared to state-of -the-art superpixel methods. IEEE t ransact ions on pat tern analysis and machine intel l igence 34(11): 2274?2282. Available f rom: ht tp:/ / dx.doi.org/10.1109/TPAMI.2012.120.
Machine Learning - Hands-On for Developers by Jason Bel l (Wiley, 2015).pdf (n.d.).
Ojala T, Piet ikäinen M and Mäenpää T (n.d.) Mul t iresolut ion Gray Scale and Rotat ion Invariant Texture Classif icat ion with Local Binary Pat terns.
Piet ikäinen M and Heikkilä J (n.d.) Image and Video Descript ion with Local Binary Pat tern Variants. Available f rom: ht tp:/ /www.ee.oulu.f i / research/ imag/mvg/ f i les/pdf /CVPR-tutorial -f inal .pdf .
Rodriguez-Teiles FG, Geovani Rodriguez-Teiles F, Ricardo P-A, et al . (2014) Vision-based react ive autonomous navigat ion with obstacle avoidance: Towards a non-invasive and caut ious explorat ion of marine habitat . In: 2014 IEEE Internat ional Conference on Robot ics and Automat ion (ICRA). Available f rom: ht tp:/ / dx.doi.org/10.1109/ icra.2014.6907412.
References
SLIC Superpixel LBP obstacle avoidance
Uniform Local Binary Pat tern
-low computat ional
-high qual ity descript ion
-rotat ion invariant texture classif icat ion
- i l luminat ion invariant
Machine Learning
Linear Support Vector
Classif ication
Feature extract ion Classif ier deciderAcquire image Segmentation
Simple l inear i terat ive
clustering
Classify model
Compute the histogram of the LBP
Histogram computat ion
If (segment(classifer) == "other")
Calculate center of given object.
Calculate possible path away from object
Send calculated path directions using UDP to
path program
Mask image
The disparity methodPros
+ fast
+robust
+ possibil ity to create 3D
point clouds from data
Cons
-need calibration
-need 2 cameras( heavier
payload + more
expensive)
SLIC Superpixel LBP
Pros
+robust in dif ferent l ighting
+no need for camera
calibration
+ need only one camera
Cons
-slower
-no depth information
No obstacle case
Obstacle case
Top Related