Scenario

– Bridges are critical components of modern transportation infrastructure.

– However, bridges deteriorate over time due to natural and human factors.

– Therefore, bridge deck inspection is in great demand nowadays.

In-traffic bridge deck scanning device

– Multi-modal

• Camera, LIDAR, and Ground Penetrating Radar (GPR)

– Approach

• Using LIDAR and camera to detect bridge deck surface

• Using GPR to obtain bridge deck subsurface structure

• Fusing all data to reconstruct the bridge deck in 3D structure

Introduction Multi-modal Calibration

Publications

CSE Department, Texas A&M University, USA

PI: Dezhen Song, NRI-1426752

NRI: Collaborative Research: Minimally Invasive Robotic Non-Destructive

Evaluation and Rehabilitation for Bridge Decks (Bridge-MINDER)

*This work was supported in part by National Science Foundation NRI-1426752 and Texas Department of Transportation (TxDot) 0-6869.

Testbed Development

Ground penetrating radar

– GSSI SIR-3000

– Antenna freq 1.6 GHz

Camera

– 10 Megapixel CMOS

– Model DS-CFMT1000-H

LIDAR

– Hokuyo, model UST-20LX

Calibration object

– Metal balls

– Checkerboard

Calibration platform

– Artificial bridge

• PVC pipes

• Wood broad

System Design

Fusing all data to reconstruct the bridge deck in 3D structure

– Visual SLAM provides surface information

– GPR provides subsurface information

– Calibration allows us to register GPR readings with SLAM outcomes

In traffic SLAM

– Using motion vector and optical flow for SLAM

– Using homography for segmentation

– Using Lidar data for further verification

Calibration of a GPR

– Estimating hyperbolas from GPR images

– Recovering metal ball positions in {G}

– Estimating rigid body transformation

Calibration of a GPR and a camera

– Using metal balls to calibrate the GPR

– Using chessboard to calibrate the camera

– Finding the relationship between GPR and camera

Calibration of a LIDAR and a camera

– Using planar pattern

Bridge Crack Searching and Mapping

• Joseph Lee, Yan Lu, Yiliang Xu, Dezhen Song, Visual Programming for Mobile Robot Navigation Using High-level Landmarks, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Korea, Oct. 9-14, 2016

• Chieh Chou, Shu-Hao Yeh, Jingang Yi, Dezhen Song, Extrinsic Calibration of a Ground Penetrating Radar, IEEE Conference on Automation Science and Engineering (CASE), Fort Worth, TX, USA, August 21-24, 2016

• Yan Lu and Dezhen Song, Robust RGB-D Odometry Using Point and Line Features, IEEE International Conference on Computer Vision (ICCV), Santiago, Chile, Dec. 13-16, 2015

• Yan Lu and Dezhen Song, Robustness to Lighting Variations: An RGB-D Indoor Visual Odometry Using Line Segments, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany, September 28 - October 02, 2015

• Wen Li and Dezhen Song, Featureless Motion Vector-based Simultaneous Localization, Planar Surface Extraction, and Moving Obstacle Tracking, The Eleventh International Workshop on the Algorithmic Foundations of Robotics (WAFR), August 2014, İstanbul, Turkey

• Joseph Lee, Yan Lu, and Dezhen Song, Planar Building Facade Segmentation and Mapping Using Appearance and Geometric Constraints, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Chicago, US, Sept., 2014

• Wen Li and Dezhen Song, Toward Featureless Visual Navigation: Simultaneous Localization and Planar Surface Extraction Using Motion Vectors in Video Streams, IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China, May-June, 2014

• Yan Lu, Dezhen Song, and Jingang Yi, High Level Landmark-Based Visual Navigation Using Unsupervised Geometric Constraints in Local Bundle Adjustment, IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China, May-June, 2014

• Yan Lu, Dezhen Song, Haifeng Li, and Jingtai Liu, Automatic Recognition of Spurious Surface in Building Exterior Survey, IEEE International Conference on Automation Science and Engineering Madison (CASE), Wisconsin, USA, August 17-21, 2013

• Yan Lu, Dezhen Song, Yiliang Xu, A. G. Amitha Perera, and Sang Min Oh, Automatic Building Exterior Mapping Using Multilayer Feature Graphs, IEEE International Conference on Automation Science and Engineering Madison (CASE), Wisconsin, USA, August 17-21, 2013

• Haifeng Li, Dezhen Song, Yan Lu, and Jingtai Liu, A Two-View based Multilayer Feature Graph for Robot Navigation, IEEE International Conference on Robotics and Automation (ICRA), May 14-18, 2012, St. Paul, MN

Current Project TeamDezhen Song Concept Design and Project Director

Chieh(Jay) Chou System Design and Website Construction

Shu-Hao(Eric) Yeh Algorithm Design and Implementation

Binbin Li Algorithm Design and Implementation

Jinhao Chen Software Developer

Hojun Ji Software Developer

Aaron Kingery Software Developer

• Motion Vector based SLAM

• Planar facade segmentation

• Multimodal data fusion and crack detection (Current work)