Autonomous Navigation using Model Based Design · 20 Localization System using Model Based Design...
Transcript of Autonomous Navigation using Model Based Design · 20 Localization System using Model Based Design...
![Page 1: Autonomous Navigation using Model Based Design · 20 Localization System using Model Based Design ROS as Communication ... and code generation, while using MATLAB to automate development](https://reader036.fdocuments.in/reader036/viewer/2022070923/5fbb66c1b953071aaa5ba37d/html5/thumbnails/1.jpg)
1© 2018 The MathWorks, Inc.
Autonomous Navigation using Model Based Design
MATLAB EXPO, San Jose
November 6, 2018
Pulkit Kapur
Industry Lead– Robotics and Autonomous Systems
Carlos Santacruz-Rosero
Application Lead– Robotics and Autonomous Systems
![Page 2: Autonomous Navigation using Model Based Design · 20 Localization System using Model Based Design ROS as Communication ... and code generation, while using MATLAB to automate development](https://reader036.fdocuments.in/reader036/viewer/2022070923/5fbb66c1b953071aaa5ba37d/html5/thumbnails/2.jpg)
2
Autonomous Systems Development using MATLAB and Simulink
Autonomous Ground Robots Manipulator Arms/Humanoids
Mapping the Environment Machine Learning and Controls
![Page 3: Autonomous Navigation using Model Based Design · 20 Localization System using Model Based Design ROS as Communication ... and code generation, while using MATLAB to automate development](https://reader036.fdocuments.in/reader036/viewer/2022070923/5fbb66c1b953071aaa5ba37d/html5/thumbnails/3.jpg)
3
Components of Autonomous Systems
PlanningLocalization
ControlPerception
![Page 4: Autonomous Navigation using Model Based Design · 20 Localization System using Model Based Design ROS as Communication ... and code generation, while using MATLAB to automate development](https://reader036.fdocuments.in/reader036/viewer/2022070923/5fbb66c1b953071aaa5ba37d/html5/thumbnails/4.jpg)
4
Key Takeaway of this Talk
Success in developing an autonomous robotics system requires:
1. Multi-domain simulation with newer technologies
2. Trusted tools which make complex workflows easy and
integrate with other tools
3. Model-based design provides the flexibility for changing
development needs
![Page 5: Autonomous Navigation using Model Based Design · 20 Localization System using Model Based Design ROS as Communication ... and code generation, while using MATLAB to automate development](https://reader036.fdocuments.in/reader036/viewer/2022070923/5fbb66c1b953071aaa5ba37d/html5/thumbnails/5.jpg)
5
Task: Autonomous Delivery Robot
![Page 6: Autonomous Navigation using Model Based Design · 20 Localization System using Model Based Design ROS as Communication ... and code generation, while using MATLAB to automate development](https://reader036.fdocuments.in/reader036/viewer/2022070923/5fbb66c1b953071aaa5ba37d/html5/thumbnails/6.jpg)
6
Autonomous Delivery Workflow
![Page 7: Autonomous Navigation using Model Based Design · 20 Localization System using Model Based Design ROS as Communication ... and code generation, while using MATLAB to automate development](https://reader036.fdocuments.in/reader036/viewer/2022070923/5fbb66c1b953071aaa5ba37d/html5/thumbnails/7.jpg)
7
C++ Code
Networking
Data Import and Playback
Code Generation
Localization
Mapping and Planning
Path Following
Code Generation
![Page 8: Autonomous Navigation using Model Based Design · 20 Localization System using Model Based Design ROS as Communication ... and code generation, while using MATLAB to automate development](https://reader036.fdocuments.in/reader036/viewer/2022070923/5fbb66c1b953071aaa5ba37d/html5/thumbnails/8.jpg)
8
MappingPlanning & Decision
making
Localization &
Control
Key Components for Enabling Full Autonomy
Navigation Stack – Key Components
Physical System + Environment
Sensing &
Perception
![Page 9: Autonomous Navigation using Model Based Design · 20 Localization System using Model Based Design ROS as Communication ... and code generation, while using MATLAB to automate development](https://reader036.fdocuments.in/reader036/viewer/2022070923/5fbb66c1b953071aaa5ba37d/html5/thumbnails/9.jpg)
9
Complex workflows made easy with MATLAB
obj1
obj3
obj2
obj4
Training data Preprocessing Feature Extraction Training
Classifier
% Detect regions
BW = createMask(videoFrame);
% Fill image regions
BW = imfill(BW,'holes');
% Get bounding boxes
stats = regionprops('table',BW,'BoundingBox','Area');
% Filter based on area size
targetIndex = stats.Area > 500;
% Get bounding boxes from detected regions
testFeatures(k,:) = extractHOGFeatures(Icr);
![Page 10: Autonomous Navigation using Model Based Design · 20 Localization System using Model Based Design ROS as Communication ... and code generation, while using MATLAB to automate development](https://reader036.fdocuments.in/reader036/viewer/2022070923/5fbb66c1b953071aaa5ba37d/html5/thumbnails/10.jpg)
10
Deep Learning with MATLAB
New App for
Ground Truth
Labeling
Label pixels
and regions for
semantic
segmentation
Caffe model
importer
LSTM (time series, text)
DAG Networks
Library of
pretrained
models
Multi-GPUs in
parallel
Optimized GPU
code
Training plots
NEW
PRODUCT:
GPU Coder-
Convert to
NVIDIA CUDA
code
Deploy / ShareTrain / PredictModelsData
“How do I label
my data?”
“How do I access
the latest models?”
“How do I make training
and prediction faster?”
“How do I deploy
my new model?”
![Page 11: Autonomous Navigation using Model Based Design · 20 Localization System using Model Based Design ROS as Communication ... and code generation, while using MATLAB to automate development](https://reader036.fdocuments.in/reader036/viewer/2022070923/5fbb66c1b953071aaa5ba37d/html5/thumbnails/11.jpg)
11
MappingPlanning & Decision
making
Localization &
Control
Key Components for Enabling Full Autonomy
Navigation Stack – Key Components
Physical System + Environment
Sensing &
Perception
![Page 12: Autonomous Navigation using Model Based Design · 20 Localization System using Model Based Design ROS as Communication ... and code generation, while using MATLAB to automate development](https://reader036.fdocuments.in/reader036/viewer/2022070923/5fbb66c1b953071aaa5ba37d/html5/thumbnails/12.jpg)
12
Simultaneous Localization and Mapping (SLAM)
Robotics System Toolbox™
![Page 13: Autonomous Navigation using Model Based Design · 20 Localization System using Model Based Design ROS as Communication ... and code generation, while using MATLAB to automate development](https://reader036.fdocuments.in/reader036/viewer/2022070923/5fbb66c1b953071aaa5ba37d/html5/thumbnails/13.jpg)
13
Lidar SLAM Components
Estimate robot motion locally
Recognize previously visited
places
Globally
re-align all
scans
Sensor
Specific
Sensor
Agnostic
Drifts over
time
“Loop closure
detection”
Front-End Back-EndSLAM
Estimate
Sensor
Data
![Page 14: Autonomous Navigation using Model Based Design · 20 Localization System using Model Based Design ROS as Communication ... and code generation, while using MATLAB to automate development](https://reader036.fdocuments.in/reader036/viewer/2022070923/5fbb66c1b953071aaa5ba37d/html5/thumbnails/14.jpg)
14
SLAM Map Builder App
▪ Build 2D map of
environment based on Lidar
and odometry data
▪ Modify loop closures and
incremental scan matches to
improve map quality
▪ Export the resulting
occupancy grid and use for
path planning
![Page 15: Autonomous Navigation using Model Based Design · 20 Localization System using Model Based Design ROS as Communication ... and code generation, while using MATLAB to automate development](https://reader036.fdocuments.in/reader036/viewer/2022070923/5fbb66c1b953071aaa5ba37d/html5/thumbnails/15.jpg)
15
Integrate your Own Sensors for Custom SLAM Implementation
▪ Use your own custom sensor
processing
▪ Re-use infrastructure for building and
optimizing maps
![Page 16: Autonomous Navigation using Model Based Design · 20 Localization System using Model Based Design ROS as Communication ... and code generation, while using MATLAB to automate development](https://reader036.fdocuments.in/reader036/viewer/2022070923/5fbb66c1b953071aaa5ba37d/html5/thumbnails/16.jpg)
16
MappingPlanning & Decision
making
Localization &
Control
Key Components for Enabling Full Autonomy
Navigation Stack – Key Components
Physical System + Environment
Sensing &
Perception
![Page 17: Autonomous Navigation using Model Based Design · 20 Localization System using Model Based Design ROS as Communication ... and code generation, while using MATLAB to automate development](https://reader036.fdocuments.in/reader036/viewer/2022070923/5fbb66c1b953071aaa5ba37d/html5/thumbnails/17.jpg)
17
Path Planning
▪ Optimization or search problem
▪ Map representation is needed
▪ Data structures like graphs, heaps, and
queues
▪ Post-processing is sometimes required:
smoothing
![Page 18: Autonomous Navigation using Model Based Design · 20 Localization System using Model Based Design ROS as Communication ... and code generation, while using MATLAB to automate development](https://reader036.fdocuments.in/reader036/viewer/2022070923/5fbb66c1b953071aaa5ba37d/html5/thumbnails/18.jpg)
18
Plan Paths for Nonholonomic Vehicles using RRT*
![Page 19: Autonomous Navigation using Model Based Design · 20 Localization System using Model Based Design ROS as Communication ... and code generation, while using MATLAB to automate development](https://reader036.fdocuments.in/reader036/viewer/2022070923/5fbb66c1b953071aaa5ba37d/html5/thumbnails/19.jpg)
19
MappingPlanning & Decision
making
Localization &
Control
Key Components for Enabling Full Autonomy
Navigation Stack – Key Components
Physical System + Environment
Sensing &
Perception
![Page 20: Autonomous Navigation using Model Based Design · 20 Localization System using Model Based Design ROS as Communication ... and code generation, while using MATLAB to automate development](https://reader036.fdocuments.in/reader036/viewer/2022070923/5fbb66c1b953071aaa5ba37d/html5/thumbnails/20.jpg)
20
Localization System using Model Based Design
ROS as CommunicationFramework
![Page 21: Autonomous Navigation using Model Based Design · 20 Localization System using Model Based Design ROS as Communication ... and code generation, while using MATLAB to automate development](https://reader036.fdocuments.in/reader036/viewer/2022070923/5fbb66c1b953071aaa5ba37d/html5/thumbnails/21.jpg)
21
Control - Path Following
[𝒙𝒂 𝒚𝒂 𝜽𝒂]
[𝒙𝒃 𝒚𝒃 𝜽𝒃]
Path PlannerMap Path
Final Pose
Initial Pose Path FollowerSteering
Acceleration
![Page 22: Autonomous Navigation using Model Based Design · 20 Localization System using Model Based Design ROS as Communication ... and code generation, while using MATLAB to automate development](https://reader036.fdocuments.in/reader036/viewer/2022070923/5fbb66c1b953071aaa5ba37d/html5/thumbnails/22.jpg)
22
Controls: Path Following
▪ Control Problem (feedback)
▪ Map representation (most of the
time)
▪ Suited for graphical programming
▪ State machine for Supervisory
logic Angular velocity to go from current position to the lookahead
![Page 23: Autonomous Navigation using Model Based Design · 20 Localization System using Model Based Design ROS as Communication ... and code generation, while using MATLAB to automate development](https://reader036.fdocuments.in/reader036/viewer/2022070923/5fbb66c1b953071aaa5ba37d/html5/thumbnails/23.jpg)
23
Path Controller
Path Controller
𝑥 𝑡0 𝑦(𝑡0)
𝑥 𝑡1 𝑦(𝑡1)⋮
𝑥 𝑡𝑁 𝑦(𝑡𝑁)
Vehicle Model
We need a vehicle model to design and test our algorithm
𝑥 𝑡 𝑦(𝑡)
![Page 24: Autonomous Navigation using Model Based Design · 20 Localization System using Model Based Design ROS as Communication ... and code generation, while using MATLAB to automate development](https://reader036.fdocuments.in/reader036/viewer/2022070923/5fbb66c1b953071aaa5ba37d/html5/thumbnails/24.jpg)
24
Pure Pursuit in Action
![Page 25: Autonomous Navigation using Model Based Design · 20 Localization System using Model Based Design ROS as Communication ... and code generation, while using MATLAB to automate development](https://reader036.fdocuments.in/reader036/viewer/2022070923/5fbb66c1b953071aaa5ba37d/html5/thumbnails/25.jpg)
25
Complete System
![Page 26: Autonomous Navigation using Model Based Design · 20 Localization System using Model Based Design ROS as Communication ... and code generation, while using MATLAB to automate development](https://reader036.fdocuments.in/reader036/viewer/2022070923/5fbb66c1b953071aaa5ba37d/html5/thumbnails/26.jpg)
26
Table of Features for Autonomous Navigation
Algorithm Application Area MATLAB Implementation
SLAM Ground Robots, ADAS, UAVs
robotics.LidarSLAM
robotics.PoseGraph
robotics.PoseGraph3D
Localization All Autonomous Systems robotics.MonteCarloLocalization
Scan Matching ADAS, Ground RobotsmatchScans
matchScansGrid
Point Cloud Registration ADAS, Computer Vision
pcregrigid
pcregistericp
pcregisterndt
Estimation Filters All Autonomous SystemstrackingKF,trackingEKF,trackingUKF
robotics.ParticleFilter
Path Planning All Autonomous Systems
robotics.PRM
pathPlannerRRT(parkMap)
![Page 27: Autonomous Navigation using Model Based Design · 20 Localization System using Model Based Design ROS as Communication ... and code generation, while using MATLAB to automate development](https://reader036.fdocuments.in/reader036/viewer/2022070923/5fbb66c1b953071aaa5ba37d/html5/thumbnails/27.jpg)
27
Third-Party Simulator Integration
▪ Use co-simulation with third-party simulators for rich sensor and
environment simulation
Gazebo Unreal
![Page 28: Autonomous Navigation using Model Based Design · 20 Localization System using Model Based Design ROS as Communication ... and code generation, while using MATLAB to automate development](https://reader036.fdocuments.in/reader036/viewer/2022070923/5fbb66c1b953071aaa5ba37d/html5/thumbnails/28.jpg)
28
Clearpath Robotics Accelerates Algorithm Development for
Industrial Robots
ChallengeShorten development times for laser-based
perception, computer vision, fleet management, and
control algorithms used in industrial robots
SolutionUse MATLAB to analyze and visualize ROS data,
prototype algorithms, and apply the latest advances
in robotics research
Results▪ Data analysis time cut by up to 50%
▪ Customer communication improved
▪ Cutting-edge SDV algorithms quickly
incorporated
“ROS is good for robotics research and development, but not for
data analysis. MATLAB, on the other hand, is not only a data
analysis tool, it’s a data visualization and hardware interface tool
as well, so it’s an excellent complement to ROS in many ways.”
- Ilia Baranov, Clearpath Robotics
Link to user story
An OTTO self-driving vehicle from Clearpath Robotics.
![Page 29: Autonomous Navigation using Model Based Design · 20 Localization System using Model Based Design ROS as Communication ... and code generation, while using MATLAB to automate development](https://reader036.fdocuments.in/reader036/viewer/2022070923/5fbb66c1b953071aaa5ba37d/html5/thumbnails/29.jpg)
29
Voyage develops longitudinal controls
for self-driving taxis
ChallengeDevelop a controller for a self-driving car to follow a target velocity
and maintain a safe distance from obstacles
SolutionUse Simulink to design a longitudinal model predictive controller
and tuned parameters based on experimental data imported into
MATLAB using Robotics System Toolbox. Deploy the controller as
a ROS node using Robotics System Toolbox. Generate source
code using Simulink Coder into a Docker Container.
Results
▪ Development speed tripled
▪ Easy integration with open-source software
▪ Simulink algorithms delivered as production software
“We were searching for a prototyping solution that was
fast for development and robust for production. We
decided to go with Simulink for controller development
and code generation, while using MATLAB to automate
development tasks.”
- Alan Mond, Voyage
Voyage’s self driving car in San Jose, California.
![Page 30: Autonomous Navigation using Model Based Design · 20 Localization System using Model Based Design ROS as Communication ... and code generation, while using MATLAB to automate development](https://reader036.fdocuments.in/reader036/viewer/2022070923/5fbb66c1b953071aaa5ba37d/html5/thumbnails/30.jpg)
30
Key Takeaway of this Talk
Success in developing an autonomous robotics system requires:
1. Multi-domain simulation with newer technologies
2. Trusted tools which make complex workflows easy and
integrate with other tools
3. Model-based design provides the flexibility for changing
development needs
![Page 31: Autonomous Navigation using Model Based Design · 20 Localization System using Model Based Design ROS as Communication ... and code generation, while using MATLAB to automate development](https://reader036.fdocuments.in/reader036/viewer/2022070923/5fbb66c1b953071aaa5ba37d/html5/thumbnails/31.jpg)
31
% Thank you
mathworks.com/robotics
Come talk to us at the Autonomous
Navigation Booth!