Autonomous UAV Development and Evaluation with MATLAB …

0

Autonomous UAV Development and Evaluation

with MATLAB® and Simulink®

Julia Antoniou Andrew Grabowski

11


MATLAB® Simulink®

▪ Integrated workflows enabled

by MATLAB and Simulink

▪ Tools to design UAV systems

and autonomous applications

▪ Select appropriate methods for

your UAV development tasks

▪ Evaluating systems through

closed-loop simulations with

sensor models

22


MATLAB® Simulink®

▪ Integrated workflows enabled

by MATLAB and Simulink

▪ Tools to design UAV systems

and autonomous applications

▪ Select appropriate methods for

your UAV development tasks

▪ Evaluating systems through

closed-loop simulations with

sensor models

33

Increase in autonomous UAV usage

Mapping &

Surveying

Inspections &

Monitoring

Delivery &

Transport

Security &

Defense

44

Challenges in developing autonomous UAV systems & applications

Complexity of advanced

autonomous algorithms

Need of end-to-end workflows

Ensuring system quality and

reducing flight risk

Which of these challenges have you encountered?

Enter in chat if you have others. Also enter in chat

if you have tips on how you have resolved these

challenges.

55

Solutions for developing autonomous UAV systems & applications

Robust tools and features for

designing and testing UAV systems

and algorithms

Integrated development environment

that covers development from ideas

to production

Extensive verification and validation

tools to evaluate design quality

through virtual testing

66

Integrated workflows for developing UAV applications

MATLAB® Simulink®

System Architecture

77


MATLAB®

Model UAV

Simulink®

System Architecture

88


MATLAB®

Model UAV

Simulink®

System Architecture

Design Algorithms

PerceptionPlanning &

DecisionControl

99


MATLAB®

Model UAV

Simulink®

Ve

rific

atio

n &

Va

lida

tio

n

DO-178

System Architecture

Design Algorithms


DecisionControl

1010


MATLAB®

Model UAV

Simulink®

Ve

rific

atio

n &

Va

lida

tio

n

DO-178

System Architecture

Unreal Engine

Simulate with Sensor Models

GazeboCuboid

Design Algorithms


DecisionControl

1111


Deploy to Hardware

MATLAB®

Model UAV

Implement

Simulink®

Ve

rific

atio

n &

Va

lida

tio

n

DO-178

System Architecture

Unreal Engine


GazeboCuboid

PX4® NVIDIA® Jetson®

Design Algorithms


DecisionControl

1212


Connect

Deploy to Hardware

MATLAB®

Model UAV

Implement

Connect

Simulink®

Ve

rific

atio

n &

Va

lida

tio

n

DO-178

System Architecture

Ground Control StationUAV

Unreal Engine


GazeboCuboid


Design Algorithms


DecisionControl

1313


Connect

Deploy to Hardware

MATLAB®

Model UAV

Implement

Connect

Simulink®

Ve

rific

atio

n &

Va

lida

tio

n

DO-178

System Architecture

Analyze Data


Unreal Engine


GazeboCuboid


Design Algorithms


DecisionControl

1414


Connect

Deploy to Hardware

MATLAB®

Model UAV

Implement

Connect

Simulink®

Ve

rific

atio

n &

Va

lida

tio

n

Analyze Data


Unreal Engine


GazeboCuboid


Design Algorithms


DecisionControl

DO-178

System Architecture

▪ System Composer™ for

designing and analyzing

system and software

architecture

▪ Simulink integration and

requirement allocation for

traceability

Link

https://www.mathworks.com/products/system-composer.html

1515


Connect

Deploy to Hardware

MATLAB®

Model UAV

Implement

Connect

Simulink®

Ve

rific

atio

n &

Va

lida

tio

n

Analyze Data


Unreal Engine


GazeboCuboid


Design Algorithms


DecisionControl

DO-178

System Architecture

1616

UAV Plant Modeling: Selecting the appropriate fidelity

High-Fidelity

Building UAV

Approximate

Programming UAV

Simscape Multibody, Aerospace Blockset, UAV Toolbox

1717


High-Fidelity

Building UAV

Approximate

Programming UAV


Link

https://www.mathworks.com/help/physmod/simscape/physical-modeling.html

1818


High-Fidelity

Building UAV

Approximate

Programming UAV


Link Link


https://www.mathworks.com/help/aeroblks/equations-of-motion-1.html

1919


High-Fidelity

Building UAV

Approximate

Programming UAV


Link LinkLink


https://www.mathworks.com/help/uav/ref/guidancemodel.html

https://www.mathworks.com/help/aeroblks/equations-of-motion-1.html

2020


Connect

Deploy to Hardware

MATLAB®

Model UAV

Implement

Connect

Simulink®

Ve

rific

atio

n &

Va

lida

tio

n

Analyze Data


Unreal Engine


GazeboCuboid


Design Algorithms


DecisionControl

DO-178

System Architecture

2121

Autonomous UAV Algorithm Development

Map Data

Endpoint

Startpoint

Planned Path

Design Algorithms


DecisionControl

2222

Autonomous UAV algorithm design with robust capabilities


DecisionControl

Se

lf

Aw

are

ne

ss

Sit

uati

on

al

Aw

are

ne

ss

Sensor Fusion and Tracking Toolbox, Lidar Toolbox, Navigation Toolbox, Computer Vision Toolbox, Deep Learning Toolbox

Link Link Link

Link LinkLink

https://www.mathworks.com/help/deeplearning/ug/pedestrian-detection.html

https://www.mathworks.com/help/lidar/ug/aerial-lidar-segmentation-using-pointnet-network.html

https://www.mathworks.com/help/lidar/ug/lidar-camera-calibration.html

https://www.mathworks.com/help/fusion/ug/imu-and-gps-fusion-for-inertial-navigation.html

https://www.mathworks.com/help/nav/ug/perform-lidar-slam-using-3d-lidar-point-clouds.html

https://www.mathworks.com/help/lidar/ug/aerial-lidar-slam-using-fpfh-descriptors.html

2323



DecisionControl

Define UAV missions with waypoint

and trajectory-following algorithms

UAV motion planning with

advanced path planners

UAV Toolbox, Navigation Toolbox

Link Link

https://www.mathworks.com/help/uav/planning-and-control.html

https://www.mathworks.com/help/uav/ug/motion-planning-with-rrt-for-fixed-wing-uav.html

2424



DecisionControl

Trajectory tracking controller with nonlinear

model predictive control (MPC)

Train policies for trajectory generation

using reinforcement learning algorithms

Model Predictive Control Toolbox, Reinforcement Learning Toolbox

Link Link

https://www.mathworks.com/help/mpc/ug/control-of-quadrotor-using-nonlinear-model-predictive-control.html

https://www.mathworks.com/help/reinforcement-learning/index.html

2525


Connect

Deploy to Hardware

MATLAB®

Model UAV

Implement

Connect

Simulink®

Ve

rific

atio

n &

Va

lid

atio

n

Analyze Data


Unreal Engine


GazeboCuboid


Design Algorithms


DecisionControl

DO-178

System Architecture

2626

Tracking and automating verification and validation activities

Requirements Traceability Test Management & Automation Evaluate Completeness

Simulink Requirements, Simulink Test, Simulink Coverage, Simulink Check

Link Link Link

https://www.mathworks.com/products/simulink-requirements.html

https://www.mathworks.com/products/simulink-test.html

https://www.mathworks.com/products/simulink-coverage.html

2727

Example: Automating UAV testing with requirements linking

Requirements linking for traceability

Automating test execution and evaluation

UAV Toolbox, Simulink Requirements, Simulink TestLink

https://www.mathworks.com/help/uav/ug/automate-testing-for-uav-package-delivery.html

2929


Connect

Deploy to Hardware

MATLAB®

Model UAV

Implement

Connect

Simulink®

Ve

rific

atio

n &

Va

lida

tio

n

Analyze Data


Unreal Engine


GazeboCuboid


Design Algorithms


DecisionControl

DO-178

System Architecture

3030

Integrated simulations with sensor models

Cuboid

Performance

Unreal Engine®

Photorealistic

What sensors are important to you? What sensor

characteristics are needed in simulation to proceed

to flight testing with confidence?

Rapidly author scenarios and

generate sensor data

Realistic graphics to test autonomous

algorithms in closed-loop simulations

UAV ToolboxLink Link

https://www.mathworks.com/help/uav/ref/uavscenario.html

https://www.mathworks.com/help/uav/ug/uav-package-delivery.html

3131


Cuboid

Performance

Unreal Engine®

Photorealistic











3232


Cuboid

Performance

Unreal Engine®

Photorealistic











3333

Example: Build 3D map using simulation Lidar point cloud data

Extract and match features

Register and align point cloud

Detect loop-closures

Create pose graph

Optimize poses

Execute simulation

Obtain sensor data

UAV Toolbox, Lidar Toolbox

3434

Create 3D scenes for UAV simulations

Design 3D scenes for simulating and testing autonomous algorithms

RoadRunner, RoadRunner Asset Library, UAV Toolbox Interface for Unreal Engine Projects

Link Link

https://www.mathworks.com/products/roadrunner.html

https://www.mathworks.com/help/uav/ug/customize-3d-scenes-for-automated-driving.html

3535

Automatic code generation for hardware implementation

Connect

MATLAB®

Model UAV

Implement

Connect

Simulink®

Ve

rific

atio

n &

Va

lida

tio

n

Analyze Data


Unreal Engine


GazeboCuboid

Design Algorithms


DecisionControl

DO-178

System Architecture

Deploy to Hardware


UAV Toolbox, Simulink Coder, Embedded Coder, GPU Coder

Deploy flight controls to

autopilot hardware

Deploy autonomous

algorithms to onboard

computersFPGA

CPU GPU

ROS

Link

Link

https://www.mathworks.com/hardware-support/px4-autopilots.html

https://www.mathworks.com/hardware-support/nvidia-jetson-cpu.html

3636

Connecting to UAV hardware through MAVLink protocol

Deploy to Hardware

MATLAB®

Model UAV

Implement

Connect

Simulink®

Ve

rific

atio

n &

Va

lida

tio

n

Analyze Data

Unreal Engine


GazeboCuboid


Design Algorithms


DecisionControl

DO-178

System Architecture

Connect


Message Blocks

Autopilot

MAVLink

Host Machine

MAVLink

Compute Board

MAVLink

UAV

Remote UAV

MAVLink Connectivity

UAV Toolbox

Link

https://www.mathworks.com/help/uav/ug/tune-uav-parameters-using-mavlink.html

3737

Post-flight data analysis

Connect

Deploy to Hardware

MATLAB®

Model UAV

Implement

Connect

Simulink®

Ve

rific

atio

n &

Va

lida

tio

n


Unreal Engine


GazeboCuboid


Design Algorithms


DecisionControl

DO-178

System ArchitectureFlight Log Analysis Payload Data Analysis

Analyze Data

UAV Toolbox, Computer Vision Toolbox, Deep Learning Toolbox

Link Link

Link

https://www.mathworks.com/help/vision/ug/multispectral-semantic-segmentation-using-deep-learning.html

https://www.mathworks.com/help/vision/ug/uncalibrated-stereo-image-rectification.html

https://www.mathworks.com/help/uav/ref/flightloganalyzer-app.html

3939

Integrated Workflows for Developing UAV Applications

Connect

Deploy to Hardware

MATLAB®

Model UAV

Implement

Connect

Simulink®

Ve

rific

atio

n &

Va

lida

tio

n

Analyze Data


Unreal Engine


GazeboCuboid


Design Algorithms


DecisionControl

DO-178

System Architecture

4040

Key Takeaways

Robust tools/features for

autonomous UAV design and

simulations with sensor models

Integrated development workflows

from prototyping to productization

with MATLAB and Simulink

Quality through verification & validation

tools for traceability, test completeness,

and test management/automation

Call To Action:

• Download presentation file

and investigate linked

examples and pages

• Contact us for to learn

more details or for trials

41

© 2021 The MathWorks, Inc. MATLAB and Simulink are registered trademarks of The MathWorks, Inc. See mathworks.com/trademarks

for a list of additional trademarks. Other product or brand names may be trademarks or registered trademarks of their respective holders.

Thank you

42

Q&A

Autonomous UAV Development and Evaluation with MATLAB …

Documents

Transcript of Autonomous UAV Development and Evaluation with MATLAB …