Company Public – NXP, the NXP logo, and NXP secure connections for a smarter world are trademarks of NXP

B.V. All other product or service names are the property of their respective owners. © 2018 NXP B.V.

Drone Program Lead,

Systems Innovation Automotive

NXP Semiconductors

Iain Galloway

NXP Dronecode Platform for Developing

Low Cost Small Autonomous Vehicles

Leveraging Automotive, IoT and Other Technologies for Drones and Rovers

Dronecode Program Manager

The Linux Foundation

Ramon Roche

June 2018 | AMF-AUT-T3177

COMPANY PUBLIC 1

Industry Trends: Commercial Application Taking Off and

Industry Becoming Horizontally Integrated

2009 2012 Today 2021

$21B

DIY

Cargo,

industrial &

‘Flying Cars’Commercial

Consumer

Service

Pilot

ODM

Compon

ents

Marketplace / search

App App App

Mission planning / UTM

ODM ODM

Components Comp.

Fleet Fleet Fleet

Vertical Horizontal

Flight stack

COMPANY PUBLIC 2

What do typical “drone” devices

look like?Think “Small Autonomous Vehicles” Instead

COMPANY PUBLIC 3

Octocopter -Flat

Quadcopter

Octocopter - Coax Multicopter

Tricopter Hexacopter

COMPANY PUBLIC 4

AirshipVTOL transitioning Wing

Flying WingPlane VTOL

Pusher prop

Longer flight Duration

COMPANY PUBLIC 5

Submarine

Boat Security Lawnmower or Agricultural?

Delivery rover

COMPANY PUBLIC 6

Sidewalk

Delivery rover

Rover Delivery trials in cities today for food and package delivery

(DoorDash, Dominos, Tesco Grocery store, EAT24)

Fewer regulatory hurdles?

Cooperation with Drones?

Street Delivery

Rover

COMPANY PUBLIC 7

What are typical uses of drones?

COMPANY PUBLIC 8

Hobby, Racing

COMPANY PUBLIC 9

Commercial Photography,

Cinematography &

TV News

COMPANY PUBLIC 10

Police, Fire, and

Rescue (Medical)

• Search and rescue

• Situational awareness

• Safety – egress

• Medical delivery

COMPANY PUBLIC 11

Construction and Mining

Photogrammetry

COMPANY PUBLIC 12

Inspection

COMPANY PUBLIC 13

Inventory Management

• NXP RAIN UHF RFID Tags

• Indoor or outdoor logistics or inventory

management

• Augment existing fixed RAIN

installations

• Find open storage locations in

warehouses

• Outdoor - Inventory City property?

COMPANY PUBLIC 14

Anatomy of a Drone/UAV

COMPANY PUBLIC 15

Base Drone/UAV functions

One motor controller (ESC) per

motor

One flight controller (FMU) with

motion control (Accel/Gyro/Mag) and

at least one radio interface

(MCUs: NXP’s Kinetis, LPC, i.MX RT)

1

COMPANY PUBLIC 16

Advanced Drone/UAV function

Companion computer and communications for

Autonomy, Data Analysis and Vision systems

RADAR, LiDAR, V2X, GPS, UWB

Sense and avoid technology

(Linux/SafetyOS + ROS)

Advanced battery management,

wireless charging, security, power

distribution, payload, actuators…

2

COMPANY PUBLIC 17

Some Architectures

1. Basic hobby drone

2. Advanced hobby drone

3. Industrial small autonomous vehicle• Drone/UAV

• Delivery Rover

• Other (Commercial Lawnmower, agricultural product)

COMPANY PUBLIC 18

Hobby Drone (Basic)

FMU Flight

Management

UnitIMU

Sensors

LightingGimbal:

ESC/encoder/sensors

Battery

Power Management

ESCKinetis V

Series

ESCKinetis V

Series

ESCKinetis V

SeriesESC

RC Radio

GPS

(advanced option)

• Lots of wires

• Lots of point-to-point

connections

COMPANY PUBLIC 19

WIFI/BT

Hobby Drone (Advanced with Vision)

FMU Flight

Management

Unit

UART

SDIO/USB

MIPI

CSI

BLE/Thread

(KW41)

IMU

Sensors

PCIe

Lighting

Gimbal:

ESC/encoder/sensors

(Single chip with CAN)

Aux

IMU

sensors

SSD/M.2

HDD

Storage

(Video)

Companion Computer

Data Analytics

Video

Camera

Sensor(s)

FPV

Camera

Bolometer

(NXP Fab)

Battery

Power Management

Battery Cell Monitoring

ESCKinetis V

Series

ESCKinetis V

Series

ESCKinetis V

Series

ESCKinetis V

Series

RC Radio

GPS

• Companion computer and FMU may be combined

Vision

Subsystem

COMPANY PUBLIC 20

FMU Flight

Management

UnitKinetis K, V or L

WIFI/BT

Industrial-Grade Small AV – Modular with CAN and Ethernet

NFC NCF3320

(522)

I2S

UART

SDIO/USB2x CAN BUS (1 primary 1 aux)

MIPI

CSI

D2X

(radio only)

BLE/Thread

(KW41)

RADAR

S32R

IMU

Sensors

PCIe

Lighting

ADS-B

(Automatic Dependent

Surveillance –

Broadcast

(SDR chip)

Payload/manipulator/

medical devices

Gimbal:

ESC/encoder/sensors

(Single chip with CAN)

Aux IMU

sensors

CA

N-F

D?

USB

CAN

Digital

SERV

O

GPS

Cellular

Radio

SSD/M.2

HDD

Storage

(Video)

Audio Codec

100BaseT1 2-Wire

Automotive ABV

hub/Switch

POE/PODL LIDARSmart optical

flow Camera?

Smart Ethernet

Sensor/CameraSmart Ethernet

Sensor/CameraCamera,

Sensors

Smart

Companion Computer, Data Analytics,

Vision

S32V244 (A53),

i.MX 6Q MPU (Cortex-A9)

Layerscape MPU (Cortex-A53/72)

or

i.MX 8 series MPU (future)

(ARM Cortex-A53 + A72)

Secure

Element

Camera

Sensor(s)

FPV

Camera

Bolometer

(NXP Fab)

USB- C RMII

Mantra

Sub-GHz

Battery

Battery Authentication SecureID

Power Management

Wireless Charging System

RX

Battery cell monitoring

Authentication

Authentication

ESCKinetis V

Series

ESCKinetis V

Series

ESCKinetis V

Series

ESCKinetis V

Series

Authentication

RC Radio

GPS

D2X-IDU

FMU Flight

Management UnitKinetis K, V

or i.MX RTSecure

Element

Ethernet

COMPANY PUBLIC 21

Tomorrow:

Self-Driving

Robots with

End-to-End

Services

SENSE THINK ACT

COMPANY PUBLIC 22

WIFI/BT

Industrial-Grade UAV – Functional Areas

FMU Flight Management

UnitKinetis K, V or L

D2X (radio only)

BLE/Thread(KW41)

Secure Element

RADARS32R

LightingPayload/manipulator

/medical devices

Gimbal:ESC/encoder/sensors(Single chip with CAN)

Aux IMU sensors

CAN Digital SERVO

GPS

Cellular Radio

SSD/M.2 HDD Storage

(Video)

Audio Codec

Companion Computer, Data Analytics,

visionS32V,i.MX

Secure Element

MantraSub-GHz RC Radio

GPS

Connectivity

Vision, Sense and Avoid Powertrain and Vehicle Dynamics, Motor Control

Battery and Power Management

Authentication

Authentication

Authentication Authentication

In Vehicle networking

COMPANY PUBLIC 23

Hands-on Learning in Small VehiclesIntroducing a flexible program for new vehicle concepts

HoverGames DroneI NXPhlite Flight Management unit

I DroneCode PX4 Flight Stack

I Complete starter kit

Modular and flexible development platform

I New technologies - UAVCAN and 100BaseT1 capable

I Suitable for Rovers and other Robotic vehicle systems

I System test bed for adjacent technologies and products

HoverGames

COMPANY PUBLIC 24

WIFI/BT

Industrial Reference UAV

NFC NCF3320

(522)

I2S

FMU Flight

Management

UnitKinetis K, V or L

UART

SDIO/USB

2x CAN (1 primary 1 aux)

MIPI

CSI

D2X

(radio only)

BLE/Thread

(KW41)

Secure

Element

RADAR

S32R

IMU

Sensors

PCIe

Lighting

ADS-B

(Automatic Dependent

Surveillance –

Broadcast

(SDR chip)

Payload/manipulator/

medical devices

Gimbal:

ESC/encoder/sensors

(Single chip with CAN)

Aux IMU

sensors

CA

N-F

D?

USB

CAN

Digital

SERV

O

GPS

Cellular

Radio

SSD/M.2

HDD

Storage

(Video)

Audio Codec

100BaseT1 2-Wire

Automotive ABV

hub/Switch

POE/PODL LIDARSmart optical

flow Camera?

Smart Ethernet

Sensor/CameraSmart Ethernet

Sensor/CameraSmart Ethernet

Sensor/Camera

Companion Computer, Data Analytics,

Vision

S32V244 (A53),

i.MX 6Q MPU (Cortex-A9)

Layerscape MPU (Cortex-A53/72)

or

i.MX 8 series MPU (future)

(ARM Cortex-A53 + A72)

Secure

Element

Camera

Sensor(s)

FPV

Camera

Bolometer

(NXP Fab)

USB- C RMII

Mantra

Sub-GHz

Battery

Battery Authentication SecureID

Power Management

Wireless Charging System

RX

Battery cell monitoring

Authentication

Authentication

ESCKinetis V

Series

ESCKinetis V

Series

ESCKinetis V

Series

ESCKinetis V

Series

Authentication

RC Radio

GPS

D2X-IDU

COMPANY PUBLIC 25

Hovergames Platform – Starting Point

NFC

FMU Flight

Management

UnitKinetis K, V or L

UART

2x CAN (1 primary 1 aux)

D2X

BLE/Thread

(KW41)

IMU

Sensors

GPS

ESCKinetis V

Series

ESCKinetis V

Series

ESCKinetis V

Series

ESCKinetis V

Series

RC Radio

2wire Ethernet

COMPANY PUBLIC 26

NXPhliteV3RC15 FMU ComponentsVehicle Management Unit with Automotive-Grade Components

Onboard components for

• Legacy RC PWM

• USB debug terminal for

QGroundControl config.

• Automotive IMU

components

(accel/gyro/mag)

• Barometric pressure

• Dual CAN bus

• 2 wire 100BaseT1

Ethernet

Kinetis K66, Arm M4

2M Flash @ 180Mhz

FXAS21002CQ

Gyroscope

FXOS8700CQ

Accelerometer Compass

MPL3115A2 Barometer

Pressure Sensor

RC PWM

TJA1100 2-Wire Ethernet

USB

DUAL TJA1042 CAN Transceiver

TJA1100 2-wire Ethernet Phy

COMPANY PUBLIC 27

NXPhliteV3RC15 FMU ComponentsVehicle Management Unit with Automotive grade components

Onboard components for

• Legacy RC PWM

• USB debug terminal for

QGroundControl config.

• Automotive IMU

components

(accel/gyro/mag)

• Barometric pressure

• Dual CAN bus

• 2 wire 100BaseT1

Ethernet

Kinetis K66, Arm M4

2M Flash @ 180Mhz

FXAS21002CQ

Gyroscope

FXOS8700CQ

Accelerometer Compass

MPL3115A2 Barometer

Pressure Sensor

RC PWM

TJA1100 2-Wire Ethernet

USB

DUAL TJA1042 CAN Transceiver

TJA1100 2-wire Ethernet Phy

COMPANY PUBLIC 28

NXPhliteV3RC15 FMU ComponentsVehicle Management Unit with Automotive grade components

CAN0

USONIC

CAN1

PPM input

Arming

Switch

ADC6V

2 Wire ETH

POWER

GPS

UART

Telemetry

DCD-LZ

Debug

SBUS

NFC/I2C

G

A

E

P

Ports for external

• NTAG or other NFC

• Secure Authentication

• External radios such 802.11p D2X and

Kinetis KW41 BLE/Thread mesh

• USB/I2C/SPI/UARTS

• Ultrasonic transducer

• IrDA Infrared

• Arming switch+Buzzer

ADC3V

Buzzer

IRDA

SPI

28

COMPANY PUBLIC 29

HoverGames DroneSmaller than an autonomous car!

Reference Design

• Complete low cost ‘hobby’ drone

platform, but really an open design

robot.

• 500mm size big enough for easy

experimentation

• Complete system to test new

components (e.g. FOC ESC with

UAVCAN or secure battery)

• Reuse of components for Rover

29

COMPANY PUBLIC 30

COMPANY PUBLIC 31

Authentication

Secure authentication of components

Can you trust the drone if you can’t trust the

components?

• GPS spoofing

• Unapproved battery packs

• Modified components

• UL/FCC/TUV Certification

• Brand protection

Can certifying agencies protect their certifications?

Can you insure it without knowing it’s components?

COMPANY PUBLIC 32

HoverGames DroneMobile IoT

Integration with

Rapid IOT Platform

• DroneCode UAVCAN

connectors

• Onboard KW42 Radio -

IPV6, 6lowpan, Thread

radio. Bluetooth,

802.15.4 Zigbee

• A71CH Secure Element

• Onboard sensors

• 250+ Pluggable “Click”

sensor/actuator modules

32

COMPANY PUBLIC 3333

COMPANY PUBLIC 34

THE OPEN SOURCE UAV PLATFORM

A neutral place where industry and

community developers can work together

to build the world’s leading open UAV

software platform

COMPANY PUBLIC 35

Leading products and platforms

based on the Dronecode Platform

35 35

COMPANY PUBLIC 36

The Dronecode Platform

36

▪ 651 Contributors

▪ 6,140 Closed issues

▪ 10,094 Closed pull requests

Key platform stats (feb 18):

▪ 43,240 commits

▪ 118 Releases

CommunicationVehicle Ground Control Station

SDKMiddlewareAutonomy

Vision based

localization and

avoidance

Qualcomm

Snapdragon

Flight

Intel® Aero

Hardware

36

COMPANY PUBLIC 37

Proprietary

differentiation:

- Capabilities

- Customization

- Features

- Certification

- UX

How do companies use the platform?

2008 Today

Pla

tform

COMPANY PUBLIC 38

PX4 Pro Autopilot

PX4 is an open-source, open-hardware platform

aiming to become the safest and most reliable flight

stack to the commercial, industrial, and academic

markets.

• The core of a complete end-to-end UAV platform

• One code base for all vehicle types

• IP-friendly license (BSD License)

• World Class Devs

• The best from academia, industry, and agile

startups

• An active community supporting users and

developers across the flight stack.

38

COMPANY PUBLIC 39

QGroundControl: The opensource Ground Control Station

• State of the art software runs on Android, iOS, Windows, Linux and Mac OS.

• User friendly for flying and mission planning

• Survey, mapping support

• Digital video streaming support

• Enables firmware customization

• Simple airframe setup, including firmware, joysticks, sensors and flight modes.

39 39

COMPANY PUBLIC 40

QGroundControl

flight planner and

configuration tool

COMPANY PUBLIC 41

Demo in Techlab

DroneCode.org QGroundControl

optimized by Qt Company to run on a

Technexion TEP-15 industrial panel computer.

− NXP i.MX6 Quad processor

− Low power, rugged, fan-less, cost effective reference solution

− QGroundControl is an intuitive and powerful Ground Control Station for PX4 and ArduPilot UAVs

− Technexion TEP-15 industrial panel computer running Ubuntu or Yocto Linux

− The Qt Company optimized HMI & app

Technexion offers both SBCs SOMs and Panel computers using NXP i.MX family processors

Qt Company provides optimized solutions and consulting services for Qt framework

Linux Foundation is the Dronecode Open source Community supporting QGC, MAVLINK, and PX4

Auterion offers commercial go to market support for vehicles using Dronecode software stack

COMPANY PUBLIC 42

What’s Next

COMPANY PUBLIC 43

Maximum quality of components is

mandatory for high-value system

Looks at

intentional hazardsUnpredictable and irregular

Component Reliability

& Robustness

Looks at

unintentional hazardsPredictable and regular

Automotive Functional Safety and Security Is Critical

COMPANY PUBLIC 44

Industry Demands a Common MPU Architecture PlatformMaximizes hardware and software reuse across product and applications

Safety Radar Gateway Vision

Braking

Island

Arm Cortex

Interconnect

Memory

Radar

Island

Gateway

Island

Vision

Island

General

Purpose

Island

General Purpose Vision/ Autonomous

Fusion

Island

DebugSecurity

Safety

Common Architecture

Peripherals

Common Chassis

Arm Cortex

Interconnect

Memory

Common Chassis

Arm Cortex

Interconnect

Memory

Common Chassis

Arm Cortex

Interconnect

Memory

Common Chassis

Arm Cortex

Interconnect

Memory

Common Chassis

Arm Cortex

Interconnect

Memory

Common Chassis

One Common Architecture BaseFixed set of Cores and Interconnect

Consistent Safety Concept & Security

Identical Base Peripheral set

Consistent Boot and Debug

COMPANY PUBLIC 45

COMPANY PUBLIC 46

Complimented with Automotive-grade MCUs

Common Architecture

One Common Architecture BaseLIN, CAN and Ethernet end nodes

Sensors, interfaces and actuators

COMPANY PUBLIC 47

Medium Range

RADARBlind Side Detection

Short range/

Medium range

RADARPark Assist

Cross-Traffic Alert

Junction Assist

Long Range

RADARAdaptive Cruise Control

Automatic Emergency Braking

Forward Collision Warning

TOMORROW

Higher

Resolution

77GHz CMOS

RadarEvolves to 360° view with high performance integration

COMPANY PUBLIC 48

V2X IEEE 802.11p extends sensing

beyond your vehicle and line of sight (LOS)

Platooning / cooperative driving5m gap @ 80km/h based on 802.11p low latency

Roadworks beyond line-of-sight Emergency vehicle around corner

Motorcycle approaching / „do not pass!“

Low Latency, Secure

and

Beyond-line-of-sight

(LOS)

Providing additional safety

data earlier than any other

sensor can „see“

COMPANY PUBLIC 49

D2X CommunicationIEEE 802.11p

• Drone suitability performance evaluation

• Reliable secure medium range (2Km+?) digital communications for

• Command and control

• Digital Video

• Mesh networking, Drone to Drone

▪ Integration with DSRC/ITS-G5?

COMPANY PUBLIC 50

Next steps in the Drone program

• We are working with partners to enable:

• i.MX RT consumer market fit FMU

• S32K Industrial FMU

• Modular components− BLDC ESC motor controller (FOC,UAVCAN)

− Functional safety capable smart BMS

− Functional safety capable Power module (FS6500/4500)

− UAVCAN sensors and actuator

COMPANY PUBLIC 51

Tools and

Resources

• Local NXP Sales office and Distributors

• FTF-Connects and Tech Days

• Website: http://www.nxp.com/UAV

• HoverGames: www.nxp.com/hovergames

• Video: https://youtu.be/Sifn1p6fhVg

• Contact: iain.galloway@nxp.com or

uav.solutions@nxp.com

We want to hear from you!

If interested in NXP Drone reference design, send an email

and we will keep you informed.

Many additional topic areas and parts, lets talk!

NXP, the NXP logo, and NXP secure connections for a smarter world are trademarks of NXP B.V. All other product or service names are the property of their respective owners. © 2018 NXP B.V.

www.nxp.com