RF Silent Drone Navigation - UCSByoga/capstone/static/img/projects/slides/vishawk.pdfPerforms CV...
Transcript of RF Silent Drone Navigation - UCSByoga/capstone/static/img/projects/slides/vishawk.pdfPerforms CV...
![Page 1: RF Silent Drone Navigation - UCSByoga/capstone/static/img/projects/slides/vishawk.pdfPerforms CV tasks to identify the target and communicate target transform to FCU Side view. Raspberry](https://reader034.fdocuments.in/reader034/viewer/2022051807/60072a027b74140353550c06/html5/thumbnails/1.jpg)
RF Silent Drone Navigation
![Page 2: RF Silent Drone Navigation - UCSByoga/capstone/static/img/projects/slides/vishawk.pdfPerforms CV tasks to identify the target and communicate target transform to FCU Side view. Raspberry](https://reader034.fdocuments.in/reader034/viewer/2022051807/60072a027b74140353550c06/html5/thumbnails/2.jpg)
Matthew DupreeStudent Lead
OUR T
EAM
ANNA JAMES
Xihan LiuData Analysis
Yingchao ZhuPCB/Schematic
![Page 3: RF Silent Drone Navigation - UCSByoga/capstone/static/img/projects/slides/vishawk.pdfPerforms CV tasks to identify the target and communicate target transform to FCU Side view. Raspberry](https://reader034.fdocuments.in/reader034/viewer/2022051807/60072a027b74140353550c06/html5/thumbnails/3.jpg)
OVERVIEW
01
![Page 4: RF Silent Drone Navigation - UCSByoga/capstone/static/img/projects/slides/vishawk.pdfPerforms CV tasks to identify the target and communicate target transform to FCU Side view. Raspberry](https://reader034.fdocuments.in/reader034/viewer/2022051807/60072a027b74140353550c06/html5/thumbnails/4.jpg)
❏ Liftoff from a boat at sea❏ Land on a boat at sea
○ Limited-size landing area○ Moving landing area○ Miss == wet hardware
❏ RF-silent: No radio communication○ No pilot○ No landing beacon○ No external processing
➢ Computer vision is hard on an energy budget!
Over
view
![Page 5: RF Silent Drone Navigation - UCSByoga/capstone/static/img/projects/slides/vishawk.pdfPerforms CV tasks to identify the target and communicate target transform to FCU Side view. Raspberry](https://reader034.fdocuments.in/reader034/viewer/2022051807/60072a027b74140353550c06/html5/thumbnails/5.jpg)
Prob
lem
Form
aliza
tion
![Page 6: RF Silent Drone Navigation - UCSByoga/capstone/static/img/projects/slides/vishawk.pdfPerforms CV tasks to identify the target and communicate target transform to FCU Side view. Raspberry](https://reader034.fdocuments.in/reader034/viewer/2022051807/60072a027b74140353550c06/html5/thumbnails/6.jpg)
Bloc
k Diag
ram
Ethernet
Companion Computer:Raspberry Pi 3B+
Camera:Raspberry Pi Camera
Module V2
LiPo to 5VVoltage Regulator
Lipo Battery
Flight Controller:Pixhawk 2.4.8UART
GPS:Micro M8N GPS
Module
MotorsElectronic
Speed Controllers
Power Management Board
I2C
PWM
Para
llel
3-Phase
![Page 7: RF Silent Drone Navigation - UCSByoga/capstone/static/img/projects/slides/vishawk.pdfPerforms CV tasks to identify the target and communicate target transform to FCU Side view. Raspberry](https://reader034.fdocuments.in/reader034/viewer/2022051807/60072a027b74140353550c06/html5/thumbnails/7.jpg)
Bloc
k Diag
ram
Ethernet
Companion Computer:Raspberry Pi 3B+
Flight Controller:Pixhawk 2.4.8UART
Camera:Raspberry Pi Camera
Module V2
Para
llel
Lipo Battery
Power Management Board
LiPo to 5VVoltage Regulator
Servo
GPS:Micro M8N GPS
Module
I2C
MotorsElectronic
Speed Controllers
PWM
3-Phase
![Page 8: RF Silent Drone Navigation - UCSByoga/capstone/static/img/projects/slides/vishawk.pdfPerforms CV tasks to identify the target and communicate target transform to FCU Side view. Raspberry](https://reader034.fdocuments.in/reader034/viewer/2022051807/60072a027b74140353550c06/html5/thumbnails/8.jpg)
Hardware02
![Page 9: RF Silent Drone Navigation - UCSByoga/capstone/static/img/projects/slides/vishawk.pdfPerforms CV tasks to identify the target and communicate target transform to FCU Side view. Raspberry](https://reader034.fdocuments.in/reader034/viewer/2022051807/60072a027b74140353550c06/html5/thumbnails/9.jpg)
Three-layer design❏ Top: GPS, Flight Controller,
Telemetry radio❏ Middle: Companion Computer,
Camera❏ Bottom: 4s Lipo Battery
![Page 10: RF Silent Drone Navigation - UCSByoga/capstone/static/img/projects/slides/vishawk.pdfPerforms CV tasks to identify the target and communicate target transform to FCU Side view. Raspberry](https://reader034.fdocuments.in/reader034/viewer/2022051807/60072a027b74140353550c06/html5/thumbnails/10.jpg)
Hard
ware
Pixhawk v2.4.8❏ Flight control unit (FCU)❏ Uses ArduPilot firmware❏ Ensures drone stability
in flightTop view
![Page 11: RF Silent Drone Navigation - UCSByoga/capstone/static/img/projects/slides/vishawk.pdfPerforms CV tasks to identify the target and communicate target transform to FCU Side view. Raspberry](https://reader034.fdocuments.in/reader034/viewer/2022051807/60072a027b74140353550c06/html5/thumbnails/11.jpg)
Hard
ware
uBlox M8N Micro GPS❏ GPS unit❏ Provides drone lat/long
coordinate reference❏ Used until landing target
detected
Top view
![Page 12: RF Silent Drone Navigation - UCSByoga/capstone/static/img/projects/slides/vishawk.pdfPerforms CV tasks to identify the target and communicate target transform to FCU Side view. Raspberry](https://reader034.fdocuments.in/reader034/viewer/2022051807/60072a027b74140353550c06/html5/thumbnails/12.jpg)
Hard
ware
Transceiver Telemetry Radio ❏ Broadcasts debugging
information to GCS❏ Used for our testing -- not
required for flightTop view
![Page 13: RF Silent Drone Navigation - UCSByoga/capstone/static/img/projects/slides/vishawk.pdfPerforms CV tasks to identify the target and communicate target transform to FCU Side view. Raspberry](https://reader034.fdocuments.in/reader034/viewer/2022051807/60072a027b74140353550c06/html5/thumbnails/13.jpg)
❏ Middle: Companion Computer,Camera
![Page 14: RF Silent Drone Navigation - UCSByoga/capstone/static/img/projects/slides/vishawk.pdfPerforms CV tasks to identify the target and communicate target transform to FCU Side view. Raspberry](https://reader034.fdocuments.in/reader034/viewer/2022051807/60072a027b74140353550c06/html5/thumbnails/14.jpg)
Hard
ware
Raspberry Pi 3B+❏ Companion computer❏ Performs CV tasks to
identify the target and communicate target transform to FCU
Side view
![Page 15: RF Silent Drone Navigation - UCSByoga/capstone/static/img/projects/slides/vishawk.pdfPerforms CV tasks to identify the target and communicate target transform to FCU Side view. Raspberry](https://reader034.fdocuments.in/reader034/viewer/2022051807/60072a027b74140353550c06/html5/thumbnails/15.jpg)
Raspberry Pi Camera Module v2❏ 8 megapixel camera capable of
taking photographs of 3280 x 2464 pixels
❏ Used to detect AprilTags in 10FPS 640x480 mode
Hard
ware
Bottom view
![Page 16: RF Silent Drone Navigation - UCSByoga/capstone/static/img/projects/slides/vishawk.pdfPerforms CV tasks to identify the target and communicate target transform to FCU Side view. Raspberry](https://reader034.fdocuments.in/reader034/viewer/2022051807/60072a027b74140353550c06/html5/thumbnails/16.jpg)
❏ Bottom: 4s Lipo Battery
![Page 17: RF Silent Drone Navigation - UCSByoga/capstone/static/img/projects/slides/vishawk.pdfPerforms CV tasks to identify the target and communicate target transform to FCU Side view. Raspberry](https://reader034.fdocuments.in/reader034/viewer/2022051807/60072a027b74140353550c06/html5/thumbnails/17.jpg)
Printed Circuit board (PCB)
03
![Page 18: RF Silent Drone Navigation - UCSByoga/capstone/static/img/projects/slides/vishawk.pdfPerforms CV tasks to identify the target and communicate target transform to FCU Side view. Raspberry](https://reader034.fdocuments.in/reader034/viewer/2022051807/60072a027b74140353550c06/html5/thumbnails/18.jpg)
Sche
matic
❏ Function as a voltage regulator to convert the input voltage from 14-22V to 5V output voltage
❏ Diode after the input voltage in order to prevent reverse current
![Page 19: RF Silent Drone Navigation - UCSByoga/capstone/static/img/projects/slides/vishawk.pdfPerforms CV tasks to identify the target and communicate target transform to FCU Side view. Raspberry](https://reader034.fdocuments.in/reader034/viewer/2022051807/60072a027b74140353550c06/html5/thumbnails/19.jpg)
PCB b
oard
![Page 20: RF Silent Drone Navigation - UCSByoga/capstone/static/img/projects/slides/vishawk.pdfPerforms CV tasks to identify the target and communicate target transform to FCU Side view. Raspberry](https://reader034.fdocuments.in/reader034/viewer/2022051807/60072a027b74140353550c06/html5/thumbnails/20.jpg)
PCB B
oard
❏ 2-layer PCB❏ 45 * 59 mm❏ Holes for thermal
dissipation
![Page 21: RF Silent Drone Navigation - UCSByoga/capstone/static/img/projects/slides/vishawk.pdfPerforms CV tasks to identify the target and communicate target transform to FCU Side view. Raspberry](https://reader034.fdocuments.in/reader034/viewer/2022051807/60072a027b74140353550c06/html5/thumbnails/21.jpg)
Software
04
![Page 22: RF Silent Drone Navigation - UCSByoga/capstone/static/img/projects/slides/vishawk.pdfPerforms CV tasks to identify the target and communicate target transform to FCU Side view. Raspberry](https://reader034.fdocuments.in/reader034/viewer/2022051807/60072a027b74140353550c06/html5/thumbnails/22.jpg)
Land
ing T
arge
t
AprilTag 3❏ Landing Detection Target❏ Works on resource-constrained
platforms (like our RPi 3B+)❏ Full transforms from single stills!
![Page 23: RF Silent Drone Navigation - UCSByoga/capstone/static/img/projects/slides/vishawk.pdfPerforms CV tasks to identify the target and communicate target transform to FCU Side view. Raspberry](https://reader034.fdocuments.in/reader034/viewer/2022051807/60072a027b74140353550c06/html5/thumbnails/23.jpg)
StatsWith current onboard processing, we have tested that at 5 meter altitude the craft can search at 10FPS in a 5x6.7 meter box below the vehicle for a 16.5cm square tag.
Tag I
mage
s
Could do far better with more processing power!
(Example photo taken at ~2m alt. and has been cropped.)
![Page 24: RF Silent Drone Navigation - UCSByoga/capstone/static/img/projects/slides/vishawk.pdfPerforms CV tasks to identify the target and communicate target transform to FCU Side view. Raspberry](https://reader034.fdocuments.in/reader034/viewer/2022051807/60072a027b74140353550c06/html5/thumbnails/24.jpg)
First-choice:
PX4 Autopilot
❏ Great droning OS❏ Great debug tools
(eg MAVLink Shell)
❏ Doesn’t support our precision landing use!
Old but gold:
ArduPilot Multicopter
❏ Heavily used❏ Many configurations❏ Advanced features
❏ Spaghetti-code❏ Difficult configuration❏ Missing debugging shells
FCU F
irmw
are
![Page 25: RF Silent Drone Navigation - UCSByoga/capstone/static/img/projects/slides/vishawk.pdfPerforms CV tasks to identify the target and communicate target transform to FCU Side view. Raspberry](https://reader034.fdocuments.in/reader034/viewer/2022051807/60072a027b74140353550c06/html5/thumbnails/25.jpg)
❏ All open-source parts❏ Free to use❏ Spreads tasks among
processes to make best use of hardware
❏ Lots of logging tools!
Comp
anio
n OS
Ubiquity Robotics’Ubuntu Mate 18.04With ROS Kinetic
![Page 26: RF Silent Drone Navigation - UCSByoga/capstone/static/img/projects/slides/vishawk.pdfPerforms CV tasks to identify the target and communicate target transform to FCU Side view. Raspberry](https://reader034.fdocuments.in/reader034/viewer/2022051807/60072a027b74140353550c06/html5/thumbnails/26.jpg)
Processing Pipeline❏ A large stack of software
packages from the Robot Operating System (ROS) ecosystem
❏ High-throughput communication between nodes via ROS topics
❏ Relies on many complex configuration files
Pipe
line
LoggingTarget
Positions
MAVROSPixHawk COTS
FCU
Tag Positions
Vision_to_MAVROS
Images
Apriltag_ROS
Raspberry Pi Camera Module
V_sub.py(Our custom visualizer)
Logging
LegendHardware
ROS Node
DataDebugging Software
RasPiCam_node
MAVExplorer
Logging
Packages in bold we had to largely rewrite or write entirely ourselves!
![Page 27: RF Silent Drone Navigation - UCSByoga/capstone/static/img/projects/slides/vishawk.pdfPerforms CV tasks to identify the target and communicate target transform to FCU Side view. Raspberry](https://reader034.fdocuments.in/reader034/viewer/2022051807/60072a027b74140353550c06/html5/thumbnails/27.jpg)
SSH Pipe
Position Rotation
Data
EXTR
ACTIO
N ROS topicecho
ROS/MAVLink/UART Flight Controller
MAVLink Shell
Shell Pipe
YAML Converter
v_sub.py
Location Data ❏ Drone location found
relative to the camera position
❏ Data points are converted to angle and distance and North-East-Down
❏ For debugging, our v_sub.py plots the streaming data onto a Matplotlib 3D plot
CompanionComputer
![Page 28: RF Silent Drone Navigation - UCSByoga/capstone/static/img/projects/slides/vishawk.pdfPerforms CV tasks to identify the target and communicate target transform to FCU Side view. Raspberry](https://reader034.fdocuments.in/reader034/viewer/2022051807/60072a027b74140353550c06/html5/thumbnails/28.jpg)
Demo05
![Page 29: RF Silent Drone Navigation - UCSByoga/capstone/static/img/projects/slides/vishawk.pdfPerforms CV tasks to identify the target and communicate target transform to FCU Side view. Raspberry](https://reader034.fdocuments.in/reader034/viewer/2022051807/60072a027b74140353550c06/html5/thumbnails/29.jpg)
![Page 30: RF Silent Drone Navigation - UCSByoga/capstone/static/img/projects/slides/vishawk.pdfPerforms CV tasks to identify the target and communicate target transform to FCU Side view. Raspberry](https://reader034.fdocuments.in/reader034/viewer/2022051807/60072a027b74140353550c06/html5/thumbnails/30.jpg)
Ackn
owle
dgem
ents
Special Thanks to:
Navsea, Project Sponsor
Alan Jaeger, Navsea Representative
Dr. Yogananda Isukapalli, CE Capstone Project Instructor
Aditya Wadaskar, TA
Kyle Douglas, TA
![Page 31: RF Silent Drone Navigation - UCSByoga/capstone/static/img/projects/slides/vishawk.pdfPerforms CV tasks to identify the target and communicate target transform to FCU Side view. Raspberry](https://reader034.fdocuments.in/reader034/viewer/2022051807/60072a027b74140353550c06/html5/thumbnails/31.jpg)
THANKS!
![Page 32: RF Silent Drone Navigation - UCSByoga/capstone/static/img/projects/slides/vishawk.pdfPerforms CV tasks to identify the target and communicate target transform to FCU Side view. Raspberry](https://reader034.fdocuments.in/reader034/viewer/2022051807/60072a027b74140353550c06/html5/thumbnails/32.jpg)
Questions?