From Control Model to Program:Investigating Robotic Aerial Vehicle Accidents with

MAYDAY

Taegyu Kim1, Chung Hwan Kim2, Altay Ozen1, Fan Fei1, Zhan Tu1, Xiangyu Zhang1, Xinyan Deng1, Dave (Jing) Tian1, Dongyan Xu1

1Purdue University 2UT Dallas

Drone (Robotic Aerial Vehicle) Accidents

RAV Control and Control-Semantic Bugs

SensorModule

MissionModule

Control ProgramObserved vehicle states in “6DoFs”

Physical Environment

Control Station

Control Model

Aerodynamics

Motor

Control-Semantic Bug

• Accident root cause inside control program

• Incorrect or incomplete implementation of control model

𝑧

𝑦

𝑥

𝑦𝑎𝑤

𝑟𝑜𝑙𝑙

𝑝𝑖𝑡𝑐ℎ

A Motivating Accident

Challenges in Investigating the Accident

• “Two Gaps”

• Domain gap• Control domain

• Time gap

• Attack time → Impact time

Domain Gap

Control ModelControl

Attack impact

Control ProgramProgram

Root Cause

• Our solution: MAYDAY

• Bridge the gaps

• Enable cross-domain investigation

→ Program domain

Reference VelocityActual Velocity

0

20

40

60

80

4800 4900 5000 5100 5200

Control Loop Iteration

Velo

city (

cm

/s)

Time Gap

AttackCMD

Control-levelLog

?

Impact

MAYDAY Workflow

Control Program(Source Code)

ProgramInstrumentation

ProgramAnalysis

Crash Log

Offline Analysis & InstrumentationRuntimeLogging

Post-Accident Investigation

Program-levelInvestigation

Control-levelInvestigation

Control Variable Dependency Graph

(CVDG)

Result

RAV Control Model

6DoF

Inter-dependency

between controllers

Cascading controller

x 4

X-axis Cascading Controller

Y-axis Cascading Controller

Z-axis Cascading Controller

Pitch Cascading Controller

Roll Cascading Controller

Motor Controller

Yaw Cascading Controller

PS

M

: Sensor Input

: Mission Input

: Parameter Input

ANGLEController

𝑥𝜓

ሶ𝑥𝜓

ሷ𝑥𝜓

𝑟𝜓

ሶ𝑟𝜓

ሷ𝑟𝜓

𝑘𝜓

ሶ𝑘𝜓

ሷ𝑘𝜓

VELController

ACCELController

PS M

ሶ𝑥𝑥

ሷ𝑥𝑥

ሶ𝑟𝑥

ሷ𝑟𝑥

POSController

𝑥𝑥 𝑟𝑥 𝑘𝑥

ሶ𝑘𝑥

ሷ𝑘𝑥

VELController

ACCELController

PS M

ሶ𝑥𝑦

ሷ𝑥𝑦

ሶ𝑟𝑦

ሷ𝑟𝑦

POSController

𝑥𝑦 𝑟𝑦 𝑘𝑦

ሶ𝑘𝑦

ሷ𝑘𝑦

VELController

ACCELController

PS M

ANGLEController

𝑥𝜑 𝑟𝜑 𝑘𝜑

ሶ𝑥𝜑 ሶ𝑟𝜑 ሶ𝑘𝜑VEL

Controller

ሷ𝑥𝜑 ሷ𝑟𝜑 ሷ𝑘𝜑ACCEL

Controller

PS M

ANGLEController

𝑥𝜃 𝑟𝜃 𝑘𝜃

ሶ𝑥𝜃 ሶ𝑟𝜃 ሶ𝑘𝜃VEL

Controller

ሷ𝑥𝜃 ሷ𝑟𝜃 ሷ𝑘𝜃ACCEL

Controller

PS M

POSController

𝑥𝑧

ሶ𝑥𝑧

ሷ𝑥𝑧

𝑟𝑧

ሶ𝑟𝑧

ሷ𝑟𝑧

𝑘𝑧

ሶ𝑘𝑧

ሷ𝑘𝑧

VELController

ACCELController

PS M

𝜑 = 𝑎𝑡𝑎𝑛− ሷ𝑥𝑠𝑖𝑛𝜓 + ሷ𝑦𝑐𝑜𝑠𝜓

𝑔𝜃 = −𝑎𝑡𝑎𝑛

ሷ𝑥𝑐𝑜𝑠𝜓 + ሷ𝑦𝑠𝑖𝑛𝜓

𝑔

S

Control Variable Dependency Graph (CVDG)

Mapping Control Model to Control Program

void AC_PosControl::rate_to_accel_z(…vel_err.z = vel_target.zp = * vel_err.z;accel_target.z = accel_ff.z + p ;…

POSController

𝑥𝑧

ሶ𝑥𝑧

ሷ𝑥𝑧

𝑟𝑧

ሶ𝑟𝑧

ሷ𝑟𝑧

𝑘𝑧

ሶ𝑘𝑧

ሷ𝑘𝑧

VELController

ACCELController

PS M

• Control model variable → Control program variable

• Control model data flow

: Parameter : Vehicle state: Reference

- cur_vel.z

ሶ𝑥𝑧ሶ𝑟𝑧ሶ𝑘𝑧

Control Model

PS M: Sensor input : Mission input : Parameter input

→ Control program execution paths

Control Program

_p_velz._kP()

Mapping

Logging Enhancement

• Control/vehicle operation log

• Recorded by default• Supported by major drone control programs

• Recorded by control-level logging functions

• Program execution log

• Enabled by MAYDAY

• Logging functions inserted via LLVM-level instrumentation

• Guided by mapping between control model and program

If err.z -= cur.z;else err.z = 0.0;p = kP* err.z;

Control-Level Investigation

-200

0

200

400

600

800

1000

8000 15000 22000 29000 36000

X-a

xis

Velo

city Initial

Digression

Investigation

• Identify initial digressing controller

• [Controller, corrupted variable, initial digression time]

• Infer control-level corruption path based on CVDG

: Reference

: Actual state

InitialDigression

Control Loop Iteration

Moving from Control Domain to Program Domain

• Corrupted control variable →Corrupted program variable

InitialDigression

Investigation

-200

0

200

400

600

800

1000

8000 15000 22000 29000 36000

X-a

xis

Velo

city Initial

Digression

: Reference

: Actual state

Control Loop Iteration

Program-Level Investigation

Program-levelCorruption Path

InitialDigression

Investigation

-200

0

200

400

600

800

1000

8000 15000 22000 29000 36000

X-a

xis

Velo

city Initial

Digression

: Reference

: Actual state

Control Loop Iteration

AttackInput

Attack Input

• Control-level corruption path →

• From initial digression to attack input

• Bug localized in basic blocks that implement the corruption path

Program-level corruption path

Evaluation: Effectiveness of MAYDAY

Evaluation: Solving the Earlier Case

-200

0

200

400

600

800

1000

8000 15000 22000 29000 36000

X-a

xis

Velo

city

Control Loop Iteration

Initial Digression

AttackInput

Control-Level Log

Program-Level Log

• Initial digressing controller: X, Y-axis velocity controller• Corrupted control variable: X, Y-axis acceleration reference• Control-level corruption path:

• Attack input:Control gain kP

• Number of BBs on corruption path: 34

• Source LoC: 89

Evaluation: Runtime Overhead of MAYDAY

Conclusion

• Drone accident may be caused by control semantic bugs

• Control-level logs alone are not sufficient for bug-tracing

• MAYDAY: a cross-domain accident investigation tool• Bridging the domain gap and the time gap

• Mapping control model to control program

• Integrating control-level and program-level logging

• Connecting control-level and program-level investigation

Thank you!This work was supported in part by ONR Grant #N00014-17-1-2045.

tgkim@purdue.edu