Synthesis and Robotics

Hadas Kress-Gazit Sibley School of Mechanical and Aerospace Engineering

Cornell University

hadaskg@cornell.edu Joint work (this talk) with: Jim Jing, Ben Johnson, Cameron Finucane,

Nir Piterman and Vasu Raman

• “Drive around the city and obey traffic laws. Do not hurt anyone or anything”

• Teams of engineers

from industry and academia and…

Jalopnik.com TerraMax at DUC – from flickr (kwc)

DUC highlights

• “Go fetch me a beer”

• Week long hackathon at Willow Garage

Willow garage blog

Specifications (“What”)

Implementation (“How”)

Testing (“Works?”)

Control (probably works)

Now

Specifications (“What”)

Implementation (“How”)

Testing (“Works?”)

Control (works!)

In the (near) future

Automatic Correct-by-construction

synthesis

Jing, Finucane, Raman and KG, ICRA 2012

Continuous task (motion and action of a robot)

Sensing and Control for the robot

Automatic Provably correct

Synthesize provably correct solution

Discrete specification and model

Discrete Abstraction

Continuous implementation of solution

Continuous task (motion and action of a robot)

Sensing and Control for the robot

Synthesize provably correct solution

Discrete specification and model

Discrete Abstraction

Continuous implementation of solution

Continuous task (motion and action of a robot)

Sensing and Control for the robot

Reactive to the environment

Synthesize provably correct solution

Discrete specification and model

Discrete Abstraction

Continuous implementation of solution

Continuous task (motion and action of a robot)

Sensing and Control for the robot

Reactive to the environment

Sensor abstraction, atomic controllers

Synthesize provably correct solution

Discrete specification and model

Discrete Abstraction

Continuous implementation of solution

Continuous task (motion and action of a robot)

Sensing and Control for the robot

Reactive to the environment

Sensor abstraction, atomic controllers

Linear Temporal Logic, (Structured) language

Structured English Grammar

KG, Fainekos and Pappas, Advanced Robotics 2008, KG and Pappas ICRA 2010

Synthesize provably correct solution

Discrete specification and model

Discrete Abstraction

Continuous implementation of solution

Continuous task (motion and action of a robot)

Sensing and Control for the robot

Reactive to the environment

Sensor abstraction, atomic controllers

Linear Temporal Logic, (Structured) language

LTL synthesis, probabilistic model checking

Hybrid controller

(Partial) Related work

• Robot Control from Temporal Logic specifications: Kavraki and Vardi, Belta, Karaman and frazzoli, Fainekos, Wongpiromsarm Topcu and Murray,,…

• Symbolic control, languages and grammars for control: Egerstedt, Frazzoli and Dahleh, Klavins, Stilman, Tanner, …

• Hybrid systems • Formal methods – synthesis • Planning community

Human controlling a robot

Raman and KG: CAV 2011, ICRA 2012, IEEE TRO

Human controlling a robot

• Specification Unsatisfiable:

“Start in Deck. Always stay there. go to kitchen. ” • Specification Unrealizable:

“Start in Deck. If you see a person stay there. go to kitchen. ”

• Trivial solution:

“Always person and not person. ”

Raman and KG: CAV 2011, ICRA 2012, IEEE TRO

Human controlling a robot

Raman and KG: CAV 2011, ICRA 2012, IEEE TRO

Human controlling a robot

Raman and KG: CAV 2011, ICRA 2012, IEEE TRO

Correct Controller

Correct Behavior

High-Level Tasks

EnvironmentModel

Robot Abstraction

EnvironmentPerfect Sensors

Johnson and KG, RSS 2011, Johnson, Havlak, Campbell and KG, ICRA 2012

Human controlling a robot

Correct Controller

?

High-Level Tasks

EnvironmentModel

Robot Abstraction

EnvironmentImperfect Sensors

Human controlling a robot

Φ X Y

Controller Synthesis

DTMC Construction

R s.t. E R Φ

ProbabilisticModel Checker Φ

E, E

P( E R Φ)

Johnson and KG, RSS 2011, Johnson, Havlak, Campbell and KG, ICRA 2012

Human controlling a robot

Johnson and KG, RSS 2011, Johnson, Havlak, Campbell and KG, ICRA 2012

Human controlling a robot

Johnson and KG, RSS 2011, Johnson, Havlak, Campbell and KG, ICRA 2012

Human controlling a robot

Raman, Finucane and KG, IROS 2012, Raman, Piterman and KG, ICRA 2013 (To appear)

Human controlling a robot

Raman, Finucane and KG, IROS 2012, Raman, Piterman and KG, ICRA 2013 (To appear)

Human controlling a robot

Raman, Finucane and KG, IROS 2012, Raman, Piterman and KG, ICRA 2013 (To appear)

Option 1: Actions end together

Human controlling a robot

Raman, Finucane and KG, IROS 2012

Option 2: Actions start together

Human controlling a robot

Raman, Piterman and KG, ICRA 2013 (To appear)

Option 3: Model activation and completion of Actions

LTLMoP

• Linear Temporal Logic MissiOn Planner • Use structured English to control a robot • Python-based • Modular • Integrates with simulators (Gazebo) and

physical robots • http://ltlmop.github.com/

Robot waiter scenario – hot off the press