Learning(to(InterpretNatural( Language(Commands(through ... fileCommanding(Robots(•...

25
Learning to Interpret Natural Language Commands through HumanRobot Dialog Jesse Thomason Shiqi Zhang, Raymond Mooney, Peter Stone The University of Texas at AusHn 1

Transcript of Learning(to(InterpretNatural( Language(Commands(through ... fileCommanding(Robots(•...

Page 1: Learning(to(InterpretNatural( Language(Commands(through ... fileCommanding(Robots(• Autonomous(robots(in(human(environments(• Simplestto(interactwith(vianatural(language(2

Learning  to  Interpret  Natural  Language  Commands  through  

Human-­‐Robot  Dialog      

Jesse  Thomason  Shiqi  Zhang,  Raymond  Mooney,  Peter  Stone  

 The  University  of  Texas  at  AusHn  

1  

Page 2: Learning(to(InterpretNatural( Language(Commands(through ... fileCommanding(Robots(• Autonomous(robots(in(human(environments(• Simplestto(interactwith(vianatural(language(2

Commanding  Robots  

•  Autonomous  robots  in  human  environments  

•  Simplest  to  interact  with  via  natural  language  

2  

Page 3: Learning(to(InterpretNatural( Language(Commands(through ... fileCommanding(Robots(• Autonomous(robots(in(human(environments(• Simplestto(interactwith(vianatural(language(2

Our  Task  

•  Command  a  robot  operaHng  in  an  office  environment  

•  Robot  autonomously  wanders  by  default  

•  Robot  can  navigate  to  rooms  and  deliver  items  

3  

Page 4: Learning(to(InterpretNatural( Language(Commands(through ... fileCommanding(Robots(• Autonomous(robots(in(human(environments(• Simplestto(interactwith(vianatural(language(2

System  Goals  

•  Require  liRle  iniHal  data  – More  domain  independent  

•  Reason  using  composiHon  – “Alice’s  office”  

•  Robust  to  lexical  variaHon  – “bring”,  “deliver”,  “take”  

•  Execute  the  right  acHon  – Perform  clarificaHons  with  user  

4  

Page 5: Learning(to(InterpretNatural( Language(Commands(through ... fileCommanding(Robots(• Autonomous(robots(in(human(environments(• Simplestto(interactwith(vianatural(language(2

Closest  Previous  Work  

•  Service  robot  that  accepts  commands    (Kollar,  2013)  

•  SemanHcs  match  spans  of  words  to  known  acHons/people/locaHons  

•  Can  learn  new  referring  expressions  through  dialog  

 •  This  system  would  explicitly  match  “Alice’s  office”  to  room  3  

5  

Human   Go  to  Alice’s  office  Robot   Where  is  “Alice’s  office”?  Human   Room  3  

Page 6: Learning(to(InterpretNatural( Language(Commands(through ... fileCommanding(Robots(• Autonomous(robots(in(human(environments(• Simplestto(interactwith(vianatural(language(2

Closest  Previous  Work  

•  When  system  sees  “Bob’s  office”,  will  have  to  ask  where  that  is  

•  Want  to  take  advantage  of  composiHonality  instead  – Reason  about  possessive  marker  “’s”  and  what  enHHes  “office”  picks  out  

•  Need  a  more  powerful  formalism  for  represenHng  sentence  semanHcs  – Want  to  keep  iniHal  training  data  light  

6  

Page 7: Learning(to(InterpretNatural( Language(Commands(through ... fileCommanding(Robots(• Autonomous(robots(in(human(environments(• Simplestto(interactwith(vianatural(language(2

Helpful  Previous  Work  

•  Augment  a  semanHc  parser  through  conversaHon  logs  (Artzi,  2011)  

   •  Key  idea  for  us:  use  known  system  semanHc  meanings  to  guess  human  uRerance  word  meanings  

7  

Human   I  would  like  to  fly  out  of  boston  arriving  to  new  york  and  back  from  new  york  to  boston  

System   Leaving  boston  (CONFIRM:from(fl1,BOS))  on  what  date?  (ASK:λx.departdate(fl1,x))  

Page 8: Learning(to(InterpretNatural( Language(Commands(through ... fileCommanding(Robots(• Autonomous(robots(in(human(environments(• Simplestto(interactwith(vianatural(language(2

Tag  Token  Sequence  

8  

(S/PP)/NP  :  λx.λP.(ac7on(bring)∧pa7ent(bring,x)∧P(bring))  

NP  :  coffee  

PP/NP  :  λx.λy.(recipient(y,x))  

bring  

coffee  

to  

Bob   NP  :  bob  

syntax  :  seman7cs  token(s)  

University  of  Washington  SemanHc  Parsing  Framework  (SPF);  (Artzi,  2011)  Known  possibiliHes  for  each  token  stored  in  a  lexicon  Use  Combinatory  Categorial  Grammar  (CCG)-­‐driven  parsing  

Page 9: Learning(to(InterpretNatural( Language(Commands(through ... fileCommanding(Robots(• Autonomous(robots(in(human(environments(• Simplestto(interactwith(vianatural(language(2

Construct  Meaning  Hierarchically  

9  

(S/PP)/NP  :  λx.λP.(ac7on(bring)∧pa7ent(bring,x)∧P(bring))   NP  :  coffee   PP/NP  :  λx.λy.(recipient(y,x))  

bring   coffee   to   Bob  

NP  :  bob  

S/PP  :  λP.(ac7on(bring)∧pa7ent(bring,coffee)∧P(bring))   PP  :  λy.(recipient(y,bob))  

S  :  ac7on(bring)∧pa7ent(bring,coffee)∧recipient(bring,bob)  

Page 10: Learning(to(InterpretNatural( Language(Commands(through ... fileCommanding(Robots(• Autonomous(robots(in(human(environments(• Simplestto(interactwith(vianatural(language(2

Tag  Token  Sequence  –  Missing  Entry  

10  

(S/PP)/NP  :  λx.λP.(ac7on(bring)∧pa7ent(bring,x)∧P(bring))  

?  

PP/NP  :  λx.λy.(recipient(y,x))  

bring  

java  

to  

Bob   NP  :  bob  

Given  semanHc  form,  can  guess  about  missing  token  syntax/semanHcs  

S  :  ac7on(bring)∧pa7ent(bring,coffee)∧recipient(bring,bob)  

Human   bring  java  to  bob  Robot   what  should  I  bring  to  bob?  Human   coffee  

Page 11: Learning(to(InterpretNatural( Language(Commands(through ... fileCommanding(Robots(• Autonomous(robots(in(human(environments(• Simplestto(interactwith(vianatural(language(2

Tag  Token  Sequence  –  Missing  Entry  

11  

(S/PP)/NP  :  λx.λP.(ac7on(bring)∧pa7ent(bring,x)∧P(bring))  

?  

PP/NP  :  λx.λy.(recipient(y,x))  

bring  

java  

to  

Bob   NP  :  bob  

:-­‐  (S/PP)/NP  :  λx.λP.(ac7on(bring)∧pa7ent(bring,x)∧P(bring))  :-­‐  (S/NP)/NP  :  λx.  λy.(ac7on(bring)∧recipient(bring,x)  ∧pa7ent(bring,y))  :-­‐  NP  :  coffee  :-­‐  NP  :  bob  

ac7on(bring)∧pa7ent(bring,coffee)∧  recipient(bring,bob)  

bring  :-­‐  (S/PP)/NP  :  λx.λP.(ac7on(bring)∧pa7ent(bring,x)∧P(bring))  bring  :-­‐  (S/NP)/NP  :  λx.λy.(ac7on(bring)∧recipient(bring,x)  ∧pa7ent(bring,y))  coffee  :-­‐  NP  :  coffee  Bob  :-­‐  NP  :  bob  

Given  form:  

Lexicon  entries  that  produce  parts  of  this  form:  

Candidates  for  ‘java’  lexical  entry:  

Page 12: Learning(to(InterpretNatural( Language(Commands(through ... fileCommanding(Robots(• Autonomous(robots(in(human(environments(• Simplestto(interactwith(vianatural(language(2

Tag  Token  Sequence  –  Missing  Entry  

(S/PP)/NP  :  λx.λP.(ac7on(bring)∧pa7ent(bring,x)∧P(bring))  

NP  :  coffee  

PP/NP  :  λx.λy.(recipient(y,x))  

bring  

java  

to  

Bob   NP  :  bob  

With  new  lexicon  entry,  we  can  construct  the  correct  semanHc  form  

S  :  ac7on(bring)∧pa7ent(bring,coffee)∧recipient(bring,bob)  

Page 13: Learning(to(InterpretNatural( Language(Commands(through ... fileCommanding(Robots(• Autonomous(robots(in(human(environments(• Simplestto(interactwith(vianatural(language(2

MeeHng  System  Goals  

•  Require  liRle  iniHal  data  – Bootstrap  parser  with  5  expressions,  105  words  

•  Handle  composiHon  used  by  speakers  – Use  CCG-­‐driven  semanHc  parsing  (Artzi,  2011)  

•  Robust  to  lexical  variaHon  –  Incrementally  train  parser  to  obtain  new  words  

•  Execute  the  right  acHon  – Use  dialog  to  clarify  meanings  with  user  (Kollar,  2013)  

13  

Page 14: Learning(to(InterpretNatural( Language(Commands(through ... fileCommanding(Robots(• Autonomous(robots(in(human(environments(• Simplestto(interactwith(vianatural(language(2

Mechanical  Turk  Experiment  

•  Users  given  one  navigaHon  and  one  delivery  goal  – Train/test  goals  chosen  at  random  from  possibiliHes  

•  Chat  with  robot’s  dialog  agent  unHl  goal  is  understood  

14  

Page 15: Learning(to(InterpretNatural( Language(Commands(through ... fileCommanding(Robots(• Autonomous(robots(in(human(environments(• Simplestto(interactwith(vianatural(language(2

Mechanical  Turk  Interface  

15  

Page 16: Learning(to(InterpretNatural( Language(Commands(through ... fileCommanding(Robots(• Autonomous(robots(in(human(environments(• Simplestto(interactwith(vianatural(language(2

Large-­‐Scale  Experiment  

•  Tested  in  4  phases  •  ~50  users  received  test  goals,  ~50  train  goals  – Unique  users  in  each  phase  

•  System  incrementally  trained  via  train  goal  conversaHons  only  

16  

Page 17: Learning(to(InterpretNatural( Language(Commands(through ... fileCommanding(Robots(• Autonomous(robots(in(human(environments(• Simplestto(interactwith(vianatural(language(2

Mechanical  Turk  Dialog  Turns  

17  

2.7   2.5  3.4   3.5  

18  

5.6   5.2  3.8  

0  2  4  6  8  10  12  14  16  18  20  

Batch  0   Batch  1   Batch  2   Batch  3  

Average  Tu

rker  Turns  fo

r  Success  

NavigaHon  task  turns   Delivery  task  turns  

*  

Page 18: Learning(to(InterpretNatural( Language(Commands(through ... fileCommanding(Robots(• Autonomous(robots(in(human(environments(• Simplestto(interactwith(vianatural(language(2

Mechanical  Turk  Survey  Responses  

18  

2.2  

2.6   2.7   2.8  

2.2  2   1.9  

1.5  

0  

0.5  

1  

1.5  

2  

2.5  

3  

Batch  0   Batch  1   Batch  2   Batch  3  

Average  

The  robot  understood  me   The  robot  frustrated  me  

Strongly  Agree      

Somewhat  Agree            

Somewhat  Disagree  

   

Strongly  Disagree  

*   *  

*  

Page 19: Learning(to(InterpretNatural( Language(Commands(through ... fileCommanding(Robots(• Autonomous(robots(in(human(environments(• Simplestto(interactwith(vianatural(language(2

Robot  Experiment  

•  Same  setup,  but  real  robot  and  fewer  users  – Users  type  to  robot  to  mimic  Mechanical  Turk  setup  

•  10  users  in  iniHal  test  batch  •  System  interacted  freely  with  people  on  the  floor  for  four  days  as  training  (34  conversaHons  in  total)  

•  10  users  in  the  second  test  batch,  arer  retraining  

19  

Page 20: Learning(to(InterpretNatural( Language(Commands(through ... fileCommanding(Robots(• Autonomous(robots(in(human(environments(• Simplestto(interactwith(vianatural(language(2

Office  Robot  Dialog  CompleHon  

20  

90   90  

20  

60  

0  10  20  30  40  50  60  70  80  90  

100  

Batch  0   Batch  1  

Percen

t  of  U

sers  Com

pleF

ng  Task  

NavigaHon  task  compleHon   Delivery  task  compleHon  

+  

Page 21: Learning(to(InterpretNatural( Language(Commands(through ... fileCommanding(Robots(• Autonomous(robots(in(human(environments(• Simplestto(interactwith(vianatural(language(2

Office  Robot  Survey  Responses  

21  

1.6  

2.9  

2.5  

1.5  

0  

0.5  

1  

1.5  

2  

2.5  

3  

3.5  

Batch  0   Batch  1  

Average  

The  robot  understood  me   The  robot  frustrated  me  

Strongly  Agree      

Somewhat  Agree          

Somewhat  Disagree  

   

Strongly  Disagree  

*  

+  

Page 22: Learning(to(InterpretNatural( Language(Commands(through ... fileCommanding(Robots(• Autonomous(robots(in(human(environments(• Simplestto(interactwith(vianatural(language(2

Conclusions  

•  Lexical  acquisiHon  reduces  dialog  lengths  for  mulH-­‐argument  predicates  like  delivery  

•  Causes  users  to  perceive  the  system  as  more  understanding  

•  Leads  to  less  user  frustraHon  •  Allows  improving  language  understanding  without  large,  annotated  corpora  

22  

Page 23: Learning(to(InterpretNatural( Language(Commands(through ... fileCommanding(Robots(• Autonomous(robots(in(human(environments(• Simplestto(interactwith(vianatural(language(2

Learning  to  Interpret  Natural  Language  Commands  through  

Human-­‐Robot  Dialog      

Jesse  Thomason  Shiqi  Zhang,  Raymond  Mooney,  Peter  Stone  

 The  University  of  Texas  at  AusHn  

23  

Page 24: Learning(to(InterpretNatural( Language(Commands(through ... fileCommanding(Robots(• Autonomous(robots(in(human(environments(• Simplestto(interactwith(vianatural(language(2

Related  Work  

•  Command  processing  has  taken  many  forms  •  Specify  tasks  step-­‐by-­‐step  (Meriçli,  2014)  – Assumes  parHcular  words  in  parHcular  order  

•  Specify  low-­‐level  acHon  sequences  (Misra,  2014;  Tellex,  2011)  – Uses  a  parser  trained  on  a  huge  corpus  

•  Map  language  to  acHon  specificaHons  (Matuszek,  2013)  – Cannot  learn  new  words/expressions  

24  

Page 25: Learning(to(InterpretNatural( Language(Commands(through ... fileCommanding(Robots(• Autonomous(robots(in(human(environments(• Simplestto(interactwith(vianatural(language(2

Future  Work  

•  Perceptual  grounding  (`blue’,`ler  of’)  •  Predicate  invenHon  (`ruddy’)  •  Learning  a  mulH-­‐objecHve  dialog  policy  that  trades  off  learning  and  user  saHsfacHon    

 

25