Human-­‐aware  Robo.cs  

1

CSE 591: Human-aware Robotics

Instructor: Dr. Yu (“Tony”) Zhang

Location & Times: CAVC 359, Tue/Thu, 9:00--10:15 AM Office Hours: BYENG 558, Tue/Thu, 10:30--11:30AM

Sep 22, 2016

This set of slides borrow from various online sources; it is used for educational purposes only.

Slides  adapted  from  Henry  Kautz  and  Oregon  State  University  

Human-­‐aware  Robo.cs  Plan/Goal  Recogni>on  

2

Human-­‐aware  Robo.cs  Approaches  to  goal/plan  recogni>on  •  Consistency-­‐based  

–  Hypothesize  &  revise  –  Closed-­‐world  reasoning  –  Version  spaces  

•  Probabilis>c  –  Stochas>c  grammars  –  Pending  sets  –  Layered  hidden  Markov  models  –  Policy  recogni>on  –  Hierarchical  hidden  semi-­‐Markov  models  –  Dynamic  probabilis>c  rela>onal  models  –  Dynamic  Bayes  networks  –  Example  applica>on:  Assisted  Cogni>on  

Can  be  complementary..        First  pick  the  consistent        plans,  and  check  which        of  them  is  most  likely    (tricky  if  the  agent          can  make  errors)  

3

Human-­‐aware  Robo.cs  Hypothesize  &  Revise  

•  The  Plan  Recogni.on  Problem  C.  Schmidt,  1978  

Based  on  psychological  theories  of  human  narra>ve  understanding  

Men>on  of  objects  suggest  hypothesis  

Pursue  single  hypothesis  un>l  matching  fails  

 

4

Human-­‐aware  Robo.cs  Closed-­‐world  reasoning  

•  A  Formal  Theory  of  Plan  Recogni>on  and  its  Implementa>on  Henry  Kautz,  1991  

• Infers  the  minimum  set(s)  of  independent  plans  that  entail  the  observa>ons  

• Observa>ons  may  be  incomplete  

• Infallible  agent  • Complete  plan  library  

5

Human-­‐aware  Robo.cs  Version  Space  Algebra  

•  A  sound  and  fast  goal  recognizer  Lesh  &  Etzioni  •  Programming  by  Demonstra>on  Using  Version  Space  

Algebra  Lau,  Wolfman,  Domingos,  Weld.  

• Recognizes  novel  plans  • Complete  observa>ons  

• Sensi>ve  to  noise  • Start  with  all  goals  

• Remove  goals  when  ac>ons  are  not  consistent  

6

Human-­‐aware  Robo.cs  Approaches  to  goal/plan  recogni>on  •  Consistency-­‐based  

–  Hypothesize  &  revise  –  Closed-­‐world  reasoning  –  Version  spaces  

•  Probabilis>c  –  Stochas>c  grammars  –  Pending  sets  –  Layered  hidden  Markov  models  –  Policy  recogni>on  –  Hierarchical  hidden  semi-­‐Markov  models  –  Dynamic  probabilis>c  rela>onal  models  –  Dynamic  Bayes  networks  –  Example  applica>on:  Assisted  Cogni>on  

Can  be  complementary..        First  pick  the  consistent        plans,  and  check  which        of  them  is  most  likely    (tricky  if  the  agent          can  make  errors)  

7

Human-­‐aware  Robo.cs  Stochas>c  grammars  

–  Huber,  Durfee,  &  Wellman,  "The  Automated  Mapping  of  Plans  for  Plan  Recogni>on",  1994    

–  Darnell  Moore  and  Irfan  Essa,  "Recognizing  Mul>tasked  Ac>vi>es  from  Video  using  Stochas>c  Context-­‐Free  Grammar",  AAAI-­‐02,  2002.    

CF  grammar  w/  probabilis>c  rules  

Successful  for  highly  structured  tasks  (e.g.  playing  cards)  

Problems:  errors,  context  

Human-­‐aware  Robo.cs  Pending  sets  

•  A  new  model  of  plan  recogni>on.  Goldman,  Geib,  and  Miller  •  Probabilis>c  plan  recogni>on  for  hos>le  agents.  Geib,  Goldman  

Explicitly  models  the  agent’s  “plan  agenda”  using  Poole’s  “probabilis>c  Horn  abduc>on”  rules  

Handles  mul>ple  concurrent  interleaved  plans  &  nega>ve  evidence  

Number  of  different  possible  pending  sets  can  grow  exponen>ally  

Context  problema>c?    Metric  >me?  

Happen(X,T+1)  ß            Pending(P,T),          X  in  P,          Pick(X,P,T+1).  

               

Pending(P’,T+1)ß          Pending(P,T),          Leaves(L),          Progress(L,  P,  P’,  T+1).  

               

Human-­‐aware  Robo.cs  Layered  hidden  Markov  models  

•  N.  Oliver,  E.  Horvitz,  and  A.  Garg.  Layered  Representa>ons  for  Recognizing  Office  Ac>vity,  Proceedings  of  the  Fourth  IEEE  Interna.onal  Conference  on  Mul.modal  Interac.on  (ICMI  2002)  

Cascade  of  HMM’s,  opera>ng  at  different  temporal  granulari>es  

Inferen>al  output  at  layer  K  is  “evidence”  for  layer  K+1  

 

Human-­‐aware  Robo.cs  Policy  Recogni>on  

•  Tracking  and  Surveillance  in  Wide-­‐Area  Spa>al  Environments  Using  the  Hidden  Markov  Model.  Hung  H.  Bui,  Svetha  Venkatesh  and  West.    

•  Bui,  H.  H.,  Venkatesh,  S.,  and  West,  G.  (2000)  On  the  recogni>on  of  abstract  Markov  policies.  Seventeenth  Na>onal  Conference  on  Ar>ficial  Intelligence  (AAAI-­‐2000),  Aus>n,  Texas  

Model  agent  using  hierarchy  of  abstract  policies  (e.g.  abstract  by  spa>al  decomposi>on)  

Compute  the  condi>onal  probability  of  top-­‐level  policy  given  observa>ons  

Compiled  into  DBN  

 

Human-­‐aware  Robo.cs  Hierarchical  hidden  semi-­‐Markov  models  

Combine  hierarchy  (func>on  call  seman>cs)  with  metric  >me  

Compile  to  DBN  Time  nodes  represent  a  

distribu>on  over  the  >me  of  the  next  state  “switch”  

“Linear  >me”  smoothing  –  Research  issues  –  

parametric  >me  nodes,  varying  granularity  

l  Hidden  semi-­‐Markov  models  (segment  models)    Kevin  Murphy.  November  2002.  

l  HSSM:  Theory  into  Prac>ce,  Deibel  &  Kautz,  forthcoming.  

Human-­‐aware  Robo.cs  Dynamic  probabilis>c  rela>onal  

models  

•  Friedman,  N.,  L.  Getoor,  D.  Koller,  A.  Pfeffer.  Learning  Probabilis>c  Rela>onal  Models.    IJCAI-­‐99,  Stockholm,  Sweden  (July  1999).    

•  Rela>onal  Markov  Models  and  their  Applica>on  to  Adap>ve  Web  Naviga>on,  Anderson,  Domingos,  Weld  2002.    

•  Dynamic  probabilis>c  rela>onal  models,  Anderson,  Domingos,  Weld,  forthcoming.  

PRM  -­‐  reasons  about  classes  of  objects  and  rela>ons  

Lajce  of  classes  can  capture  plan  abstrac>on    

DPRM  –  efficient  approximate  inference  by  Rao-­‐Blackwellized  par>cle  filtering  

Open:  approximate  smoothing?  

 

Human-­‐aware  Robo.cs  

Dynamic  Bayesian  Network  

Time  and  Change  in  Probabilis>c  Reasoning  

Human-­‐aware  Robo.cs  

Human-­‐aware  Robo.cs  

Human-­‐aware  Robo.cs  

Issues:  1-­‐order  Markov;  huge  CPT  table    

Human-­‐aware  Robo.cs  

Dynamic Bayes Networks are “templates” for specifying the relation between the values of a random variable across time-slices !e.g. How is Rain at time t related to Rain at time t+1? We call them templates because they need to be expanded (unfolded) to the required number of time steps to reason about the connection between variables at different time points

Human-­‐aware  Robo.cs  Special  Cases  of  DBNs  are  well  known  in  the  literature  

•  Restrict  number  of  variables  per  state  – Markov  Chain:  DBN  with  one  variable  that  is  fully  observable  

–  Hidden  Markov  Model:  DBN  with  only  one  state  variable  that  is  hidden  and  can  be  es>mated  through  evidence  variable(s)  

•  Restrict  the  type  of  CPD  –  Kalman  Filters:  DBN  where  the  system  transi>on  func>on  as  well  as  the  observa>on  variable  are  linear  gaussian  

•  The  advantage  of  Gaussians  is  that  the  posterior  distribu>on  remains  Gaussian  

Human-­‐aware  Robo.cs  

Human-­‐aware  Robo.cs  

Human-­‐aware  Robo.cs  

Plan  Recogni>on  Approaches  based  on  sejng  up  DBNs  

Human-­‐aware  Robo.cs  Dynamic  Bayes  nets  

•  E.  Horvitz,  J.  Breese,  D.  Heckerman,  D.  Hovel,  and  K.  Rommelse.  The  Lumiere  Project:  Bayesian  User  Modeling  for  Inferring  the  Goals  and  Needs  of  Solware  Users.  Proceedings  of  the  Fourteenth  Conference  on  Uncertainty  in  Ar.ficial  Intelligence,  July  1998.  

•  Towards  a  Bayesian  model  for  keyhole  plan  recogni>on  in  large  domains  Albrecht,  Zukermann,  Nicholson,  Bud  

Models  rela>onship  between  user’s  recent  ac>ons  and  goals  (help  needs)  

Probabilis>c  goal  persistence  

Human-­‐aware  Robo.cs  Cogni>ve  mode          {  normal,  error  }  

Dynamic  Bayesian  Nets  

GPS  reading  

Edge,  velocity,  posi>on  

Data  (edge)  associa>on  

Transporta>on  mode  

Trip  segment  

Goal  

zk-­‐1   zk  

xk-­‐1   xk  

θk-­‐1   θk  

Time k-1 Time k

mk-­‐1   mk  

tk-­‐1   tk  

gk-­‐1   gk  

ck-­‐1   ck  

Learning and Inferring Transportation Routines Lin Liao, Dieter Fox, and Henry Kautz, Nineteenth National Conference on Artificial Intelligence, San Jose, CA, 2004.

Human-­‐aware  Robo.cs  

Applica>ons  

Human-­‐aware  Robo.cs  Assisted  cogni>on  

l  Understanding  human  behavior    from  low-­‐level  sensory  data  l  Using  commonsense  knowledge  l  Learning  individual  user  models  

l  Ac>vely  offering  prompts  and  other  forms  of  help  as  needed  

l  Aler>ng  human  caregivers  when  necessary                                                          hpp://www.cs.washington.edu/assistcog/  

Computer systems that improve the independence and safety of people suffering from cognitive limitations by…

Human-­‐aware  Robo.cs  Ac>vity  Compass  •  Zero-­‐configura>on  personal  

guidance  system  –  Learns  model  of  user’s  travel  

on  foot,  by  public  transit,  by  bike,  by  car    

–  Predicts  user’s  next  des>na>on,  offers  proac>ve  help  if  lost  or  late  

•  Integrates  user  data  with  external  constraints  –  Maps,  bus  schedules,  

calendars,  …  –  EM  approach  to  clustering  &  

segmen>ng  data  

The  Ac>vity  Compass    Don  Paperson,  Oren  Etzioni,  and  Henry  Kautz  (2003)  

Human-­‐aware  Robo.cs  Ac>vity  of  daily  living  monitor  &  prompter  

Founda>ons  of  Assisted  Cogni>on  Systems.  Kautz,  Etzioni,  Fox,  Weld,  and  Shastri,  2003  

Human-­‐aware  Robo.cs  Recognizing  unexpected  events  using  online  model  selec>on  

•  User  errors,  abnormal  behavior  

•  Select  model  that  maximizes  likelihood  of  data:  –  Generic  model  –  User-­‐specific  model  –  Corrupt  (impaired)  user  

model  •  Neurologically-­‐plausible  

corrup>ons  –  Repe>>on  –  Subs>tu>on  –  Stalling  

Fox,  Kautz,  &  Shastri  

fill  keple   put  keple  on  stove  

fill  keple   put  keple  on  stove  

put  keple  in  closet