Discriminative Training of Kalman Filters P. Abbeel, A. Coates, M. Montemerlo, A. Y. Ng, S. Thrun
Joelle Pineau Michael Montemerlo Martha Pollack * Nicholas Roy Sebastian Thrun
-
Upload
nolan-bullock -
Category
Documents
-
view
26 -
download
0
description
Transcript of Joelle Pineau Michael Montemerlo Martha Pollack * Nicholas Roy Sebastian Thrun
![Page 1: Joelle Pineau Michael Montemerlo Martha Pollack * Nicholas Roy Sebastian Thrun](https://reader035.fdocuments.in/reader035/viewer/2022062408/5681301c550346895d959970/html5/thumbnails/1.jpg)
Probabilistic Control of Human Robot Interaction:Probabilistic Control of Human Robot Interaction:
Experiments with a Robotic Assistant for Nursing HomesExperiments with a Robotic Assistant for Nursing Homes
Joelle PineauMichael MontemerloMartha Pollack *
Nicholas RoySebastian Thrun
Carnegie Mellon University*University of Michigan
![Page 2: Joelle Pineau Michael Montemerlo Martha Pollack * Nicholas Roy Sebastian Thrun](https://reader035.fdocuments.in/reader035/viewer/2022062408/5681301c550346895d959970/html5/thumbnails/2.jpg)
The Nursebot Project Joelle Pineau
Introducing Pearl – A mobile robotic assistant for elderly people and nurses
cameras
sonars
handle bars
mobile base
carrying tray
LCD mouth
touchscreen
microphone& speakers
laser
ROLE:
Moving thingsaroundMoving thingsaround
Management support of ADLsManagement support of ADLs
Providing physical assistance
Providing physical assistance
Remotehealth servicesRemotehealth services
Supportingcommunication
Supportingcommunication
Calling for helpin emergenciesCalling for helpin emergencies
Monitoring Rxadherence & safety
Monitoring Rxadherence & safety
Providing info(TV, weather)Providing info(TV, weather)
Reminding to eat,drink, take meds
Reminding to eat,drink, take meds
Linking caregiverand resources
Linking caregiverand resources
![Page 3: Joelle Pineau Michael Montemerlo Martha Pollack * Nicholas Roy Sebastian Thrun](https://reader035.fdocuments.in/reader035/viewer/2022062408/5681301c550346895d959970/html5/thumbnails/3.jpg)
The Nursebot Project Joelle Pineau
The Nursebot project in its early days
![Page 4: Joelle Pineau Michael Montemerlo Martha Pollack * Nicholas Roy Sebastian Thrun](https://reader035.fdocuments.in/reader035/viewer/2022062408/5681301c550346895d959970/html5/thumbnails/4.jpg)
The Nursebot Project Joelle Pineau
Architecture
Cognitive supportNavigation Communication
High-level controller
![Page 5: Joelle Pineau Michael Montemerlo Martha Pollack * Nicholas Roy Sebastian Thrun](https://reader035.fdocuments.in/reader035/viewer/2022062408/5681301c550346895d959970/html5/thumbnails/5.jpg)
The Nursebot Project Joelle Pineau
• Localization and map building(Burgard et al., 1999)
• People detection and tracking(Montemerlo et al., 2002)
Architecture
Cognitive supportNavigation Communication
High-level controller
![Page 6: Joelle Pineau Michael Montemerlo Martha Pollack * Nicholas Roy Sebastian Thrun](https://reader035.fdocuments.in/reader035/viewer/2022062408/5681301c550346895d959970/html5/thumbnails/6.jpg)
The Nursebot Project Joelle Pineau
• Autominder system (Pollack et al., 2002)
Architecture
Cognitive supportNavigation Communication
High-level controller
![Page 7: Joelle Pineau Michael Montemerlo Martha Pollack * Nicholas Roy Sebastian Thrun](https://reader035.fdocuments.in/reader035/viewer/2022062408/5681301c550346895d959970/html5/thumbnails/7.jpg)
The Nursebot Project Joelle Pineau
• Speech recognition: Sphinx system(Ravishankar, 1996)
• Speech synthesis: Festival system(Black et al., 1999)
Architecture
Cognitive supportNavigation Communication
High-level controller
![Page 8: Joelle Pineau Michael Montemerlo Martha Pollack * Nicholas Roy Sebastian Thrun](https://reader035.fdocuments.in/reader035/viewer/2022062408/5681301c550346895d959970/html5/thumbnails/8.jpg)
The Nursebot Project Joelle Pineau
The role of the top-level controller
• Établir les priorités parmi les objectifs des différents modules
• Négocier entre plusieurs objectifs ayant des coûts/gains variés
• Négocier entre l’acquisition d’information et la rencontre des objectifs
• Passer d’une tâche à l’autre en partageant l’information sensorielle
• Planifier malgré la présence d’incertitude
Cognitive supportNavigation Communication
ACTION SELECTION - based on the trade-off between:
- goals from different modules;
- goals with varying costs / rewards;
- reducing uncertainty versus accomplishing goals.
High-level controller
![Page 9: Joelle Pineau Michael Montemerlo Martha Pollack * Nicholas Roy Sebastian Thrun](https://reader035.fdocuments.in/reader035/viewer/2022062408/5681301c550346895d959970/html5/thumbnails/9.jpg)
The Nursebot Project Joelle Pineau
Speech recognition with Sphinx
![Page 10: Joelle Pineau Michael Montemerlo Martha Pollack * Nicholas Roy Sebastian Thrun](https://reader035.fdocuments.in/reader035/viewer/2022062408/5681301c550346895d959970/html5/thumbnails/10.jpg)
The Nursebot Project Joelle Pineau
Robot control under uncertainty
Belief StateP(st=weather-today)=0.5
P(st=appointment-today )=0.5
USER
Action={ say-weather,update-appointment,clarify-query}
Speech=“today”
Stateweather-today
![Page 11: Joelle Pineau Michael Montemerlo Martha Pollack * Nicholas Roy Sebastian Thrun](https://reader035.fdocuments.in/reader035/viewer/2022062408/5681301c550346895d959970/html5/thumbnails/11.jpg)
The Nursebot Project Joelle Pineau
Robot control using Partially Observable MarkovPartially Observable Markov Decision Processes Decision Processes (POMDPs)
Belief state
USER + ENVIRONMENT + WORLD
Actions
ObservationsCosts / Rewards
State
Problem: Which action allows the robot to maximize its reward?
P(s1)P(s2)
![Page 12: Joelle Pineau Michael Montemerlo Martha Pollack * Nicholas Roy Sebastian Thrun](https://reader035.fdocuments.in/reader035/viewer/2022062408/5681301c550346895d959970/html5/thumbnails/12.jpg)
The Nursebot Project Joelle Pineau
Methods to solve POMDPs
Objective: Find a policy, (b), which maximizes reward.
Complexity
Performance
QMDP
MDP
FIB
UMDP
AMDP
O(S2A) O(S2AT) O(S2AO )O(S2AO) O(S2AB) T
POMDP
New methods?
![Page 13: Joelle Pineau Michael Montemerlo Martha Pollack * Nicholas Roy Sebastian Thrun](https://reader035.fdocuments.in/reader035/viewer/2022062408/5681301c550346895d959970/html5/thumbnails/13.jpg)
The Nursebot Project Joelle Pineau
New approach: A hierarchy of POMDPs
Idea: Exploit domain knowledge to divide one POMDP into many smaller ones.
Motivation: Complexity of POMDP solving grows exponentially with # of actions.
Assumption: We are given POMDP M = {S,A,,b,T,O,R} and hierarchy H
Act
ExamineHealth Navigate
MoveVerifyPulse
ClarifyGoal
North South East West
VerifyMeds
subtask
abstract action
primitive action
![Page 14: Joelle Pineau Michael Montemerlo Martha Pollack * Nicholas Roy Sebastian Thrun](https://reader035.fdocuments.in/reader035/viewer/2022062408/5681301c550346895d959970/html5/thumbnails/14.jpg)
The Nursebot Project Joelle Pineau
PolCA: Planning with a hierarchy of POMDPs
Step 1: Select the action setNavigate
Move ClarifyGoal
South East WestNorth
AMove = {N,S,E,W}
ACTIONSNorthSouthEastWest
ClarifyGoalVerifyPulseVerifyMeds
ACTIONSNorthSouthEastWest
ClarifyGoalVerifyPulseVerifyMeds
![Page 15: Joelle Pineau Michael Montemerlo Martha Pollack * Nicholas Roy Sebastian Thrun](https://reader035.fdocuments.in/reader035/viewer/2022062408/5681301c550346895d959970/html5/thumbnails/15.jpg)
The Nursebot Project Joelle Pineau
PolCA: Planning with a hierarchy of POMDPs
Step 1: Select the action set
Step 2: Minimize the state set
STATE FEATURESX-positionY-position
X-goalY-goal
HealthStatus
STATE FEATURESX-positionY-position
X-goalY-goal
HealthStatus
Navigate
Move ClarifyGoal
South East WestNorth
AMove = {N,S,E,W}
SMove = {X,Y}
ACTIONSNorthSouthEastWest
ClarifyGoalVerifyPulseVerifyMeds
ACTIONSNorthSouthEastWest
ClarifyGoalVerifyPulseVerifyMeds
![Page 16: Joelle Pineau Michael Montemerlo Martha Pollack * Nicholas Roy Sebastian Thrun](https://reader035.fdocuments.in/reader035/viewer/2022062408/5681301c550346895d959970/html5/thumbnails/16.jpg)
The Nursebot Project Joelle Pineau
PolCA: Planning with a hierarchy of POMDPs
Step 1: Select the action set
Step 2: Minimize the state set
Step 3: Choose parameters
STATE FEATURESX-positionY-position
X-goalY-goal
HealthStatus
STATE FEATURESX-positionY-position
X-goalY-goal
HealthStatus
Navigate
Move ClarifyGoal
South East WestNorth
AMove = {N,S,E,W}
SMove = {X,Y}
ACTIONSNorthSouthEastWest
ClarifyGoalVerifyPulseVerifyMeds
ACTIONSNorthSouthEastWest
ClarifyGoalVerifyPulseVerifyMeds
PARAMETERS
{bh,Th,Oh,Rh}
PARAMETERS
{bh,Th,Oh,Rh}
![Page 17: Joelle Pineau Michael Montemerlo Martha Pollack * Nicholas Roy Sebastian Thrun](https://reader035.fdocuments.in/reader035/viewer/2022062408/5681301c550346895d959970/html5/thumbnails/17.jpg)
The Nursebot Project Joelle Pineau
PolCA: Planning with a hierarchy of POMDPs
Step 1: Select the action set
Step 2: Minimize the state set
Step 3: Choose parameters
Step 4: Plan task h
STATE FEATURESX-positionY-position
X-goalY-goal
HealthStatus
STATE FEATURESX-positionY-position
X-goalY-goal
HealthStatus
Navigate
Move ClarifyGoal
South East WestNorth
AMove = {N,S,E,W}
SMove = {X,Y}
ACTIONSNorthSouthEastWest
ClarifyGoalVerifyPulseVerifyMeds
ACTIONSNorthSouthEastWest
ClarifyGoalVerifyPulseVerifyMeds
PLAN
h
PLAN
h
PARAMETERS
{bh,Th,Oh,Rh}
PARAMETERS
{bh,Th,Oh,Rh}
![Page 18: Joelle Pineau Michael Montemerlo Martha Pollack * Nicholas Roy Sebastian Thrun](https://reader035.fdocuments.in/reader035/viewer/2022062408/5681301c550346895d959970/html5/thumbnails/18.jpg)
The Nursebot Project Joelle Pineau
PolCA in the Nursebot domain
• Goal: A robot is deployed in a nursing home, where it provides reminders to elderly users and accompanies them to appointments.
• Domain: |S|=512, |A|=20, |O|=19 • Hierarchy:
![Page 19: Joelle Pineau Michael Montemerlo Martha Pollack * Nicholas Roy Sebastian Thrun](https://reader035.fdocuments.in/reader035/viewer/2022062408/5681301c550346895d959970/html5/thumbnails/19.jpg)
The Nursebot Project Joelle Pineau
Sample scenario
![Page 20: Joelle Pineau Michael Montemerlo Martha Pollack * Nicholas Roy Sebastian Thrun](https://reader035.fdocuments.in/reader035/viewer/2022062408/5681301c550346895d959970/html5/thumbnails/20.jpg)
The Nursebot Project Joelle Pineau
Results for dialogue system
0.1 0.1
0.18
POMDP policy MDP policy
![Page 21: Joelle Pineau Michael Montemerlo Martha Pollack * Nicholas Roy Sebastian Thrun](https://reader035.fdocuments.in/reader035/viewer/2022062408/5681301c550346895d959970/html5/thumbnails/21.jpg)
The Nursebot Project Joelle Pineau
Summary
• We have developed a first prototype robot able to serve as a mobile nursing assistant for elderly people.
• The top-level controller uses a hierarchical variant of POMDPs to select actions.
• This allows it to acquire necessary information and successfully complete assigned tasks.
• Probabilistic techniques have been found to be very useful to flexibly model and track individuals.
![Page 22: Joelle Pineau Michael Montemerlo Martha Pollack * Nicholas Roy Sebastian Thrun](https://reader035.fdocuments.in/reader035/viewer/2022062408/5681301c550346895d959970/html5/thumbnails/22.jpg)
The Nursebot Project Joelle Pineau
For more details: www.cs.cmu.edu/~nursebot
The Nursebot team
CMU - Robotics:Greg Armstrong
Michael MontemerloJoelle PineauNicholas RoyJamie Schulte
Sebastian Thrun
CMU - HCI/Design:Francine Gemperle
Jennifer GoetzSarah KieslerAaron Powers
U. of Pittsburgh - Nursing:Jacqueline Dunbar-Jacobs
Sandra EngbergJudith Matthews
U. of Pittsburgh - CS:Don Chiarulli
Colleen McCarthy
U. of Freiburg - CS:Maren BennewitzWolfram Burgard
Dirk Schulz
U. of Michigan - CS:Laura BrownDirk ColbryCheryl OroszBart PeintnerMartha PollackSailesh Ramakrishnan
Standard Robotics:Greg Baltus