Solution
description
Transcript of Solution
![Page 1: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/1.jpg)
Solution
Prevents collisions Infers the user's goal location/activity and
provides automated reminders Provides navigation assistance using prompts
that account for the user’s cognitive state
Intelligent powered wheelchair for older adults with cognitive impairment that:
![Page 2: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/2.jpg)
System overviewThe system consists of:Nimble Rocket TM Powered
WheelchairBumblebee Stereovision
Camera from Point Grey Research
Fujitsu Lifebook P7120 Laptop (under seat)
![Page 3: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/3.jpg)
System Overview
![Page 4: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/4.jpg)
Prompting strategyFulfill the following (possibly conflicting) goals according to the following order of priority: 1.Ensure safety (through navigation assistance, medication
reminders, etc.)2.Assist in the effective completion of daily activities3.Minimize user frustration (minimize incorrect and excessive
prompting)4.Maximize user independence (minimize caregiver
intervention)5.Maximize user awareness (issue appropriate level of
prompts with justification)
![Page 5: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/5.jpg)
Control Strategy
Semi-Autonomous
ManualAutonomous
Strength:
No need for user input
Weakness:
User might want some control
Strength:
User has full control
Weakness: Tedious, user might not have ability
Combines strengths of other 2 systems
How do we determine who has control and when?
![Page 6: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/6.jpg)
Collision Avoidance
• Find the distance to objects – stored in depth maps
• Use this to create a map of all obstacles in front of the wheelchair – occupancy map
![Page 7: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/7.jpg)
Depth• Stereopsis
Left Image
Right Image Depth Map
Point Grey’s Bumblebee Camera
![Page 8: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/8.jpg)
Occupancy Grid
Depth Map 2D Projection - Occupancy Map
![Page 9: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/9.jpg)
Example OGs
![Page 10: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/10.jpg)
Example OGs
![Page 11: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/11.jpg)
Example OGs
![Page 12: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/12.jpg)
Example OGs
![Page 13: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/13.jpg)
Example OGs
![Page 14: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/14.jpg)
Example OGs
![Page 15: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/15.jpg)
Example OGs
![Page 16: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/16.jpg)
Collision Avoidance
• If object detected within a specified distance threshold, wheelchair is stopped
• Compute direction around obstacle with greatest amount of free space
![Page 17: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/17.jpg)
Collision Avoidance
Prompt: “Try turning left”
Most free space is to
the left of the object
![Page 18: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/18.jpg)
Demo
• Anti-collision demo
![Page 19: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/19.jpg)
Pilot Study
• Experiments conducted to test efficacy of anti-collision and prompting system
• Conducted within controlled environment
![Page 20: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/20.jpg)
Pilot Study
• Trials tested:–Detection of objects commonly found
in LTC facility–Collision avoidance–Correct prompt issued
![Page 21: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/21.jpg)
Object Detection
• Anti-collision system was tested with the following commonly-found objects:– A painted white wall with a flat finish– A light green aluminum 4-wheeled walker– A silver aluminum walking cane– A person who was standing still– A person who was moving
![Page 22: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/22.jpg)
Results
• Misses occurred during wall and cane conditions
• System performs better on larger and more textured objects
Overall Anti-collision Results
![Page 23: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/23.jpg)
Results
Distance between wheelchair and object when stopped
![Page 24: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/24.jpg)
Results
Overall Prompting Results
![Page 25: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/25.jpg)
Now what???
• Example Scenario: I’m hungry…
It’s 11:50 a.m. Mary
eats lunch at 12:00
![Page 26: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/26.jpg)
Now what???
• Example Scenario: I’m hungry…
It’s lunch time! Let’s go to the dining
hall!
![Page 27: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/27.jpg)
Navigation Assistance
• To assist in navigation, wheelchair must know three things:
– Where the user wants to go (destination)– Where the destination is located– Where the chair is located
• User destination - learned user schedules and/or from past behaviours
• Locations – need maps!!
![Page 28: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/28.jpg)
Automated Mapping
• Wheelchair automatically builds map of environment using visual landmarks
• Wheelchair can then find its current location by matching landmarks in the incoming images with those in the map
• Known as SLAM
![Page 29: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/29.jpg)
Navigation Assistance
User Model(responsiveness, awareness etc.)
1. Annotate Map
1. Compute Path
Lounge
Kitchen
Bedroom
Lounge
Kitchen
Lounge
Kitchen
BedroomBedroom
LoungeBedroom
Kitchen
1. Issue Prompt
This step involves using a POMDP as in Hoey et al. 2006
![Page 30: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/30.jpg)
Automated Labeling
CuriousGeorge
Recognition
![Page 31: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/31.jpg)
Planning and Prompting
• Remind the user of where he/she needs to be• Plan the shortest (?) path to the destination• Prompt the user as necessary• Avoid obstacles on the way
![Page 32: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/32.jpg)
Planning and Prompting
• The MDP (and POMDP) framework is great for task specification and planning
• A task is specified via the Reward function• Planning can be done “efficiently” using value
or policy iteration (exact and approximate methods)
• Problems:– Sensor noise– Large state, action and observation spaces
![Page 33: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/33.jpg)
Flat vs. Structured POMDPs
• Flat – States, Actions, Observations• Structured
– States State variables– Actions Action variables– Observations Observation variables
• State variables - X = {X1,…,Xn}
• State - s = <x1,…, xn>
![Page 34: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/34.jpg)
Structured POMDPs• Dynamic Bayesian Networks – 2-layered, model dynamic
changes• Nodes – Variables• Edges – dependency• CPT – conditional probability table
Ot Ot+1 Ot+2
At-1 At At+1
Bt Bt+1 Bt+2
Dt Dt+1 Dt+2
Actions
State
Observations
![Page 35: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/35.jpg)
CPT as Decision Diagrams• Decision Diagrams
– Inner nodes – variables– Edges – values (left = False, right = True)– Leaves hold values
• Algebraic Decision Diagrams (ADD) – Nodes with identical children are removed– Context specific independence
X1 X3 X’1F F 0.5F T 0.5T F 0.2T T 0.9
X1
X3
.5 .9.2
X3
X1
.5
.9.2
X3
CPT ADDDecision Diagram
.5
![Page 36: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/36.jpg)
Point-based Value Iteration
• Find a solution for a sub-set of all states• Not all states are necessarily reachable• Generalize the solution to all states• Solution methods include: PERSEUS, PBVI, and
HSVI and other similar approaches (FSVI, PEGASUS)
![Page 37: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/37.jpg)
Symbolic Perseus
• Symbolic Perseus - point-based value iteration algorithm that uses Algebraic Decision Diagrams (ADDs) as the underlying data structure to tackle large factored POMDPs
• Flat methods: 10 states at 1998, 200,000 states at 2008
• Factored methods: 50,000,000 states• http://www.cs.uwaterloo.ca/~ppoupart/
software.html#symbolic-perseus
![Page 38: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/38.jpg)
Another Example: COACH
![Page 39: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/39.jpg)
Demos
• Trial B• Trial C• Real demo
![Page 40: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/40.jpg)
Issues
• Ethics• Liability• Privacy• ??
![Page 41: Solution](https://reader035.fdocuments.in/reader035/viewer/2022062520/568163c5550346895dd4f18e/html5/thumbnails/41.jpg)
Acknowledgements
A few slides were borrowed from:• Pantelis Elinas, University of Sydney• Alex Mihailidis, University of Toronto• Guy Shani, Microsoft Research