CS2420: Lecture 11 Vladimir Kulyukin Computer Science Department Utah State University.
ShopTalkresna.stanford.edu/Capture/2007/PS9-2.pdf · 2008. 4. 1. · ShopTalk: Independent Blind...
Transcript of ShopTalkresna.stanford.edu/Capture/2007/PS9-2.pdf · 2008. 4. 1. · ShopTalk: Independent Blind...
![Page 1: ShopTalkresna.stanford.edu/Capture/2007/PS9-2.pdf · 2008. 4. 1. · ShopTalk: Independent Blind Shopping = Verbal Route Directions + Bar Code Scans. John Nicholson and Vladimir Kulyukin.](https://reader035.fdocuments.in/reader035/viewer/2022081616/5ff5d6af4678e3654b0db8e8/html5/thumbnails/1.jpg)
ShopTalk: Independent Blind Shopping =
Verbal Route Directions + Bar Code Scans
John Nicholson and Vladimir KulyukinComputer Science Assistive Technology Laboratory
Department of Computer ScienceUtah State University
![Page 2: ShopTalkresna.stanford.edu/Capture/2007/PS9-2.pdf · 2008. 4. 1. · ShopTalk: Independent Blind Shopping = Verbal Route Directions + Bar Code Scans. John Nicholson and Vladimir Kulyukin.](https://reader035.fdocuments.in/reader035/viewer/2022081616/5ff5d6af4678e3654b0db8e8/html5/thumbnails/2.jpg)
Grocery Shopping for the Visually Impaired
Grocery shopping is a difficult taskAverage supermarket has 45,000 products
Current optionsAt Store
Another person must be available
WebsitesNot all websites service local areaDelivery is not immediate
![Page 3: ShopTalkresna.stanford.edu/Capture/2007/PS9-2.pdf · 2008. 4. 1. · ShopTalk: Independent Blind Shopping = Verbal Route Directions + Bar Code Scans. John Nicholson and Vladimir Kulyukin.](https://reader035.fdocuments.in/reader035/viewer/2022081616/5ff5d6af4678e3654b0db8e8/html5/thumbnails/3.jpg)
Backpack
CPU
Keypad
Bar code
scanner
ShopTalkVerbal instructions guide userUser Input
Numeric keypadBar code scanner
Off-the-shelf (OTS) components Does not require change to store environment
![Page 4: ShopTalkresna.stanford.edu/Capture/2007/PS9-2.pdf · 2008. 4. 1. · ShopTalk: Independent Blind Shopping = Verbal Route Directions + Bar Code Scans. John Nicholson and Vladimir Kulyukin.](https://reader035.fdocuments.in/reader035/viewer/2022081616/5ff5d6af4678e3654b0db8e8/html5/thumbnails/4.jpg)
Macro-Navigation
Topological MapStore entranceEntrances to aislesCashier lane
ShopTalk provides verbal directions at user’s requestUser uses own navigation skillsScanning a bar code provides error correction
![Page 5: ShopTalkresna.stanford.edu/Capture/2007/PS9-2.pdf · 2008. 4. 1. · ShopTalk: Independent Blind Shopping = Verbal Route Directions + Bar Code Scans. John Nicholson and Vladimir Kulyukin.](https://reader035.fdocuments.in/reader035/viewer/2022081616/5ff5d6af4678e3654b0db8e8/html5/thumbnails/5.jpg)
Product Search
Database of productsIn general area of product, search for exact position of product on shelvesShopper uses bar code scannerSystem provides verbal instructions on where to scan nextWhenever a bar code is scanned, system knows where user is
![Page 6: ShopTalkresna.stanford.edu/Capture/2007/PS9-2.pdf · 2008. 4. 1. · ShopTalk: Independent Blind Shopping = Verbal Route Directions + Bar Code Scans. John Nicholson and Vladimir Kulyukin.](https://reader035.fdocuments.in/reader035/viewer/2022081616/5ff5d6af4678e3654b0db8e8/html5/thumbnails/6.jpg)
Modified Bar Code Scanner
Shelf lip
OTS bar code scanner
Extensions attached
to scanner
![Page 7: ShopTalkresna.stanford.edu/Capture/2007/PS9-2.pdf · 2008. 4. 1. · ShopTalk: Independent Blind Shopping = Verbal Route Directions + Bar Code Scans. John Nicholson and Vladimir Kulyukin.](https://reader035.fdocuments.in/reader035/viewer/2022081616/5ff5d6af4678e3654b0db8e8/html5/thumbnails/7.jpg)
Describing Product Location
AisleSide of aisleShelf sectionShelfProduct position
![Page 8: ShopTalkresna.stanford.edu/Capture/2007/PS9-2.pdf · 2008. 4. 1. · ShopTalk: Independent Blind Shopping = Verbal Route Directions + Bar Code Scans. John Nicholson and Vladimir Kulyukin.](https://reader035.fdocuments.in/reader035/viewer/2022081616/5ff5d6af4678e3654b0db8e8/html5/thumbnails/8.jpg)
Single Participant Pilot Study
1,655 shelf bar codes in one aisle of Lee's Supermarket scanned7 product sets
3 products per product set
16 runs completed At least one run for each product set
![Page 9: ShopTalkresna.stanford.edu/Capture/2007/PS9-2.pdf · 2008. 4. 1. · ShopTalk: Independent Blind Shopping = Verbal Route Directions + Bar Code Scans. John Nicholson and Vladimir Kulyukin.](https://reader035.fdocuments.in/reader035/viewer/2022081616/5ff5d6af4678e3654b0db8e8/html5/thumbnails/9.jpg)
ResultsAll runs completedAll products successfully found.Distance visually impaired participant compared favorably with sighted person walking same route.Average of 4.2 barcode scans to find individual product.
![Page 10: ShopTalkresna.stanford.edu/Capture/2007/PS9-2.pdf · 2008. 4. 1. · ShopTalk: Independent Blind Shopping = Verbal Route Directions + Bar Code Scans. John Nicholson and Vladimir Kulyukin.](https://reader035.fdocuments.in/reader035/viewer/2022081616/5ff5d6af4678e3654b0db8e8/html5/thumbnails/10.jpg)
Video: Navigating from Aisle to Aisle
![Page 11: ShopTalkresna.stanford.edu/Capture/2007/PS9-2.pdf · 2008. 4. 1. · ShopTalk: Independent Blind Shopping = Verbal Route Directions + Bar Code Scans. John Nicholson and Vladimir Kulyukin.](https://reader035.fdocuments.in/reader035/viewer/2022081616/5ff5d6af4678e3654b0db8e8/html5/thumbnails/11.jpg)
Video: Searching for a Product
![Page 12: ShopTalkresna.stanford.edu/Capture/2007/PS9-2.pdf · 2008. 4. 1. · ShopTalk: Independent Blind Shopping = Verbal Route Directions + Bar Code Scans. John Nicholson and Vladimir Kulyukin.](https://reader035.fdocuments.in/reader035/viewer/2022081616/5ff5d6af4678e3654b0db8e8/html5/thumbnails/12.jpg)
Future Work
In groceryAlready scanned two more aislesTest system with more users over more complex routes
Test outdoor navigation using only verbal instruction