Zagat Restaurant Finder Final
-
Upload
zina-saadi -
Category
Documents
-
view
31 -
download
3
Transcript of Zagat Restaurant Finder Final
Tufts University
Zagat Restaurant Finder A Speech Recognizer Application
Zina Saadi & Anila Chowdhury
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 2
TABLE OF CONTENTS
Introduction .................................................................................................................................................. 3
Project Timeline ........................................................................................................................................ 3
Automated Speech Recognition Nowadays .............................................................................................. 4
What is Zagat restaurant Finder? ............................................................................................................. 4
Meet The Team ......................................................................................................................................... 6
Major Design Specifications .......................................................................................................................... 7
Branding .................................................................................................................................................... 7
Why Is this a good design? ........................................................................................................................ 9
Project Plan ............................................................................................................................................... 9
Flow-Chart ............................................................................................................................................... 10
Sample Scripts ......................................................................................................................................... 12
Script 1 ................................................................................................................................................ 12
Script 2 ................................................................................................................................................ 13
Script 3 ................................................................................................................................................ 15
State Tables ............................................................................................................................................. 16
Technical Design ..................................................................................................................................... 27
Application Testing ..................................................................................................................................... 29
Test Plan ...................................................................................................................................................... 29
Target user(s) .......................................................................................................................................... 29
Usability challenges................................................................................................................................. 29
What we plan to test .............................................................................................................................. 30
How we plan to test ................................................................................................................................ 30
Conclusions we hope to make ................................................................................................................ 30
Test Subjects ........................................................................................................................................... 30
Ideal conditions for usability test ............................................................................................................ 31
Key Use Cases.......................................................................................................................................... 31
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 3
Test Cases we plan to run ....................................................................................................................... 31
The scale for scoring the results of the test ............................................................................................ 35
Present our test findings for developmental changes ............................................................................ 36
Compare with similar systems (1-800-GOOG-411) ................................................................................. 36
Final Usability Testing and Findings ........................................................................................................ 36
APPENDIX .................................................................................................................................................... 37
Recording Prompts.................................................................................................................................. 37
Demo Script ................................................................................................ Error! Bookmark not defined.
Survey Questionaires .............................................................................................................................. 37
Test Cases ................................................................................................... Error! Bookmark not defined.
Restaurant Information .......................................................................................................................... 39
Selected Emails Responses ..................................................................................................................... 40
INTRODUCTION
PROJECT TIMELINE
Important Dates
April 19th 2008: Initial Draft
May 5th 2008: Prompts Recording
Update Project Plan
May 6th 2008: Usability Testing
May 7th 2008: Paper wrap-up, printing & binding
May 8th 2008: Presentation Day (20 minutes)
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 4
AUTOMATED SPEECH RECOGNITION NOWADAYS
In the early 90’s, Raymond Kurzweil1 stated in his book “The Age of Intelligent Machines”2 that
automated speech recognition was receiving considerable attention because of its potential for
commercial application. One of the major goals of Artificial Intelligence in speech at that time was to
make the interaction for human with computers more natural and intuitive. Nowadays, “Speech-
Recognition technology increasingly is being used in a variety of over-the-phone applications in the
transportation financial, telecommunications and other industries.” stated by Blade Kotelly in his book
“The Art and Business of Speech Recognition”. In addition, building a speech recognition application that
provides natural and intuitive interaction had been the major focus for speech recognizer designers,
managers and companies in order to provide callers with a unique computer-caller experience. In
return, a compelling speech recognizer would increase usability, build and spread the identity of
companies, thus increase revenues, customer satisfaction and retention.
WHAT IS ZAGAT RESTAURANT FINDER?
Zagat Restaurant Finder is a speech recognizer application that allows callers to find the right quality dining place in New York City regions. The application browses, parses, retrieves and lists restaurant information from “Zagat Survey”1 at http://www.zagat.com/, which is the world’s leading provider of consumer survey-based information on where to Eat, Drink, Stay and Play worldwide. Zagat Restaurant Finder contains restaurants from over 86 different cuisines, with Italian, and American as the lead cuisines. The figure below illustrates the cuisines distribution provided by the application.
1 He is the principal developer of the world’s first print-to-speech reading machines for the blind, the first large –
vocabulary voice-activated word processor.
2 The book won the Most Outstanding Computer Science Book of 1990 award by the Association of American
Publishers.
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 5
FIGURE 1: CUISINE COVERAGE DISTRIBUTION
Callers would call to:
Retrieve restaurants details by the name of a restaurant
Learn about the top restaurants according to preferred criteria
Browse a list of restaurants by narrowing down the results
Application specification in 4 characteristics:
Happy to help people
Fun and Joyful prompts
Simpler lingo
Clearer navigation
On April 15th, 2008, Anila and Zina conducted a survey to study the importance of the Zagat Restaurant Finder among a sample of 22 subjects. Figure 1 shows the distribution of age across the subjects, with over 70% young-adults (18-30) and 23% over 30 years old.
FIGURE 1: Subjects Age Distribution
To Illustrate the need of an Automatic Speech Recognition System, the survey’s questionnaire was designed to study how often subjects would dine in a restaurant, whether they were in a situation where they needed to lookup information about a restaurant from the internet but could not, and finally whether they were in favor of an Automated Speech Recognition System that would take care of that for them (for the exact questions asked see Appendix D). The survey’s results illustrates that:
1) Interest in a Speech Recognition Application for Restaurant finder does not depend on how often people dine. In fact, the highest interest rank/score which was 9, was selected by 6 people who stated that they dine on a weekly (5 out of 6) and on every-other week (one person only) basis, whereas, 2 people who dine more frequently showed similar interest (rank of 9).
17%
16%
6%4%4%4%3%
46%
Cuisines DistributionItalian
American
French Bistro
Japanese
French
Chinese
Eclectic and International
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 6
2) There were 10 people who showed interest in a Speech Recognizer for dining with an interest rank of 7-8, which is higher that the number of people who showed total interest (rank of 9). This is an interesting result because we will observe in later section how many of these people who were selected to test the application, show higher interest in using the application in the future.
For more specific results from the initial survey, you can consult Appendix D.
Figure 2 Illustrates the Relationship between the Degrees of Preference from 0 to 9 (very interested) for using an Automated Speech Recognizer for a Restaurant Finder and how often do people dine.
MEET THE TEAM
The completion of Automated Speech Recognizer application “Zagat Restaurant Finder” wouldn’t have been possible without the efforts of Anila Chowdhury and Zina Saadi. The team brings together a combination of over 5 years of industry experience as Software Tester Engineer (STE) and as a Computational Linguist. Anila Chowdhury completed her B.S degree in Computer Science from Mount Holyoke College, MA and worked as a Software Test Engineer at Microsoft for 2 years before joining Tufts University for her Master’s degree in Computer Science. She worked extensively on Wireless Application Protocol (WAP). Service Indication/ Service Loading (SI/SL) and Segment and Reassemble (SAR) feature Items for Windows Mobile 2003 2nd edition, over-viewed test plan developed by external vendors, run test pass, performed ad-hoc testing and created Exit Criteria (EC) pass. She also owned Partner Response Team (PRT) Logo Test Kit (LTK) and coordinated 4 Windows Mobile 2003 2nd edition LTK
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 7
runs with Microsoft LTK Team and external vendor. She is currently working part-time for AAS (American Antiquarian Society) entering data for early 1800 American elections. Zina Saadi completed a double B.S in Linguistics and Dual Math and Computer Science from Northeastern University. She brings over 3 years of architectural software design and development as well as Data-Mining, Computational-Linguistics, and Information-Retrieval experience from “Basis Technology”3. She has worked on all aspects of product development, on architecture design, as technical lead, on implementing and testing code, and on technical documentation (publishing papers and delivering talks). In the past, Zina had gained some experience in acoustic signal processing and implementing Speech Recognition software to provide real-time visual reinforcement of vocalizations produced by infants4. She is currently a part-time graduate student at Tufts University and a full-time Computational Linguist at Basis Tech. Anila and Zina attended the course “Speech Recognition: The Art, Business & Technology of Voice
User Interfaces”5 offered at Tufts college of computer science and taught by Blade Kotelly and Chris
Roby. The Zagat Restaurant Finder also brings together user-interface and technical designing skills
learnt over a period of 3 months (Spring 2008).
MAJOR DESIGN SPECIFICATIONS
BRANDING
One feature of Zagat Restaurant Finder that is apparent to the caller from the very beginning of the system’s interaction are the quality of the welcome and goodbye recorded prompts, which used a female voice talent accompanied by excerpts from Mozart’s symphony No. 40 in G Minor. For prompts continuity, the same female voice was used without the background music so that the caller remain in focus with the content of the prompts for an easy understanding of the application’s lingo and for a better navigation through the system.
The female voice talent used in the branding of Zagat Restaurant Finder is the voice of Melissa Lucius.
Melissa is the voice talent used by Basis Technology’s multi-Lingual’s Automated Speech Recognizer. In
addition to having voice talent skills, Melissa has singing skills in many languages she is fluent with
including: French, German, and Italian. Melissa’s voice was not only used to speak out the designed
prompts of the system, but it was used to speak out a list of the top 10 French, American, German, and
French restaurants, in addition to some top Arabic restaurants names transcribed by Zina Saadi for
3 Basis provides software solutions for extracting meaningful intelligence from unstructured text in many human
languages including Asian, European and Middle Eastern Languages. http://www.basistech.com
4 Visibable project at Northeastern University: http://www.neu.edu/nupr/news/0107/visibabble.html
5 Website: http://www.cs.tufts.edu/comp/150SR/
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 8
pronunciation easiness. Melissa’s voice was also used to speak out phrases and names that would
contain accents (such as the accents used in French and Italian) to add a uniqueness to the system.
There were some careful decisions that were taken for the selection of the background music used at
the welcome and goodbye prompts. Taking in consideration the fact that Zagat Restaurant Finder is
there for callers who are confused about selecting a particular restaurant for dining, callers who are
looking for specific details of a particular restaurant they have in mind, or callers who want to find a
particular restaurant that fits one of many criteria such as cuisine type, location, or certain ratings that
are provided by the Zagat Survey website. Upon deciding to call Zagat Restaurant Finder, callers would
be optimistic since they are calling to get helped with deciding which restaurant to choose. In reflection
to this context, the symphony No. 40 in G Minor of Wolfgang Amadeus Mozart (1756-1791) comes into
the picture. This piece of music was written by Mozart in 1788 despite the fact that it was written at the
time when he was poorer than ever and suffering from the death of his six-month old daughter
“Theresa”, to illustrate the fact that Mozart was maintaining “a zest for life and unwavering optimism”6.
The fact that Mozart wrote this piece of music to escape his troubled personal life and be optimistic for
something better or happier reflects the fact that callers of Zagat Restaurant Finder are troubled by not
being able to decide on where to dine and would feel optimistic to call the application for help and for a
happy satisfaction.
Multi-cultural music was chosen as a background sounds in the system. The sound piece selection would
happen randomly and according to the cuisine type if it is specified, otherwise it plays classical music by
default. The table below gives more insight regarding the selection of the background music files.
Cuisine Type Song Title Comments
French “La Vie en Rose” Song by Edith Piaf, used 3 sound
files
Indian Chaiyya Sound track from the movie
“Inside Man”, used 3 sound files
Italian “Lasciate Mi Cantare” Song title translates to English as
“let me sing”, used 5 files. An
alternative name of the song is
“L’Italiano”
Spanish Bamboleo Song by Gypsy Kings, used 5 files
Egyptian Ah Ya Shoo’ Used 2 files
German Und Wenn ein Lied Song by Xavier Nadoo, used 3
files
Russian Katusha Russian Famous Folklore Song,
6 From page 7 in the booklet of “In Classical Mood” CD’s titled “Morning Mist”
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 9
used 2 files
Chinese Yung Chun Baixue Classical Song, used 5 files
Tunisian Avava Ynouva Berber Song From North Africa,
used 4 files
Irish Firedance By Daniel Ash, used 3 files
Morroccan Index Album Track 8 By Group Index (Algerian group),
used 3 files
Persian Bahar Used 2 files
WHY IS THIS A GOOD DESIGN?
The design of the Zagat Restaurant Finder application is a good design because it fits most of the 10
principles of good design for DIETER RAMS7:
1. It is a double-edge sword since it combines two of the important fields in Artificial Intelligence’s world, namely the automatic speech recognition field and the text processing/data mining field, which makes the system Innovative. 2. Makes the product useful since it satisfies the needs of people. About 90% of 22 people who were passed a survey on how likely they find a Speech Recognizer Application useful for Restaurant lookup, were in favor of using such application. This also makes the design more concerned with the environment since it copes with the current lifestyle of people which is characterized by a busy lifestyle, and the wide usage of wireless communication technologies including cell phones and text messaging. Zagat Restaurant Finder connects callers with people who had evaluated restaurants and shared their opinion on Zagat-Survey anytime, and anywhere throughout the Boston and New York regions. 4. It is as little design as possible since it uses the ideas of polymorphic states and dynamic retrieval. 5. It is durable since the system can be functional even if the Zagat-Survey website is down.
PROJECT PLAN
7 http://gizmodo.com/343641/1960s-braun-products-hold-the-secrets-to-apples-future
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 10
The completion of Zagat Restaurant Finder wouldn’t have been possible without a project plan
(often referred to as the schedule of a project). The project plan measures a day by 8 working
hours. The image below, illustrates the project plan created for Zagat Restaurant Finder:
FIGURE 2 PROJECT PLAN/SCHEDULE
FLOW-CHART
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 11
1000
Welcome
2000
Main_Menu
6000
zagat_top_list
8000
resturant_info
9000
menu_options
9999
Goodbye
“Goodbye”
<restaurant_name>
<cuisine>
<neighborhood>
“go back”
“main menu" <restaurant_name>
2500
Find_by_options
5000
Search_Results
7000
narrow_options
“Narrow Them”
“food rating”
“décor rating”
“service rating”
“price rating”
7500
narrow_values
<rating_value>
8500
restaurant_details
“I’m done”
8200
rest_info_menu
“address”
“phone”
“ratings”
“description”
5500
List_All
<restaurant_name>
result.size<25
“List All”“Top Zagat”
“cuisine” “neighborhood”
3000
Cuisine
4000
Neighborhood
<cuisine_name> <neighborhood_name>
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 12
SAMPLE SCRIPTS
SCRIPT 1
User Prompts Notes
System Welcome to Zagat Restaurant Finder, where we recommend and
provide information about New York City Restaurants.
Welcome prompt, played
only once
System What is the name your restaurant?
If you don’t know, you can also say a cuisine type (for example:
French) or a neighborhood name (for example: Astoria)
Main Menu
Caller Heidelberg
System Heidelberg is a German restaurant located at East 80s
neighborhood.
Voice of Katarina
(German)
System What would you like to hear? You can say: Address. Phone
number. Ratings. Description. or I'm Done.
Caller Address
System The restaurant is located at 1648 Second Ave. New York, NY,
10028.
System What would you like to hear? You can say: Address. Phone
number. Ratings. Description. or I'm Done.
Caller Phone number
System Their Phone Number is (212) 628-2332.
System Would you like me to transfer this call to the restaurant?
Caller No You can say yes, and get
connected if you want!
System What would you like to hear? You can say: Address. Phone
number. Ratings. Description. or I'm Done.
Caller I’m done
System What would you like to do next? You can say, Go back or Main
Menu. To exit the system, say Goodbye.
Caller Goodbye
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 13
System Thank you for calling Zagat Restaurant Finder. Goodbye! End of session
SCRIPT 2
User Prompts Notes
System Welcome to Zagat Restaurant Finder, where we recommend and
provide information about New York City Restaurants.
Welcome prompt, played
only once
System What is the name your restaurant?
If you don’t know, you can also say a cuisine type (for example:
French) or a neighborhood name (for example: Astoria)
Main Menu
Caller French
System I have found 203 restaurants that matched your search criteria.
System To narrow down your search results, say “Narrow Down”. To
hear top zagat rated restaurants, say “Top Zagat”.
“List All” will not be
available until result size <
25
Caller Narrow Down
System How would you like to narrow it down? You can select
Neighborhood, Food Rating, Décor Rating, Service Rating, Price
Rating, to enter new values or change Cuisine.
Caller Price Rating
System In the Price rating, I've got restaurants that have price range
between 0 (on the low end) and 86 (on the high end.) What's
the maximum Price you want to pay?
Caller 15
System I have found 16 restaurants that matched your search criteria.
System To narrow down your search results, say “Narrow Down”. To
hear top zagat rated restaurants, say “Top Zagat”. To hear the
entire list alphabetically, say “List All”.
List All option opens up if
list <25
Caller List All
System Ok I will list the restaurants….. When you hear the name you
want, just say it.
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 14
System ALAIN DUCASSE . Brasserie eight and a half . Bodega .
CHANTERELLE . Cafe Joul
Voice of Julie (French)
System Say next to hear the next 5, repeat to hear the previous 5. You
can also say start over to hear the list from the beginning.
Caller Next
System Chazal . Gaby Brasserie Francaise . LA GRENOUILLE . La
Bicyclette . Madison Bistro
System Say Repeat, next or start over
Caller Madison Bistro
System You selected “Madison Bistro”. Is that correct? Explicit confirmation
Caller Yes
System Madison Bistro is a French restaurant located at Murray Hill ,
East thirty neighborhood.
Voice of Julie (French)
System What would you like to hear? You can say: Address. Phone
number. Ratings. Description. or I'm Done.
Caller Address
System The restaurant is located at 238 Madison Ave. New York, NY,
10016 2816.
System What would you like to hear? You can say: Address. Phone
number. Ratings. Description. or I'm Done.
Caller Description
System Dim lights and terra cotta toned walls create an intime
ambiance at this newly renovated Murray Hill French; early
reports on its menu of bistro classics are positive.
System What would you like to hear? You can say: Address. Phone
number. Ratings. Description. or I'm Done.
Caller I’m done
System What would you like to do next? You can say, Go back or Main
Menu. To exit the system, say Goodbye
Caller Goodbye
System Thank you for calling Zagat Restaurant Finder. Goodbye! End of session
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 15
SCRIPT 3
User Prompts Notes
System Welcome to Zagat Restaurant Finder, where we recommend and
provide information about New York City Restaurants.
Welcome prompt, played
only once
System What is the name your restaurant?
If you don’t know, you can also say a cuisine type (for example:
French) or a neighborhood name (for example: Astoria)
Main Menu
Caller Astoria
System I have found 19 restaurants that matched your search criteria.
System To narrow down your search results, say “Narrow Down”. To
hear top zagat rated restaurants, say “Top Zagat”. To hear the
entire list alphabetically, say “List All”.
Caller Top Zagat
System Listing Top Zagat Restaurants… When you hear the name you
want, just say it.
System Ponticello . Esperides . SAgapo . Piccola Venezia . Christos
Hasapo Taverna . Karyatis . Taverna Vraka . Tellys Taverna .
Trattoria Lincontro
Caller Ponticello
System You selected “Ponticello”. Is that correct?
Caller Yes
System Ponticello is an Italian restaurant located at Astoria
neighborhood.
Voice of Jennifer (Italian)
System What would you like to hear? You can say: Address. Phone
number. Ratings. Description. or I'm Done.
Caller Phone Number
System Their Phone Number is (718) 278-4514.
System Would you like me to transfer this call to the restaurant?
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 16
Caller No You can say “Yes”, if you
want
System What would you like to hear? You can say: Address. Phone
number. Ratings. Description. or I'm Done.
Caller I’m done
System What would you like to do next? You can say, Go back or Main
Menu. To exit the system, say Goodbye
Caller Goodbye
System Thank you for calling Zagat Restaurant Finder. Goodbye! End of session
STATE TABLES
1000_Welcome
Play Prompt
Entering from
Program starts here
Prompts Name Condition Wording
10001 always Welcome to Zagat Restaurant Finder, where we recommend and provide information about New York City Restaurants.
Condition Action
Always Go to: 2000_Main_Menu
Module Settings
Default
2000_Main_Menu Speech Input
Entering from
1000_Welcome
Prompts Type Name Wording
Initial 20001 What is the name your restaurant?
If you don’t know, you can also say a cuisine type (for example: French) or a neighborhood name (for example: Astoria)
Timeout 1 20002 I'm sorry, I didn't hear you.
Timeout 2 20003 I'm sorry, I still didn't hear you. Say the name of your restaurant. or a cuisine type. or a neighborhood name.
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 17
Retry 1 20004 I'm sorry, I didn't understand.
Retry 2 20005 I'm sorry, I still didn't understand you. Would you try one more time? Say help if you need more information.
Help 20006 In order to find a restaurant I need to know how would you like to search for it.
If you would like to search for a restaurant by it’s name, say the name
If you would like to search restaurants for a specific type of cuisine for example: French, say the cuisine name.
If you would like to search restaurants for a specific of neighborhood for example: Astoria, say the neighborhood name.
Option Vocabulary DTMF Action Confirm.
<restaurant_name> Go to: 8000_restInfo If necessary
<cuisine_name> Go to: 5000_Search_Results If necessary
<neighborhood_name> Go to: 5000_Search_Results If necessary
Commands
Module Settings
Default
3000_ Cuisine Speech Input
Entering from
5000_Search_Results
Prompts Type Name Wording
Initial 41001 What type of Cuisine? To hear our list of cuisines say “List”
Timeout 1 41002 I’m sorry. I did not hear that.
Timeout 2 41003 I’m sorry. I still did not hear that. Please tell me a type of cuisine, for example: Chinese, American, French.
Retry 1 41004 I’m sorry. I did not understand you.
Retry 2 41005 I’m sorry. I still did not understand you. Would you try one more time? Or else say help, if you need more information.
Help 41006 Tell me the name of a Cuisine and I will list restaurants of that type. To hear our list of cuisines say “List”.
Option Vocabulary DTMF Action Confirm.
<cuisine_name> Go to: 3500_Confirmation If necessary
Commands
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 18
Module Settings
Default
3500_ Confirmation Speech Input
Entering from
3000_cuisine, 4000_neighborhood, 5500_list_all, 5600_list_all_options, 6000_top_list
Prompts Type Name Wording
Initial 41001 What type of Cuisine? To hear our list of cuisines say “List”
Timeout 1 41002 I’m sorry. I did not hear that.
Timeout 2 41003 I’m sorry. I still did not hear that. Please tell me a type of cuisine, for example: Chinese, American, French.
Retry 1 41004 I’m sorry. I did not understand you.
Retry 2 41005 I’m sorry. I still did not understand you. Would you try one more time? Or else say help, if you need more information.
Help 41006 Tell me the name of a Cuisine and I will list restaurants of that type. To hear our list of cuisines say “List”.
Option Vocabulary DTMF Action Confirm.
If entered from 3000_cuisine
<cuisine_name> Go to: 5000_Search_Result If necessary
If entered from 4000_neighborhood
<neighborhood_
name> Go to: 5000_Search_Result If necessary
If entered from 5500_list_all, 5600_list_all_options, 6000_top_list
<restaurant_na
me> Go to: 8000_rest_info If necessary
Commands
Module Settings
Default
4000_Neighborhood Speech Input
Entering from
5000_Search_Results
Prompts Type Name Wording
Initial 42001 What Neighborhood? To hear our list of neighborhoods say “List”
Timeout 1 42002 I’m sorry. I did not hear that.
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 19
Timeout 2 42003 I’m sorry. I still did not hear that. Please tell me the name of a neighborhood, for example: Astoria, Brooklyn, Staten Island.
Retry 1 42004 I’m sorry. I did not understand you.
Retry 2 42005 I’m sorry. I still did not understand you. Would you try one more time? Or else say help, if you need more information.
Help 42006 Tell me the name of a Neighborhood and I will list restaurants that are located near that area. To hear our list of neighborhoods say “List”.
Option Vocabulary DTMF Action Confirm.
<neighborhood_name> Go to: 3500_Confirmation If necessary
Commands
Module Settings
Default
5000_Search_Results
Play Prompt
Entering from
2000_main_menu, 3000_cuisine, 4000_neighborhood, 7000_narrow_list
Prompts Name Condition Wording
50001 always I have found <list_size> restaurants that matched your search criteria.
Condition Action
If <list_size> < 7 Go to: 5500_List_All
If <list_size> > 7 Go to: 5200_Search_Result_Options
Module Settings
Default
5200_Search_Result_options Speech Input
Entering from
5000_Search_Results
Prompts Type Name Wording
If <list_size> < 25 50001 To narrow down your search results, say “Narrow Down”. To hear top zagat rated restaurants, say “Top Zagat”. To hear the entire list alphabetically, say “List All”.
If <list_size> > 25 50002 To narrow down your search results, say “Narrow Down”. To hear top zagat rated restaurants, say “Top Zagat”.
If <list_size>==0 50003 Lets go back and select new options to narrow down your search.
Timeout 1 50004 I’m sorry. I did not hear that.
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 20
Timeout 2 50005 I’m sorry. I still did not hear that. Please select an option from “List All”, “Top Zagat” or “Narrow Down”
Retry 1 50006 I’m sorry. I did not understand you.
Retry 2 50007 I’m sorry. I still did not understand you. Would you try one more time? Or else say help, if you need more information.
Help 50008 Select an option to hear the list.
If you would like to hear the Top Zagat Rated restaurants from the list alphabetically say “Top Zagat”
If you would like to narrow down your search results say “Narrow Down”
Option Vocabulary DTMF Action Confirm.
“List All” Go to: 5500_List_All If necessary
“Top Zagat” Go to: 6000_Top_List If necessary
“Narrow Down” Go to: 7000_Narrow_List If necessary
Commands
Module Settings
Default
5500_List_All Speech Input
Entering from
5000_Search_Results
Prompts Type Name Wording
Initial 55001 Ok I will list the restaurants….. When you hear the name you want, just say it.
<restaurant_name_1>, <restaurant_name_2>, ……. ,<restaurant_name_n>
Timeout 1 55002 I’m sorry. I did not hear that.
Timeout 2 55003 I’m sorry. I still did not hear that. Please select a restaurant to hear more information.
Retry 1 55004 I’m sorry. I did not understand you.
Retry 2 55005 I’m sorry. I still did not understand you. Would you try one more time? Or else say help, if you need more information.
Help 55006 Select a restaurant to hear more information about it. You can also say “start over” to hear the list from the beginning.
Option Vocabulary DTMF Action Confirm.
<restaurant_name> Go to: 3500_confirmation If necessary
“start over” Go to: 5500_List_All
Commands
Module Settings
Default
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 21
5600_List_All_Options Speech Input
Entering from
5500_List_All
Prompts Type Name Wording
Initial prompt, If result_size>7
56001 Say next to hear the next 5, repeat to hear the previous 5. You can also say start over to hear the list from the beginning.
If not initial prompt 56002 Say Next, Repeat or Start Over.
Initial prompt, if result_size< 7
56003 Say repeat to hear the previous names.
Timeout 1 56004 I’m sorry. I did not hear that.
Timeout 2 56005 I’m sorry. I still did not hear that. Please select a restaurant, or say next, start over.
Retry 1 56006 I’m sorry. I did not understand you.
Retry 2 56007 I’m sorry. I still did not understand you. Would you try one more time? Or else say help, if you need more information.
Help 56008 Select a restaurant to hear more information about it. You can also say “start over” to hear the list from the beginning.
Option Vocabulary DTMF Action Confirm.
<restaurant_name> Go to: 3500_confirmation If necessary
“start over” Go to: 5600_List_All If necessary
Commands
Module Settings
Default
6000_Top_List Speech Input
Entering from
5000_Search_Results
Prompts Type Name Wording
Initial 60001 Listing Top Zagat Restaurants… When you hear the name you want, just say it.
<restaurant_name_1>, <restaurant_name_2>…… <restaurant_name_n>
Timeout 1 60002 I’m sorry. I did not hear that.
Timeout 2 60003 I’m sorry. I still did not hear that. Please select a restaurant to hear more information.
Retry 1 60004 I’m sorry. I did not understand you.
Retry 2 60005 I’m sorry. I still did not understand you. Would you try one more time? Or else say help, if you need more information.
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 22
Help 60006 Select a restaurant to hear more information about it. You can also say “start over” to hear the list from the beginning.
Option Vocabulary DTMF Action Confirm.
<restaurant_name> Go to: 3500_confirmation If necessary
“start over” Go to: 6000_Top_List
Commands
Module Settings
Default
7000_Narrow_Options Speech Input
Entering from
5000_Search_Results
Prompts Type Name Wording
Initial 70001 How would you like to narrow it down? You can select <list_1> to enter new values or change <list_2>
Timeout 1 70002 I’m sorry. I did not hear that.
Timeout 2 70003 I’m sorry. I still did not hear that. Please select one of the options: Cuisine, Neighborhood, Food Rating, Décor Rating, Service Rating, Or Price Rating.
Retry 1 70004 I’m sorry. I did not understand you.
Retry 2 70005 I’m sorry. I still did not understand you. Would you try one more time? Or else say help, if you need more information.
Help 70006 You will need to select an option to narrow down your search.
To narrow your search based on a specific cuisine type say “Cuisine”
To narrow your search based on a specific neighborhood say “Neighborhood”
To narrow your search based on Food quality say “Food Rating”
To narrow your search based on Decor quality say “Decor Rating”
To narrow your search based on Service quality say “Service Rating”
To narrow your search based on a Price range, say “Price Rating”
Option Vocabulary DTMF Action Confirm.
“Food Rating” Go to: 7500_Narrow_Values If necessary
“Décor Rating” Go to: 7500_Narrow_Values If necessary
“Service Rating” Go to: 7500_Narrow_Values If necessary
“Price Rating” Go to: 7500_Narrow_Values If necessary
“Cuisine” Go to: 3000_Cuisine If necessary
“Neighborhood” Go to: 4000_Neighborhood If necessary
Commands
Module Settings
Default
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 23
7500_Narrow_Values Speech Input
Entering from
7000_Narrow_Options
Prompts Type Name Wording
If caller selects “Food Rating”
75001 In the Food rating, I've got restaurants that rate between <var_food1> (on the low end) and <var_food2> (on the high end.) What's the minimum Food score you want to use?
If caller selects “Décor Rating”
75002 In the Decor rating, I've got restaurants that rate between <var_decor1> (on the low end) and <var_decor2> (on the high end.) What's the minimum Décor score you want to use?
If caller selects “Service Rating”
75003 In the Service rating, I've got restaurants that rate between <var_service1> (on the low end) and <var_service2> (on the high end.) What's the minimum Service score you want to use?
If caller selects “Price Rating”
75004 In the Price rating, I've got restaurants that have price range between <var_price1> (on the low end) and <var_price2> (on the high end.) What's the maximum Price you want to pay?
Timeout 1 75005 I’m sorry. I did not hear that.
Timeout 2 75006 I’m sorry. I still did not hear that. Please enter a value for your selected rating.
Retry 1 75007 I’m sorry. I did not understand you.
Retry 2 75008 I’m sorry. I still did not understand you. Would you try one more time? Or else say help, if you need more information.
Help 75009 Please enter a value to narrow down your search based on your selected rating.
Option Vocabulary DTMF Action Confirm.
<narrow_value> Go to: 7500_Narrow_List If necessary
Commands
Module Settings
Default
8000_Restaurant_Info
Play Prompt
Entering from
4000_Name, 5000_Search_Results, 6000_Top_List, 7000_Narrow_List
Prompts Name Condition Wording
80001 always <restaurant_name> is an <cuisine_name> restaurant located at <neighborhood_name> neighborhood.
Condition Action
Always Go to: 8200_Restaurant_Info_Menu
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 24
Module Settings
Default
8200_Restaurant_Info_Menu Speech Input
Entering from
8000_Restaurant_Info
Prompts Type Name Wording
Initial 82001 What would you like to hear? You can say: Address. Phone number. Ratings. Description. Or I’m one.
Timeout 1 82002 I’m sorry. I did not hear that.
Timeout 2 82003 I’m sorry. I still did not hear that. To hear more information about the restaurant, please select one of the followings: Address. Phone number. Ratings. Description. Or I am done.
Retry 1 83004 I’m sorry. I did not understand you.
Retry 2 84005 I’m sorry. I still did not understand you. Would you try one more time? Or else say help, if you need more information.
Help 85006 I can provide you the Address, Phone number, Ratings, Description, of the restaurant. Please select of the options.
Option Vocabulary DTMF Action Confirm.
“Address” Go to: 8500_restaurant_address If necessary
“Phone number” Go to: 8600_restaurant_phone If necessary
“Ratings” Go to: 8700_restaurant_ratings If necessary
“Description” Go to: 8800_restaurant_description If necessary
“I am done” Go to: 9000_Menu_Options
Commands
Module Settings
Default
8500_Restaurant_address
Play Prompt
Entering from
8200_Restaurant_Info_Menu
Prompts Name Condition Wording
85001 always The restaurant is located at <restaurant_address>
Condition Action
Always Go to: 8200_Restaurant_Info_Menu
Module Settings
Default
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 25
8600_Restaurant_Phone Speech Input
Entering from
8200_Restaurant_Info_Menu
Prompts Type Name Wording
Initial 86001 Their Phone number is <restaurant_phone>.
Would you like me to transfer this call to the restaurant?
Timeout 1 86002 I’m sorry. I did not hear that.
Timeout 2 86003 I’m sorry. I still did not hear that. Please say yes or no.
Retry 1 86004 I’m sorry. I did not understand you.
Retry 2 86005 I’m sorry. I still did not understand you. Would you try one more time? Or else say help, if you need more information.
Help 86006 I can connect this call to the restaurant and this session will still be active and can resume after you end your call with the restaurant. Would you like to do that?
Option Vocabulary DTMF Action Confirm.
“Yes” Transfer call to <restaurant_phone> If necessary
“No” Go to: 8200_Restaurant_Info_Menu If necessary
Commands
Module Settings
Default
8700_Restaurant_Ratings
Play Prompt
Entering from
8200_Restaurant_Info_Menu
Prompts Name Condition Wording
87001 always Food Rating is <food_rating>
Decor Rating is <décor_rating]>
Service Rating is <service_rating>
Price Rating is <price_rating>
Condition Action
Always Go to: 8200_Restaurant_Info_Menu
Module Settings
Default
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 26
8800_Restaurant_Description
Play Prompt
Entering from
8200_Restaurant_Info_Menu
Prompts Name Condition Wording
88001 always <restaurant_description>
Condition Action
Always Go to: 8200_Restaurant_Info_Menu
Module Settings
Default
9000_Menu_Options Speech Input
Entering from
8200_Restaurant_Info_Menu
Prompts Type Name Wording
Initial 90001 What would you like to do next? You can say Go back or Main Menu. To exit the system, say Goodbye!
Timeout 1 90002 I’m sorry. I did not hear that.
Timeout 2 90003 I’m sorry. I still did not hear that. Please say Go back. Main Menu Or Goodbye.
Retry 1 90004 I’m sorry. I did not understand you.
Retry 2 90005 I’m sorry. I still did not understand you. Would you try one more time? Or else say help, if you need more information.
Help 90006 To hear Address, Phone Number, Ratings and Description of the restaurant say Go Back.
If you would like to search for another restaurant say “Main Menu”.
Or say Goodbye to exit the system.
Option Vocabulary DTMF Action Confirm.
“Go back” Go to: 8200_Restaurant_Info_Menu If necessary
“Main Menu” Go to: 2000_Main_Menu If necessary
“Goodbye” Go to: 9999_Goodbye If necessary
Commands
Module Settings
Default
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 27
9999_Goodbye
Play Prompt
Entering from
9000_Menu_Options
Prompts Name Condition Wording
99991 always Thank you for calling Zagat Restaurant Finder. Goodbye!
Condition Action
Always Exit
Module Settings
Default
TECHNICAL DESIGN
The major workloads for building the Zagat Restaurant Finder Application were partitioned over the
team members based on their individual skills, willingness to accomplish the task and the overall
grasp of the major design and implementation components taught throughout the course. In
addition to workloads divisions, each team member reviewed the work of the other team member.
The table below illustrates the major workloads partition:
Project Component Component Overview Team Member
Scripts, Flow-Chart,
State Tables
Initially designed the scripts and flow charts and
finalized the implementation as summarized in the
state tables. The design was then reviewed by Zina.
Anila Chowdhury
Technical Design Met with Anila, created a technical plan and
summarized the technical design.
Zina Saadi
User-Interface
Implementation
Workind on the cgi/vxml part as well as added some
cool functionality such as confirm! Or connecting to
the restaurant.
Anila Chowdhury
Back-End
Implementation
Wrote ruby code for parsing xml, python code for
Unicode conversion, after data extraction, data were
stored in pstore for a while until bugs were raised in
using pstore, then the data were hardcoded in a
ruby file in a hash format
Zina Saadi
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 28
Creating Test Cases Created test cases that address each feature of the
application.
Anila Chowdhury
Merging Together UI
and Back-End
Implementations
Worked on merging both the front end and backend
application together and fixed most bugs, plus added
some ruby functionality.
Zina Saadi
Recording Prompts Met with the voice talent: Melissa Lucius then used
audacity to segment voice files as well as
background music.
Zina Saadi
Final Report Put together information and project
accomplishments including: diagrams, statistical
charts, images, tables and summarized paragraphs.
Zina Saadi
As for the Technical design implementation, there were 3 major components that were taken in
consideration when the load of work was divided; these 3 components are illustrated as: the Front-End
components, Back-End components and the Merging module responsible for connecting both
components together. The Figure below, illustrated the main sub-components used to implement each
of these 3 major components:
FIGURE 3 APPLICATION OVERVIEW
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 29
In summary, Coding responsibility was divided between Anila and Zina. Anila took care of implementing
the Front-End Structure. She did the cgi scripts and Zina did the xml data extraction. Anila created 9 cgi
files that contained all the vxml generation codes, passing parameters and calling Zina's functions. The
basic flow of the vxml can be found from the flowchart. All the cgi contains customized help, noinput,
nomatch and explicit/ implicit confirmation (for the most relevant ones). 2000_main_menu contains a
grammar that can recognize what the caller said and associate a type with it as well (name, cuisine or
neighborhood). 5000_searchResult returns a list o restaurant names, and depending on the length of
the list different options are available. If the size of result is less than 1 then the caller is asked to change
their narrow options. If the size is 7 or less than system just reads the entire list to the caller, and if
above 7, the caller has the option to narrow down and if above 10, listen to top Zagat rated restaurants.
The navigation through a list was designed in a different way, where 5 results were given to the caller at
a time. Each time a caller had the options to select, or say next, repeat or start over. This gave the caller
control of the list and made it simpler and user friendly. 7000_narrowList had boundary checking
conditions every time a caller entered a value to narrow down his search according to his choice of
rating (food, decor, price etc). 8000_restInfo had structures to save values extracted from the xml, and
the cgi could transfer a call to the restaurant if the caller wants to. Beautiful music was chosen by Zina
throughout the application to accompany Anila's vxml files. As for the Back-End Structure, Zina used
python to convert accents marks utilized in French, Spanish and Italian, then she used XPath from the
REXML library in ruby to extract the data from the xml file sent by the professor of the course Blade
Kotelly. Since XPAtH is basically a DOM Parser and the xml file was not in a tree structure format, it took
1:00 to extract the entire needed data in a hash that was then saved in pstore, which gets called by
<restaurantDataInterface.rb> to create the hash that is called by all the functions needed by the cgi files
that Anila created.
APPLICATION TESTING
TEST PLAN
TARGET USER(S) The target users for our application are regular web or phone users. Anyone from beginning level/
laggard users to advanced can use this application. The target audience’s age would range from 21-55.
The frequency of their use would depend on the number of times they go out to eat at restaurants.
USABILITY CHALLENGES i. How similar the test prototype experience is to real application experience.
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 30
ii. How clear and intuitive each tasks are (adding cuisines, narrowing down by food ratings or price range)
iii. List restaurants and give options to navigate through it efficiently iv. Ask minimum number of questions and collect the right information v. Compare it with other available restaurant recommendation phone application
WHAT WE PLAN TO TEST How well test subjects understand the features (can they identify what it does?)
Test basic functionalities like find a restaurant by name, by cuisine or narrow down search results according to food or price ratings
How easy is the application to learn
How smooth is the navigation between different states
How clearly they can remember the steps to accomplish different tasks
How well they can use the universal commands
How helpful the noinput, nomatch and help commands are
HOW WE PLAN TO TEST We initially did a “wizard of oz” testing with a subject where one of us pretended to be the system and
the subject was the caller. Sample scripts were used for the test, and attached to the appendix section
of this report.
We decided to have 2 phases of testing for Zagat Phone Line Application. Our pilot testing happened
with a small prototype, which has basic functionalities like finding a restaurant by cuisine, by
neighborhood etc implemented, and a much smaller data. Subjects called the system, and tried to
accomplish some tasks pre-defined by us and give us feedback. We also used the be-vocal log files to
analyze the data.
Once we got the initial feedback on the working prototype, we will fine-tune the design and implement
all the functions (top restaurants, checking boundaries, number of results, navigation through a list etc).
The final version will include a full implementation of all the design we proposed in our initial plan, play
all the audio files as prompts, and run with all the restaurant data. We hope to run extensive usability
tests and fix high priority and high frequency errors in the system.
CONCLUSIONS WE HOPE TO MAKE i. How user friendly the system is
ii. How easily the tasks can be completed iii. Can the user find all the commands/ features? iv. Can the user remember the functionalities?
TEST SUBJECTS For this project, we are planning to ask our friends and class members to test this application. We can
also ask students of our departments to take part in this activity.
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 31
IDEAL CONDITIONS FOR USABILITY TEST i. Relaxed environment (in a room)
ii. Driving a car iii. If possible subjects can make notes and time each task
KEY USE CASES i. Launch application
ii. Find a restaurant by Name (from main menu) iii. Find restaurants by Cuisine (from main menu) iv. Find restaurants by Neighborhood (from main menu) v. Navigate through a List of restaurants
vi. Listen to Top Zagat restaurant List vii. Narrow a list of restaurants by Food Rating
viii. Narrow a list of restaurants by Decor Rating ix. Narrow a list of restaurants by Service Rating x. Narrow a list of restaurants by Price Rating
xi. Narrow a list of restaurants by Neighborhood xii. Narrow a list of restaurants by Cuisine
xiii. Select a Restaurant from a list of restaurants xiv. Listen to the Address of a selected restaurant xv. Listen to the Phone No of a selected restaurant
xvi. Listen to the Ratings of a selected restaurant xvii. Listen to the Description of a selected restaurant
xviii. Get connected to the restaurant over the phone xix. Navigate to Main Menu from different states xx. Repeat information whenever desired
TEST CASES WE PLAN TO RUN i. Launch Application (call bevocal)
a. Dial 1-877-338-6225 b. Enter Pin c. Application starts Success: Welcome message is played
Listed: “Welcome to Zagat Restaurant Finder, where we recommend and provide information
about New York City Restaurants.”
ii. Find a restaurant by Name a. Launch Application b. Say “jo jo” as the name of the restaurant Success: application recognizes the restaurant
Hear: “Jo Jo is a French Bistro located at East sixty neighborhood.”
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 32
iii. Find restaurants by Cuisine a. Launch Application b. Say “German” as the cuisine Success: application recognizes the cuisine and returns a list of restaurant names.
Listed: “Gebhardt's, Hallo Berlin, Heidelberg, Killmeyer's, Niederstein's, Rolf's, Zum Stammtisch”
iv. Find restaurants by Neighborhood a. Launch Application b. Say “Bronx” as the neighborhood Success: application recognizes the neighborhood and returns a list of restaurant names.
Listen: “Bellavista Café, Dominicks, Enzos, Jimmys Bronx Café, Le Refuge Inn,
Lobster Box, Marios, Mona Lisa Gourmet Pizza, Robertos, Starbucks”
v. Navigate through a List of restaurants a. Launch Application b. Say “German” as the cuisine c. Application returns a list of restaurants Success: application correctly lists the restaurants and gives caller the options to say repeat or
next, and select a restaurant.
Listen: “Gebhardt's, Hallo Berlin, Heidelberg, Killmeyer's, Niederstein's.
Say next to hear the next 5, repeat to hear the previous 5. You can also say start over to hear the
list from the beginning.”
vi. Create Top Zagat restaurant List using criteria defined by the user a. Launch Application b. Say “Astoria” as the neighborhood c. Application returns a list of restaurants d. Select “Top List” Success: application lists the top Zagat rated restaurants and gives caller the options to say
repeat or next, and select a restaurant.
Listen: “one hundred and one, St Michel, Baci Italian Restaurant, Embers, Tuscany Grill, Chianti,
Chadwicks, Elia, Areo”
vii. Narrow a list of restaurants by Food Rating a. Launch Application
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 33
b. Say “Italian” as the cuisine c. Application returns a list of restaurants d. Select “Narrow Them” e. Select “Food Rating” f. Say “26” Success: application creates a narrowed list of restaurants using neighborhood and food rating
criteria
Listen: “BABBO, IL GIGLIO, IL MULINO, SCALINI FEDELI, Trattoria RomanaTrattoria Lincontro.”
viii. Narrow a list of restaurants by Decor Rating a. Launch Application b. Say “Italian” as the cuisine c. Application returns a list of restaurants d. Select “Narrow Them” e. Select “Decor Rating” f. Say “25” Success: application creates a narrowed list of restaurants using neighborhood and décor rating
criteria
Listen: “Palio, RAINBOW ROOM, SCALINI FEDELI.”
ix. Narrow a list of restaurants by Service Rating a. Launch Application b. Say “Italian” as the cuisine c. Application returns a list of restaurants d. Select “Narrow Them” e. Select “Service Rating” f. Say “24” Success: application creates a narrowed list of restaurants using neighborhood and service
rating criteria
Listen: “BABBO, Erminia, Rughetta, SCALINI FEDELI, Trattoria Lincontro”
x. Narrow a list of restaurants by Price Rating a. Launch Application b. Say “thai” as the cuisine c. Application returns a list of restaurants d. Select “Narrow Them” e. Select “Price Rating” f. Say “20” Success: application creates a narrowed list of restaurants using neighborhood and price rating
criteria
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 34
Listen: “Pad Thai Noodle Lounge, Rungsit Thai, Vynl”
xi. Narrow a list of restaurants by Cuisine a. Launch Application b. Say “Bronx” as the neighborhood c. Application returns a list of restaurants d. Select “Narrow Them” e. Select “Cuisine” f. Say “Italian” Success: application creates a narrowed list of restaurants using neighborhood and cuisine
criteria.
Listen: “Bellavista Café, Dominicks, Enzos, Marios, Mona Lisa Gourmet Pizza, Robertos”
xii. Narrow a list of restaurants by Neighborhood a. Launch Application b. Say “Italian” as the cuisine c. Application returns a list of restaurants d. Say “Narrow Them” e. Select “Neighborhood” f. Say “Bronx” Success: application creates a narrowed list of restaurants using neighborhood and cuisine
criteria
Listen: “Bellavista Café, Dominicks, Enzos, Marios, Mona Lisa Gourmet Pizza, Robertos”
xiii. Select a Restaurant from a list of restaurants a. Launch Application b. Select “by Neighborhood” c. Say “Bronx” as the neighborhood for the search d. A list of restaurants is created e. Select “Marios” Success: application prompts: Marios is an Italian restaurant located in the Bronx neighborhood
of New York City.
xiv. Listen to the Address of a selected restaurant a. Launch Application b. Say “Aki Sushi” c. Select “Address” Success: caller hears: The restaurant is located at1425 York Avenue (between 75th & 76th
Streets) New York, NY
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 35
xv. Listen to the Phone No of a selected restaurant a. Launch Application b. Say “Aki Sushi” c. Select “Phone” Success: application prompts: (212) 628-8885
xvi. Listen to the Ratings of a selected restaurant a. Launch Application b. Say “Aki Sushi” c. Select “Rating” Success: caller hears: Food rating 21, Décor rating 15, Service rating 19, Price rating 32
xvii. Listen to the Description of a selected restaurant a. Launch Application b. Say “Aki Sushi” c. Select “Description” Success: caller hears: "Thank God, a sushi place came to York Avenue" sigh far East Side locals
happy to have this "tiny", mid priced Japanese around; but while most rave over the "fresh,
imaginative rolls", dissenters sniff "the tuna needs a tune-up", ditto the "zero decor."
xviii. Transfer call to the restaurant a. Launch Application b. Say “Aki Sushi” c. Select “Phone” d. Say “yes” to connect to the restaurant prompt Success: application connects to the restaurant
xix. Navigate to Main Menu from different states a. Launch Application b. Say “Zephyr Grill” c. Select “I am done” d. Select “Main Menu” Success: application goes back to main menu
THE SCALE FOR SCORING THE RESULTS OF THE TEST We’ll ask our test subjects to give a value on a scale of 1-5 for each task
o How difficult it was to complete o How difficult it was to learn
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 36
How easy was it to navigate through the states (scale of 1-5)
We’ll also ask following questions:
What are the top 3 tasks they liked?
What are the top 3 tasks that can be improved?
PRESENT OUR TEST FINDINGS FOR DEVELOPMENTAL CHANGES The lowest rated tasks will be considered for a re-design, especially if they are high-frequency tasks. Also
if majority of the test subjects have complain about a feature, we’ll take that into consideration as well.
If any of the suggested fix does not push back our development cycle, and delivery date we’ll also think
of adding additional features.
COMPARE WITH SIMILAR SYSTEMS (1-800-GOOG-411) During our initial survey someone mentioned about Google’s application that does similar
functions to our system. We called the system to figure out the strengths and weaknesses of that
and to compare with ours. First of all, Google’s one has a larger database and asks the caller for
his city and the state, and it also asks for the business type. Our system for now can only handle
restaurants, located in New York City. However, when I tried with Boston (and Seattle) city they
listed only 8 restaurants, which is very small compared to our 1500+ records. Google lists the
restaurant names together with the neighborhoods they belong to, and caller can only select a
name by an associated number or pressing that key. Once selected their system can provide caller
a short description or connect him to the restaurant. Our system on the other hand can take any
name (not limited to 8), or cuisine or neighborhood, provide a list and narrow it by other relevant
categories. Google’s system seemed to be more of a yellow page over the phone type of thing.
FINAL USABILITY TESTING AND FINDINGS Please see the attached emails for usability testing results and feedback.
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 37
APPENDIX
RECORDING PROMPTS
SURVEY QUESTIONAIRES
Initial Survey
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 38
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 39
RESTAURANT INFORMATION
CUISINE LIST
Afghan Cambodian Ethiopian Jamaican Pizza
American Caribbean Filipino Japanese Polish
Argentinean Chinese French Jewish Portuguese
Armenian Colombian Hamburgers Mexican and Tex Mex South American
Asian Continental Health Food Middle Eastern Southern and Soul
Australian Cuban Hot Dogs Moroccan Southwestern
Austrian Delis and Sandwich Shops Hungarian Noodle Shops Spanish
Bakeries Dim Sum Indian Nuevo Latino Steakhouses
Bar B Q Dominican Indonesian Persian Swiss
Belgian Eclectic and International Irish Peruvian Tapas
Brasserie Egyptian Israeli Puerto Rican Thai
Brazilian English Italian Russian Tibetan
Burmese Eritrean Korean Scandinavian Tunisian Cajun and Creole
Kosher Seafood Turkish
Californian French Bistro Lebanese Senegalese Ukrainian Coffee Shops and Diners German Malaysian Soups Vegetarian Coffeehouses and Desserts Greek
Mediterranean South African Venezuelan
Vietnamese
NEIGHBORHOOD LIST
Astoria East 30s Gramercy Park Sheepshead Bay
Bay Ridge East 40s Gravesend SoHo Little Italy
Bayside East 50s Greenpoint Staten Island
Bensonhurst East 60s Greenwich Village Sunnyside
Boerum Hill East 70s Jackson Heights Sunset Park Brighton Beach East 80s Little Neck
TriBeCa Downtown
Bronx East 90s Long Island City Union Square
Brooklyn East Village Lower East Side West 30s
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 40
Heights
Carroll Gardens Elmhurst
Meatpacking District West 40s
Central Village Flatiron District Middle Village West 50s
Chelsea Flushing Murray Hill West 60s
Chinatown Forest Hills NoHo West 70s
Cobble Hill Fort Greene Ocean Parkway West 80s
Coney Island Fresh Meadows Ozone Park West 90s
Corona Garment District Park Slope Williamsburg
Downtown Glendale Prospect Heights
Rego Park
SELECTED EMAILS RESPONSES
From: Eric Montagut, Senior Principal Engineer, Basis Technology
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 41
Zagat Restaurant Finder: A Speech Recognizer Application
Zina Saadi & Anila Chowdhury Page 42
From: Aniket Ahmed, Senior Consultant, Booz Allen Hamilton