Voice User Interface for Mobile Applications

Building a Voice User Interface Android Speech Recognition and Text-To-Speech

Tuesday, January 29, 13

http://wolfpaulus.comTuesday, January 29, 13

http://wolfpaulus.com


If a computer could think, how could we tell?


In 1950, Alan Turing suggested:

“If the responses from the computer were indistinguishable from that of a human,the computer could be said to be thinking.”


Loebner Prize Solid 18 Carat Gold Medal

Grand Prize of $100,000 and a Gold Medal for the first computer whose responses were indistinguishable from a human's.

Each year an annual prize of $2000 and a bronze medal is awarded to the most human-like computer.


How can we create a Chat bot ?


Capture Speech Input

Convert Speech into Text

Create Text Response

Message or Command ?

Execute Command

Message or Action ?

Perform Action

Synthesize Voice

(Message)

SpeekMessage

Cmd

Msg

Action

access Web Serviceperform on Device

Msg

ai




Synthesize Voice (Message)

SpeekMessage


Echo Bot



void startVoiceRecognitionActivity() {Intent intent = new Intent( RecognizerIntent.ACTION_RECOGNIZE_SPEECH );

intent.putExtra( RecognizerIntent.EXTRA_PROMPT, "Speak to Bot1");intent.putExtra( RecognizerIntent.EXTRA_MAX_RESULTS, 1);

intent.putExtra( RecognizerIntent.EXTRA_CALLING_PACKAGE, getClass().getPackage().getName() );

intent.putExtra( RecognizerIntent.EXTRA_LANGUAGE_MODEL, ! ! ! ! !! RecognizerIntent.LANGUAGE_MODEL_FREE_FORM );

startActivityForResult( intent, VOICE_RECOGNITION_REQUEST_CODE );

}


@Override

protected void onActivityResult( int requestCode, int resultCode, Intent data ) {

switch (requestCode) {

case VOICE_RECOGNITION_REQUEST_CODE:

if (resultCode == RESULT_OK) {

ArrayList<String> matches = data.getStringArrayListExtra( RecognizerIntent.EXTRA_RESULTS );

say( matches.get(0) );

} else {

mTV_STT.setText("");

}

break;

}

}

... speech has been converted into text ...




t


private TextToSpeech mTts;..mTts = new TextToSpeech( this, this ); ! // Context, TextToSpeech.OnInitListener ..

// Implement TextToSpeech.OnInitListener@Overridepublic void onInit( final int status ) {! if ( status == TextToSpeech.SUCCESS && mTts != null ) {

! ! startVoiceRecognitionActivity();

! ! mTts.setOnUtteranceCompletedListener(new TextToSpeech.OnUtteranceCompletedListener() {

! ! ! @Override! ! ! public void onUtteranceCompleted( final String s ) {! ! ! ! startVoiceRecognitionActivity();! ! ! }});

! } else {! ! mTV_TTS.setText("Could not initialize TextToSpeech.");! }}



private void say(final String s) {

final HashMap<String, String> map = new HashMap<String, String>(1);

map.put(TextToSpeech.Engine.KEY_PARAM_UTTERANCE_ID, UTTERANCE_ID);

mTts.speak(s, TextToSpeech.QUEUE_FLUSH, map);

}

SpeekMessage


Code and DemoCapture Speech Input



SpeekMessage





Execute Command


SpeekMessage


“stock quote for ...”

Stock Quote Bot


@Override protected void onActivityResult( int requestCode, int resultCode, Intent data ) {


case VOICE_RECOGNITION_REQUEST_CODE: if (resultCode == RESULT_OK) {

ArrayList<String> matches = data.getStringArrayListExtra(RecognizerIntent.EXTRA_RESULTS); say( matches.get(0) );

} else {

mTV_STT.setText(""); }

break;

} }



@Override protected void onActivityResult( int requestCode, int resultCode, Intent data ) {


case VOICE_RECOGNITION_REQUEST_CODE: if (resultCode == RESULT_OK) {

ArrayList<String> matches = data.getStringArrayListExtra(RecognizerIntent.EXTRA_RESULTS);final String s = matches.get(0);

final int k = s.indexOf( KEY_WORD );

! if (0 <= k) {! final String ticker = s.substring( k + KEY_WORD.length() ).trim();

! ! ! new YQuote(mHandler).execute( ticker );! ! } else {

say( s );

} } else {

mTV_STT.setText(""); }

break;

}

}





Execute Command


SpeekMessage




What is cheese ?What is chocolate?

How old are you? Who is the President?Where is Atlantis? What’s up?

Did you have dinner already?


Artifical Intelligence Markup Language (AIML)

<?xml version="1.0" encoding="ISO-8859-1"?> <aiml>

<category><pattern>WHAT IS AIML</pattern><template>

AIML is short for Artifical Intelligence Markup Language

</template></category>

</aiml>



<category><pattern>WHAT IS AIML</pattern><template>

<random><li>First response</li><li>Second response</li><li>3rd response</li>

</random></template>

</category></aiml>


AIML Spec.• http://www.alicebot.org/TR/2011/

AIML Primer• http://www.alicebot.org/documentation/aiml-primer.html


http://www.alicebot.org/TR/2011/




Program D v4.6 http://aitools.org/

last updated:14-Mar-2006


http://aitools.org

http://aitools.org

CharlieBot 4.1.8 http://sourceforge.net/projects/charliebot/

Forked from Program D v4.1.5 works on Mac OS X or any Java 1.3 or better VM

last updated:14-Dec-2002


http://sourceforge.net/projects/charliebot/

http://sourceforge.net/projects/charliebot/

ChatterBean http://www.geocities.ws/phelio/chatterbean/

ChatterBean is an AIML interpreter (also known as "Alicebot") written in pure Java.

Fully AIML 1.0.1 compliant

last updated:11-May-2006


http://aitools.org

http://aitools.org

http://aitools.org

http://aitools.org

http://www.geocities.ws/phelio/chatterbean/

http://www.geocities.ws/phelio/chatterbean/

AIML Sets

• http://aitools.org/Free_AIML_sets

• http://code.google.com/p/aiml-en-us-foundation-alice/

• http://www.square-bear.co.uk/aiml/


http://aitools.org/Free_AIML_sets

http://aitools.org/Free_AIML_sets

http://code.google.com/p/aiml-en-us-foundation-alice/

http://code.google.com/p/aiml-en-us-foundation-alice/

http://www.square-bear.co.uk/aiml/

http://www.square-bear.co.uk/aiml/

http://myAIMLServer:PORT/talk?botid=xyz..

XML-RPC: http://myAIMLServer:PORT/talk-xml

HTTP-POSTbotid=”xzy..”input=”Hello”custid=”d22..”

RESPONSE:<result status="0" botid="xyz.."

custid="d2228e2eee12d255"> <input>Hello</input> <that>Hi there!</that></result>

HTML:


http://www.pandorabots.com/pandora/talk?botid=f6d4afd83e34564d

http://www.pandorabots.com/pandora/talk?botid=f6d4afd83e34564d

http://www.pandorabots.com/pandora/talk-xml

http://www.pandorabots.com/pandora/talk-xml





Execute Command


SpeekMessage

Cmd

Msg


Msg


@Overrideprotected void onActivityResult(final int requestCode, final int resultCode, final Intent data) {! switch (requestCode) {! ! case VOICE_RECOGNITION_REQUEST_CODE:! ! if (resultCode == RESULT_OK) {! ! ! final ArrayList<String> matches = data.getStringArrayListExtra( RecognizerIntent.EXTRA_RESULTS);! ! ! final String s = matches.get(0);! ! ! new AIML_RPC(mHandler).execute(s);! ! ! mTV_STT.setText(s);! ! } else {! ! ! mTV_STT.setText("");! ! }! ! break;! }}

Speech has been converted into text ...


@Overridepublic void handleMessage(final Message msg) {! if (msg.getData() != null && !msg.getData().isEmpty()) {! ! String s = msg.getData().getString(Bot3.BUNDLE_KEY_NAME_FOR_MSG);! ! if (s != null && 0 < s.length()) {! ! ! int i = s.indexOf("CMD ");! ! ! if (0 <= i) {! ! ! ! s = s.substring(i + 4, s.endsWith(".") ? s.length() - 1 : s.length());! ! ! ! String cmd;! ! ! ! int k = s.indexOf(" ");! ! ! ! if (0 < k) {! ! ! ! ! cmd = s.substring(0, k);! ! ! ! ! s = s.substring(k + 1).replace(" ", "");! ! ! ! ! if (Bot3.KEY_WORD.equals(cmd)) {! ! ! ! ! ! new YQuote(mHandler).execute(s);! ! ! ! ! }! ! ! ! }! ! ! } else {! ! ! ! Bot3.this.say(s);! ! ! }! ! }! }}

Handler






Execute Command


SpeekMessage

Cmd

Msg


Msg


Summary





SpeekMessage





Execute Command


SpeekMessage







Execute Command


SpeekMessage

Cmd

Msg


Msg

AIML Bot


Cora, your imaginary friendTechcasita Productions


https://play.google.com/store/apps/developer?id=Techcasita+Productions

https://play.google.com/store/apps/developer?id=Techcasita+Productions

Voice User Interface for Mobile Applications

Technology

Transcript of Voice User Interface for Mobile Applications