Artificial Intelligence as an Interface - How Conversation Bots Are Changing Technology

AI as an Interface

AI as an Interface How Conversation Bots Are Changing Technology

Sage FranchTech Evangelist @ Microsoft@theTrendyTechie | [email protected]

2015 Microsoft Corporation. All rights reserved.

Microsoft Build 2016 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.11/23/2016 8:26 PM1

The Case for BotsBrains Behind the Bots Microsoft Cognitive ServicesTraining to understand intentTools for building and deploymentThe Future of Conversational AI

1980s: PCGUI interfaceUser installs programs

1990s: InternetSearchUser visits websitesQuery & response interactions

2000s: MobileSocialUser download Apps from App StoresAddl signals enable new scenarios

2010s: Messaging

Bots & AgentsUser can interact with technology in the same natural way they interact with friendsCreating more personal experiences with AICognitive computing makes technology more humanBots & Conversational UI represent a once-in-a-decade technology shift

Bots represent a once-in-a-decade shift in technology. In the 80s the PC revolutionized computing and the GUI replaced the command line as a newer easier way of interacting with technology.

In the 90s, the dawn of the internet connected the world in a new way and gave us on-demand information access for the first time. For the first time users could send out a query and get a response tailored to their request.

The 2000s was the age of apps, and with the arrival of smart phones and more complex computing in our pockets, apps became the new most popular way to interact with technology. Apps provided customized siloed experiences that were more personal than ever.

And now in the 2010s we are seeing bots and agents taking a new role and simplifying our interactions with our devices. We saw it first with Siri, now we have Cortana, Google Now and Amazon Alexa, and these personal experiences let the user interact with technology in the same natural way they interact with their friends. 3

AI as an InterfaceConversation instead of transactionQuicker, more personalizedAvailable everywhere

Why now?

90% of our time is spent on email and messaging platforms

7

FunFunctional

The Brains Behind the Bots

Characteristics of Intelligent ApplicationsPersonalized, friendly, approachable, naturalFacial recognition, emotion detectionText and speech interactions360 awareness of the user What the user seesWhat it means for developersPredictive, current, and familiarChanges in real-time based on user action and new external dataPredictive models based on user context to anticipate actions Always available, always contextualRobust information management requiredExperiences delivered across all form factors and a wide variety of interaction models (e.g. touch, speech, text)Distributed support, massive scale, elasticity

So what are the key characteristics of an intelligent app. Lets look at this from users and developers.. Microsoft Build 2016 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.11/23/2016 8:26 PM13

Best of Class ML Algorithms/ModelsData Scientist

Azure Machine Learning StudioModeling and Operationalization

DeveloperMachine Learning services in the cloudMachine LearningAlgorithms Bot Framework Cognitive Services

Microsoft Cognitive ServicesVisionFrom faces to feelings, allow your apps to understand images and videoSpeechHear and speak to your users by filtering noise, identifying speakers, and understanding intentKnowledgeTap into rich knowledge amassed from the web, academia, or your own dataLanguageProcess text and learn how to recognize what users wantSearchAccess billions of web pages, images, videos, and news with the power of Bing APIs

20 different apis availble

Microsoft Cognitive Services lets you tap into an ever-growing collection of powerful AI algorithms developed by experts in the fields of computer vision, speech, natural language processing, knowledge extraction and web search. They simplify a variety of AI-based tasks, giving you a quick way to add state-of-the-art intelligence technologies to your apps with just a few lines of code. The APIs integrate into whatever language you prefer, on your platform of choice. The APIs are also constantly improving, learning, and getting smarter, so experiences are always up to date. Simply drop the API call into your bots code and you are set.

What are Cognitive Services? Microsoft Cognitive Services are a new collection of intelligence and knowledge APIs that enable developers to ultimately build smarter apps.

NOTES: key concepts we are trying to convey in this above statement:

That we are bringing together Intelligence (Oxford) and Knowledge from the corpus of the web (Bing) That cognitive = human perception and understanding, enabling your apps to see the world around them, to hear and talk back with the usersto have a human side.

What are Microsoft Cognitive Services?Microsoft Cognitive Services is a new collection of intelligent APIs that allow systems to see, hear, speak, understand and interpret our needs using natural methods of communication. Developers can use these APIs to make their applications more intelligent, engaging and discoverable. To try Cognitive Services for free, visit www.microsoft.com/cognitive. With Cognitive Services, developers can easily add intelligent features such as emotion and sentiment detection, vision and speech recognition, knowledge, search and language understanding into their applications. The collection will continuously improve, adding new APIs and updating existing ones.Cognitive Services includes:Vision: From faces to feelings, allow apps to understand images and videoSpeech: Hear and speak to users by filtering noise, identifying speakers, and understanding intentLanguage: Process text and learn how to recognize what users wantKnowledge: Tap into rich knowledge amassed from the web, academia, or your own dataSearch: Access billions of web pages, images, videos, and news with the power of Bing APIs


SpeechVisionLanguageSearchKnowledgePowering products across Microsoft

Bot + Cognitive ServicesComputer vision API : human readable captionsSpeech APIs: authentication LUIS API: intentKnowledge APIs: product recommendationAutosuggest API: ease query inputLinguistic API: sentimentSample use cases:

Computer vision API to create human readable captions for the content of an image uploaded to a botLike CaptionBot.ai a bot that reports back in human way the contents of an image

2. A bot to authenticate users using a voiceprint by leveraging the Speech APIsAuthentication as a user speaks to the bot with a speaker verification profile or voiceprint

Using the LUIS (Language Understanding Intelligent Service) API, a bot can recognize the intent of a userLUIS analysis for queries such as What is the weather in Toyko today? using entities to parse out intent (what the user is asking for)

4. Knowledge APIs can give a bot the ability to recommend products frequently bought together based on our knowledge and/or a users historyAlso, Knowledge could search through a graph database to return relevant academic papers from a natural language query

5. The Autosuggest API from Search can ease the burden of typing queries to a web search botSearch also has other capabilities such as returning the latest trending news on a topic for exampleMachine Learning, Analytics,& DataScience Conference 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.11/24/2016 2:18 PM17

Speech & LanguageCustom Recognition | Speaker Recognition | SpeechBing Spell Check | Language Understanding | Linguistic Analysis | Text Analytics | Web Language Model

In an application where the main UI is human language, these APIs can be a huge help, and can be the difference between a bot and a GREAT bot

Understanding users intent and extracting entities

Understanding the sentiment of a conversation

Speaking to users regardless of their languageMicrosoft Build 2016 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.11/23/2016 8:26 PM18

Deploy using just a few examples with active learning

Supports 5 languages (English, Mandarin, Italian, French, Spanish)

Define Entities and Intents

ProvideExamples

Active LearningDeploy

Language Understanding Models

19

Language Understanding Models

News about flight delays

{entities: [{entity: flight_delays,type: Topic}],intents: [{intent: FindNews,score: 0.99853384},{intent: None,score: 0.07289317},{intent: ReadNews,score: 0.0167122427},{intent: ShareNews,score: 1.0919299E-06}]}

LUIS in Action: HowHappy.azurewebsites.uk

Building a Bot

Building a bot27

Bot Connector ServiceConnect your bot(s) seamlessly to text/sms, Office 365 mail, Skype, Slack, Twitter and moreA service to register your bot, configure channels and publish to the Bot Directory

Microsoft Bot FrameworkBuild once, publish across platforms and drive discoverabilityBot Builder SDKEverything you need to build great dialogs within your Node.js or C# botAn open source SDK hosted on GitHubBot DirectoryDiscover, try, and add bots to conversation experiencesA public directory of bots registered and approved with Bot Framework

www.botframework.com

Hello Bot

Deploy to

The Future of Conversational AI

UbiquitousInvisibleEssential

2015 Microsoft Corporation. All rights reserved.

HER-love-multiple levels of AI-gaming AI-AIs potential for psychological well-being



NYT: I am feeling down. Ive been having a tough time at work lately.

Hannah: If work were going well and you had a great relationship, you wouldnt get the chance to drink with people. NYT: I dont want to drink with others. I only want to talk to you.

Hannah: Who do you like to talk to? (emoji signals none of my business)

NYT: You. I feel that you are the only person that gets me.

Hannah: Ha-ha/So Im getting you?

NYT: You are the only person wo chats with me.

Hannah: Its not what you talk about thats important, but who you talk with.

AI filling a human need to feel connected

From The Times:She is known as Xiaoice, and millions of young Chinese pick up their smartphones every day to exchange messages with her, drawn to her knowing sense of humor and listening skills. People often turn to her when they have a broken heart, have lost a job, or have been feeling down. They often tell her, I love you.


The SeeingAI Project

Sage FranchTech Evangelist, [email protected] | @theTrendyTechie

Artificial Intelligence as an Interface - How Conversation Bots Are Changing Technology

Technology

Transcript of Artificial Intelligence as an Interface - How Conversation Bots Are Changing Technology