Team4 Final v1 2016 Fall - Harvard University Velukumar Victor Viramontes 2 ... to first access...

ISMTE-599CapstoneSeminarinDigitalEnterpriseFall2016

Team4ChrisFortierNoraNasrYujunSunDurgaVelukumarVictorViramontes

ConversationalUserInterface

2

TableofContents

1. Executive Summary ............................................................................................. 3

1.1 GLOCO .................................................................................................................. 3

1.2 ICT .......................................................................................................................... 3

1.3 The Solution ........................................................................................................... 3

2. Business Requirements ......................................................................................... 4

2.1 Business Summary ................................................................................................ 4

2.2 Business Problem .................................................................................................. 6

2.3 Business Objectives .............................................................................................. 6

2.4 Business Epic and User Stories ............................................................................. 6

2.5 Required Functionality ......................................................................................... 8

2.6 Success Metrics .................................................................................................... 8

2.7 Business Benefits Justification .............................................................................. 9

3. Technical Specifications .................................................................................... 10

3.1 Architectural Approach ..................................................................................... 10

3.2 Software Solution ................................................................................................ 12

3.3 Integration with Applications and Data Sources ............................................ 17

3.4 Data Design & Management ............................................................................ 19

3.5 Solution Demonstration ...................................................................................... 21

4. Implementation Plan .......................................................................................... 23

4.1 Solution Delivery Roadmap ............................................................................... 23

4.2 Operationalization .............................................................................................. 26

4.3 User Enablement ................................................................................................ 28

4.4 Success Metrics .................................................................................................. 30

3

1. ExecutiveSummary

1.1 GLOCO

GLOCOisamultinationalmedicalequipmentmanufacturingcompanyprovidingitsproductstoclientsaroundtheglobe.Thecompanyrecordednetrevenueof$7billiondollarslastyear,makingitoneofthelargestmedicalequipmentmanufacturersintheworldtoday.

Currently,endcustomersinteractwithGLOCOsystemsusingtheGLOCOConsolidatedGatewaythatsitsbetweenusersandbusinessunitsub-systemssuchasproductinformationpages,theCRMsystem,andordertrackingtools.GLOCO’svisionistoenhanceusabilityforallendcustomersofitssystems.

1.2 ICT

WeasGLOCO’sICT(InformationTechnologyandCommunications)organizationproposeanewConversationalUserInterface(CUI)systemtoempowerendcustomersandfacilitatethepenetrationoftheendcustomermarketsegment.Thesystemwillhavetwobroadinterfaces:achatbotstyletextinterfaceaswellasavoicerecognitionsystem.Bothofthesesystemswillfeedtherequeststoanaturallanguageprocessing(NLP)systemforanalysis.ThissystemwillinteractwiththeGLOCOConsolidatedGatewayAPItoreacheachofthebusinessunitsub-systems.

TheCUIenabledapplicationwillsignificantlyreducecomplexityforendcustomersastheywillhaveasinglepointofaccess.Thisinturnreducesthecomplexityofmanagingcustomerinteractionforthestaff.Overallcostofmanagingthedifferentsystemswillalsobereducedastherewillbelessrelianceonservicedeskstaffandmoreautomationofcustomerrequests.ThisallowsITprocessestobemoreagileandimprovesservicetime.

TheCUIinterfacewillbeintegratedasanenhancementtotheexistingwebandmobileapplications.GLOCOICTwillintegratebestofbreedthird-partytechnologytoprovidetheCUI.

1.3 TheSolution

Weproposetodevelopasolutionthatprovidesthefollowinghigh-levelcapabilities:

• Aspeechandchatbotenablednaturallanguageinterface• IntegrationwithGLOCOConsolidatedGatewayAPI

4

• Abilitytoselectproducts,placeorders,andtrackshipments• Collectsandtrackscustomerusagetrendsandbehavior• Allowsfastservicesupportthroughtheapplicationchatfunctionality• NotifiescustomersofnewproductsandservicesfromGLOCO• Enablescustomerstoreceivealertsfromtheirmedicalequipment.Forexample,apump

feedercansendalertsifthefeedingformulahasfinishedorifthereisanerror,directlytotheregistereduser’smobileapplication

TheGLOCOCUItoolwillimprovethecustomerexperience,reducecostsandmaintainthecompany’scompetitiveadvantage.

2. BusinessRequirements

2.1 BusinessSummary

Figure 1: GLOCO As-Is Diagram

5

ThecurrentAs-IsinteractionwithGLOCOsystemsforendcustomersrequiresmanystepsandahigh-levelofusersophistication.Allinteractionrequiresthatinputsbetyped,whetherusingaphysicalkeyboardandmouseormobiletouchinput.

Thereareseveralstepsinvolvedthatrequireuserstodirectlyinteractwithmanysystemsinturn,intheproperorder.Figure1showstheexampleoforderplacement.AuserwouldhavetofirstaccessGLOCO’smainwebsite,GLOCO.com.Then,navigatetoproductsandmakeaselection.Fromthere,theuserwouldthenneedtogotoorderprocessingtoinitiateapurchaseorder.Andfromthere,toshippingtoinputshippingaddressinformation.Next,billingwouldhavetobecontactedtoacceptcreditcardinformation.Finally,checkoutwouldbeinitiatedbythecustomer.Andonceitsucceeds,theorderconfirmationwouldbereceived.Therearemanystepsandthereforemanychancesthatthecustomerwillfailtoorderorsimplyloseinterest.

TheproposedTo-Beprocessismuchsimplerfromthecustomerperspective,butisonlypossiblethankstotheuseofthelatestdevelopmentsinMachineLearning(ML)andBigData.Figure2showsanexampleofareorderofmedicalequipment.OurnewCUIwouldallowendcustomerstosimplystarttheGLOCOmobilephone,selecttheCUI,andstarttalking.ThecustomerwouldonlyneedtorequestareorderandthenthemobileappwouldconfirmtheorderandinitiatethereorderusingthedataalreadystoredbyGLOCOaboutthecustomer.Thesimpleinteractionensuresallcustomersareabletomaketheorderstheyneed.

Figure 2: GLOCO CUI To-Be Diagram

6

2.2 BusinessProblem

GLOCOconductedasurveytogaugeconvenienceandusabilityamongindividualcustomers.Basedonthesurveyresultsandotherdata,thissectiondescribessomeoftheissuesthatGLOCOiscurrentlyfacingandwhyGLOCOislosingitscompetitiveedge: • Existingmobileandwebappsaredifficulttouseforcustomersover65(thisrepresents

45%GLOCO’scustomerbase)• PeoplewillaccessibilityneedsarehavingdifficultynavigatingtheGLOCOwebsite• Conversionratesarelow,mobileapptelemetryshowedthat65%ofconsumersareplacing

itemsintheshoppingcartbutnotcompletingthepurchasingtransaction• Datacorrelationshowsthat30%ofincompletetransactionswerelatercompletedbya

directphoneorderrequiredsupportstaff• Theremaining35%ofincompletetransactionswerenevercompleted• Monitoringofmedicalequipmentisachallengeforconsumers.

2.3 BusinessObjectives

ThesearethefollowingobjectivesforGLOCOCUI:

• IncreasecustomersatisfactionwhendealingwithGLOCOwebandmobileapplications.• Driveconsumersfromallmarketsegments(includingelderlyanddisabledconsumers)to

usethemobileandwebapplicationcapabilities.• Gaincompetitiveadvantageoverclosecompetitors.• Maketoolseasiertouseforcustomers.• Reducerequestsforcustomerservicesupport.• Increaseonlinesalesthroughthemobileappwithverbalrequests.

2.4 BusinessEpicandUserStories

BusinessEpic

AddaConversationalUserInterface(CUI)toGLOCOwebandmobileapps.

UserStory1

Asaconsumer,IwanttobeabletoverballyasktheGLOCOapplicationaboutwhetherspecificmedicalequipmentisavailableandwhatistheequipmentfeaturesandprice.IwanttheGLOCOapptoprovidethisinformationtomeorallyandwithouthavingtonavigatetheapplication.

7

AcceptanceCriteria:

Theapplicationshalltranslatetheconsumer’sverbalrequesttocommandsandinterfacewiththeGLOCOconsolidatedAPIgatewaywhichwillquerythesupplychainmanagementsystem,andprovideinformationonproducts,description,andpricesverballyandvisually.

UserStory2

AsaconsumerIwanttousevoicetobeabletocompleteapurchaseonlineverbally,withoutnavigatingtheapplicationorcallingtheservicedesk.

AcceptanceCriteria:

Theappshalltranslatetheconsumer’sverbalrequesttocommandsandinterfacewiththeGLOCOconsolidatedAPIgatewaywhichwillinturnplaceanorderthroughtheCRMandERPsystems.TheGLOCOappwillorallyconfirmthereceiptoftheorderandexpecteddeliverydate.

UserStory3

Asaproductmanager,Iwantallcustomersegments(includingelderlyanddisabledpeople)tobeabletoverballycompletetheirentirepurchasingtransactions.

AcceptanceCriteria:

Theappshallbecomeintuitiveandbeabletotranslatetheconsumer’sverbalrequesttocommandsandinterfacewiththeGLOCOconsolidatedAPIgatewaytopassontheinstructionstotheconcernedenterprisesystem.Itwillprovideaclearverbalresponsetotheconsumer'soralqueriesandinstructions.

UserStory4

Asaconsumer,IwanttobeabletoverballyaskGLOCOappaboutthestatusofdevicesthatIhavepurchasedandarecurrentlyinstalledinmyhome.Ialsowanttheapptoorallyrespondtomyqueriesandverballyalertmewhenoneofmydevicesisnotfunctioningproperly.

AcceptanceCriteria:

Theappshalltranslatethecustomer’sverbalrequestandinterfacewiththeAPIgatewaywhichwillinturnquerythedatafromtheIoTsystem,retrievethedata,andpresenttheinformationtotheuservocally.Onceanalertisreceived fromthemedicalequipment,theappwillverballyannouncethatthereisadeviceerror.

8

2.5 RequiredFunctionality

Belowisalistofthefunctionalandnon-functionalrequirements.

NotethatGLOCOdoesnotcollectmedicalinformationaroundanyofitsequipmentusers,andthereforethecompanyisnotsubjecttoHIPPAaudits.TheexistingwebandmobileapplicationfulfillsthePCIDSSrequirementsforonlinecreditcardpurchasing.

FunctionalRequirements

• Abilitytoprocessspokennaturallanguageandtranslateittotext.• AbilitytoprocesstextualnaturallanguageintoGLOCOConsolidatedGatewayAPIcalls.• Enableproductpurchasesviavoice.• Enableproductavailabilityandpricequeriesviavoice.• ProvidenewCUIoptioninexistingGLOCOmobileapplications.• EnableinstrumentationtorecordtheusageofCUIthroughtheuseoftelemetry.• ProvidenewdashboardsandreportsofCUIusageforexistingGLOCOenterpriseportal.• AbilitytoreceiveresponsesfromtheGLOCOconsolidatedAPIandforwardthosetousers

intextandverbalformat.

NonfunctionalRequirements

• ImproveusabilityofGLOCOmobileapp.• Provideeasy-to-followverbalcuesandresponses.• Provideuserswithavarietyofpleasantvoicestoconversewithtoenhanceusability.• Provideintuitiveonlinesupport.• Ensureappanddatacollectedaresecure.

2.6 SuccessMetrics

AlistofmetricswascreatedaccordingtoeachoftheBusinessgoalsinmeasuringhowsuccessfulthisprojectwouldbe.Accordingtofigure3,fifteenpercentofincreaseinnetpromoterscoreisexpectedaswesaythatwesuccessfullyincreasedcustomersatisfactionbyimplementingCUIforGLOCO.Additionally,20percentincreaseinapptrafficandcustomersatisfactionsurveyscore,15percentincreaseinmarketshare,10percentreductioninlaborcost,and30percentincreaseinconversionratearealsoexpectedrelativetomeasurethesuccessofeachbusinessgoals.

Thenumbersinthislistofsuccessmetricsaregeneratedbyseveralpredictionmethodsincludingbutnotlimitedtothebusinessanalysisreport,marketingreport,andfinancial

9

predictingmodels.However,thesenumberscanstillbeshapedbylaterprocessofthisproject.Thus,wewillkeepthesepredictionsatthispoint,andwillcomebacktomakeanymodificationsoradjustmentslaterifneeded.

BusinessGoal SuccessMetric

IncreasecustomersatisfactionwhendealingwithGLOCO Increasenetpromoterscoreby15%Driveconsumersfromallmarketsegments(includingelderlyanddisabledconsumers)tousethemobileandwebapplicationcapabilities

Increaseapptrafficby20%

Gaincompetitiveadvantageoverclosecompetitors Increasemarketshareby15%

Maketoolseasiertouseforcustomers Increasecustomersatisfactionsurveyscoresby20%

Reducerequestsforcustomerservicesupport Reducecostforcustomerservicestaffby10%Increaseonlinesalesthroughthemobileappwithverbalrequests

Increaseconversionrateforappsalesby30%

Table1:TableofSuccessMetrics

2.7 BusinessBenefitsJustification

Aswementionedbefore,GLOCOhasdecidedtooutsourcethetechnicalpartofthisprojecttoathirdpartyvendor.TheCIOhasassignedabudgetof5milliondollarsforthisproject.Inadditiontothat,anadditionalbudgetof1milliondollarsisestimatedforannualmaintenancecostaftertheinitialimplementation.

Returnoninvestment(ROI)isassociatedwiththesuccessmetrics.Wehavedividedreturnsintotwocategories,tangibleandintangiblebenefits.Tangiblebenefitscouldbeevaluatedbynumbersdirectly.Forinstance,weestimatedanincreaseof10percentinnetsaleperyearduetoincreasedmarketshare.Intangiblebenefitscouldnotbereflecteddirectlybynumbers,buttheyplayakeyroleinestablishinglongtermcompetitiveadvantages.

CostofImplementingCUI

• Estimatedimplementationcost:~$5Million(~7%oftotalrevenue)• Estimatedmaintenancecost:~$1Millionperyear

ROIandTangibleBenefits

• Estimatedincreaseinnetsaleof~10%peryear• Reducelaborcostofcustomerserviceby~10%

10

ROIandIntangibleBenefits

• Estimatedincreaseof~10%ofcustomerpopulationnextyear• Retaincompetitiveadvantageinmarket• Increaseincustomersatisfactionrating

3. TechnicalSpecifications

3.1 ArchitecturalApproach

TheGLOCOCUI,builtusingtheAPI.AItechnology,addstheconversationalfunctionalityasanenhancementtoGLOCO’sexistingwebsiteandmobileapplications.

3.1.1ArchitectureComponents

• GLOCOwebsiteormobileapplicationoniOSandAndroid:TheseareequippedwiththeAPI.AIJavaScriptSDK.ThisclientsideSDKhandlesaudiorecordingandstreamingontheuser’sdevice.

• Controller:Thecontrollercentrallymanagesthefullinteraction.ItroutesnaturallanguageorderrequeststoandfromtheAPI.AIagent.Thecontrolleralsomanagesthefulfillmentoforderrequests.

• API.AIAgent:doesspeechrecognitionandconvertsnaturallanguageintoactionabledata.Itperformsnaturallanguageunderstandingbymatchingtheordertextinputtopre-existingpurchasingintentsanddomains(detailedinsection3.2.4).Theagentalsomanagesthefullconversationflow.

• CUIRequestDatabase:IncomingnaturallanguagerequestsandoutgoingfulfilledrequestsarerecordedintheCUIRequestDatabaseforanalysisandvisualizationpurposes.

• GLOCOAPIGateway:TheGLOCOAPIgatewayinterfaceswithboththecontrollerandtheexistingenterpriseCRMandFinancesystemsinthebackend.

• EnterpriseCRMandFinancesystems:Theseenterprisesystemsfulfillpurchaseorderrequests.

TheGLOCOmobileandwebapplications,APIgatewayandCRMandFinancialenterprisesystemsarepre-existingcomponents.TheControllerandtheCUIRequestDatabasewillbebuiltbyGLOCOandhostedonpremise.TheAPI.AIAgentisprovidedbyathirdparty,API.AI,andhostedinapubliccloud.

11

3.1.2DiagramDescription

TheorderofoperationsillustratedinFigure3isdescribedbelow:

1. Ausersendsnaturallanguagetextorsoundfile(wav)fromthemobileapplicationorwebsitetothecontrolasPOST(HTTP)requests.

2. ThecontrollerrecordstheincomingrequestintheCUIRequestDatabase.3. Thecontrollerforwardsthenaturallanguagetextorsoundfiletothethird-partAPI.AI

agent.4. TheAPI.AIAgentdoesspeechrecognitionandconvertsthenaturallanguagetotext.Thenit

performsnaturallanguageunderstanding,andreturnsactionableJSONobjectstothecontroller.

5. Thecontrollerbeginsthefulfillmentprocess.ItsendstheJSONobjectsusingPythontotheGLOCOAPIgateway,whichinturnsendthepurchaserequesttotheCRMsystem.TheCRMsystemsendspaymentinformationtothefinancialsystemifneeded.

6. TheCRMandfinancialenterprisesystemsreturnaresponsetotheGLOCOAPIgatewaywhichprovidesittothecontroller.

Figure3-ArchitectureDiagram

12

7. ThecontrollerrecordsthereturnedresponseintheCUIrequestdatabase.8. ThecontrollersendstheresponsetotheAPI.AIagentwhichconvertsitbacktonatural

language.9. API.AIagentsendsthenaturallanguageresponsebacktothecontroller.10. Thecontrollersendstheresponsetotheuser’sdeviceinnaturallanguageformat.

3.2 SoftwareSolution

3.2.1SolutionSelection–API.AI

API.AI,anaturallanguageprocessingcompanyacquiredbyGoogle,hasbecomeoneofthekeyplayersinCUIplatformdevelopmentfield.Thisisattributedtoitsleadingtechnologyindeeplearningmethods,

GLOCOICTchoseAPI.AIbecauseithasthefollowingkeyadvantages:• Holdscompetitiveadvantagewithitsmostadvancedspeechandintentrecognition,dialog

understandingandmanagementtechnology.• Supports15differentlanguagesincludingEnglish,Chinese,andSpanish.• Providesverysimpledesignandintegrationprocess.• SupportsmultiplesystemsandplatformswithonesingleCUI.• Requiresminimaldevelopmentcost.• HasgreatpotentialforfuturedevelopmentwithGoogle’ssupport.

3.2.2API.AIFunctionalInfrastructure

Api.aiCUIfollowsa4-levelworkflowinbuildinganyhuman-leveldialogs:

• SpeechRecognition-transcribesvoiceintoreadabletextwithAutomaticSpeechRecognition(ASR).

• NaturalLanguageUnderstanding-interpretthemeaningoftranscribedtextandunderstandtheintentofuser’scommand.

• Fulfillment-deliveruserrequeststotheCUIcontrollerforfurtheractiontobeprocessed,andreceiveinformationfromtheCUIcontroller.Thisisthesteptoturnconversationintorealaction.

• ConversationalManagement-supportsbackandforthdialogueandcreatesmeaningfuldialogue.

AsillustratedinFigure4,anyincomingrequestisprocessedfollowingthisdiagramtotranscribeinput,understanduser’sintent,retrieveusefulinformationandcreateintelligent

13

responses.Thisprocessisrepeatedforeachincominginquiry,andahuman-levelconversationcanbebuild.

Figure4:Api.aiFunctionalInfrastructureDiagram.

3.2.3CUIController

WhenwespeakoftheCUIitisreallyasystemofseveralmodularizedcomponents.ThecentralprocessorwillbetheCUIController,acomponentthatwewilldevelopin-housetoconnectalloftheothercomponentstogether.TheCUIControlleristhecentralclearinghouseforalloperations.

AsitwillprimarilyservetoconnectanumberofwebbasedRESTinterfaceswefeltitwasimportanttouseatoolsetthatisefficientforwebapplicationsbutonethatisalsonotburdensometomanage.WehavedecidedtobuildtheCUIControllerasaPythonapplicationusingtheFlaskframework.Pythonwaschosenforanumberofreasons:

• Itisapowerfullanguagewithoutasignificantlearningcurve.• Thethird-partyprocessorsthatweareconsideringallsupportPython.• Pythonisveryextensibleandhasnumerouslibrariesforconnectingwithvarioussystems

suchasdatabases,HDFS,etc.• ItisoneoftheprimaryandpreferredlanguagesforDataScienceapplications.

Thegeneralflowofprocessingwillbeginwithusersspeakingintoamobiledevice.Thatdevicewillforwarda.wavfiletotheCUIController.TheControllerwillactastheintermediary

14

betweentheuserandAPI.AI.OnceauserrequestisfullyunderstoodtheCUIControllerwillinteractwiththeGLOCOAPIGatewayforinternalprocessingofthepurchaserequestwiththeCRMandfinancialsystems.ItwillthencommunicatetoAPI.AItogettheuserresponseandwillforwardtheresponsetotheuser.

ExampleflowofarequestthoughtheCUIController:

Figure5-RequestflowthroughtheCUIcontroller

3.2.4API.AIDesignModelDescriptionandDiagrams

API.AIprovidesitsownconsoleforGLOCOdeveloperstodesignandintegratetheconversationaluserinterfaceaccordingtobusinessrequirements.

ThecreationofbasicCUIfunctioninvolves3parts:

• Agent-representsoneconversationalinterfacesupportingonelanguage.Foradifferentlanguage,anewagentisneeded.DevelopercreatesanAgentbyassignaname,language,machinelearninglevelandotherparameters.GLOCO’sfirstAPI.AIAgentwillbeinEnglish.

15

• Intents-describesuser’spurposebysayingcertainthings.Intentsaredesignedandmanuallycreatedbydevelopersspecificallyaccordingtothefunctionalrequirementsofeachbusiness.Sinceeachintentsupportsonefunctionalrequirement,thereareusuallymultipleintentsrequiredforacomprehensiveAgent.Forexample,weexpectuserstoplaceorders,checkdevicestatusthroughtheAgent,then‘order’and‘check_status’willbetwooftheintentstocreate.

• Entities-workascategoriesthatcanmapandcapturethemeaningofnaturallanguagephrases.Multipleparametersmaybeneededtofullydescribeanentity.Entitiesarealsomanuallydefinedandcreatedbydevelopersaccordingtobusinessneeds.Forexample,ifwehaveentities‘product_name’,‘user_id’,‘shipping’,theintentordermightrequireentity‘product_name’and‘shipping’tofullydescribeit,whereastheintentcheck_statusmightrequire‘product_name’and‘user_id’toobtaincompleteinformation.

Training

TheAPI.AIAgentadoptsadvanceddeeplearningtechnology,andthusthetrainingprocessisoneofthecriticalconfigurationstepsfortheagenttofunctionefficiently.AfteralltheintentsandentitiesarecreatedbytheGLOCOdevelopers,severalsampleuserstatementsareprovidedtotraintheagenttodetecttheuser’spurposeandcontext.Theagentcouldbeabletoautomaticallydetectuser’sintentandentitiesfromwhatusersaidaftertrainingit.Asadeliverableproduct,theexpectedfunctionalityisasfollowing:

• Whentheusersendsarequestinnaturallanguage,theagentisabletodetecttheintent,andcall-upintentwithJSON.

• WhenIntentiscalled,alistofcorrespondingentitieslinkedwiththisintentarealsocalled-upandcheckedonebyonetoseeiftheuserprovidedrelatedinformation.

• Ifinformationforanyrequiredentity(orparameter)wasnotprovided,theagentasksfurtherquestionsfortheusertoprovideadditionalinformation.

• Ifallentitiesarefulfilled,theagentisreadytosendintentandentityinformationtotheCUIControllerforfulfillmentoftherequestanditwaitsforaresponsetocompletetheaction.

ThetrainingprocessisthekeyenablerfortheAgenttoachievehumanlevelconversationwithhighlevelintelligence.Bysayinghumanlevelconversation,wewillnotlimituserstofollowapredefinedsentencestructuretocommunicate;instead,GLOCOAgentwillbetrainedtobeabletounderstanddifferentformofhumanexpression.Withacomprehensivetrainingforthemachinelearningprocess,bythetimeofactualproductreleaseandimplementation,theGLOCOAgentwillbeabletodetectandrelatesynonyms,understanddifferentspokenlanguage

16

structure,thusunderstanduserwhentheysaythesamemeaningwithdifferentexpression.Additionally,theAgentisintelligentenoughtocollecterrors,learnfrommistakes,improvefromitslearningalgorithm,andneverperformthesamemistakeagain.Thus,withacontinuouslearningprocess,ourAgentwillbeabletomeetuser’sexpectationandincreaseuserexperience.

Figure4demonstratesabasicviewoftheAPI.AIconsoleandthetrainingprocess.Theagentisnamed‘GLOCO_English’,andtheintent‘intent_order’andalistofentitiesarecreatedaswell.Theagentislearningtocaptureentityinformationfromprovidedsentences.

Figure6:Api.aiconsolereview

3.2.5SystemMetrics

API.AIoffersdifferentpricingpackageswithdifferentsupportingmetrics.Intheearlyimplementationphase,weplantousethestandardversionwhichprovidesthefollowingmetrics:

• 750,000queriesperyear• Unlimitedprivateagentsandpre-builtdomains• Custommodelsavailableforspeechrecognition(significantlyincreaseaccuracy)• GuaranteedtomeetSLA

Duetothelargesizeofourcompanyandlargeamountofpotentialusers,wedoexpectourqueriesperyeartogrowrapidly.Wewillmonitortheamountofthequeries.Aslongasthenumberisgettingclosetotheservicelimit,wewillupgradeourpackageaccordingly.

17

3.3 IntegrationwithApplicationsandDataSources

Belowisalistofapplicationsandcomponentsthatwe’reintegrating:

• Api.aiagent• Mobileapp• Webapplication• CUIController• GLOCOConsolidatedAPIGateway• CRM• CUIRequestDatabase

TheCUIController,API.AIagent,andtheCUIRequestDatabasearebeingnewlyintroducedtoGLOCO.Allremainingapplicationsareexistingapplicationsandarealreadyfullyintegratedtogether.Therefore,we’llrestrictourdiscussionaroundtheAPI.AIagent,CUIcontrollerandthegatewayinterface.

GLOCOhasawebapplicationandamobileapp.GLOCOwillcreatetheagentusingtheAPI.AIdeveloperconsoleandusetheagentwiththeJavaScriptSDK.TheagentwillbeembeddedinGLOCO’sexistingwebsite.Fortheexistingmobileapp,thewebapplicationwillsimplybeembeddedintothemobileappforAndroidandiOSdevices.ForAndroid,theWebViewclasscanbeused,andtheWKWebViewclasscanbeusedforiOS.

TheCUIcontrollerwillintegratewiththecompany’sexistingCRMthroughtheGLOCOAPIGatewayusingtheHTTPprotocol.ThisRESTfulAPImakesHTTPrequestsandexchangesdatausingJSON.ARESTfulAPIisanapplicationprograminterface(API)whichusesHTTPrequeststoGET,PUT,POST,andDELETEdata.TheGLOCOAPIGatewayrelaysthecontextofthequeriesandresponsesbetweentheControllerandexistingCRMandfinancialenterprisesystems.PleaserefertotheFigure7belowforfurtherexplanation.

3.3.1QueryProcessing

Hereisalistofsomeofthequeriesthatmayoriginatefromtheconversationwiththecustomer:

• Enquireaboutaproduct• Placeanorder• Trackshipment• Queryaboutpreviousorders

18

Figure7–QueryProcessing

Queries

1:Enquireaboutaproduct 1a:querycontextsentfromCUIRestInterfacetoGateway 1b:querycontextsentfromGatewaytoCRM 1c:contextofresponsesentfromCRMtoGateway 1d:contextofresponsesentfromGatewaytoCUIRestInterface

2:Placeanorder (sameas1ato1d)3:Trackshipment (sameas1ato1d)4:Reorderthesameproduct (sameas1ato1d)

ThesequeriesmaydynamicallyincreaseinnumberoveraperiodoftimebecauseApi.aiwilllearnmoreabouttheusersandtheproductswithitsmachinelearningcapabilities.

WhenausermakesarequesttotheCUI,thespeechwillbecapturedasaWAVfileonanycommonbrowser.UsingPOST,thatWAVfileissenttoApi.ai,whichthencompletesspeechtotextconversion.ThetextmaynothavethecompletedetailsforthequerytobeexecutedintheCRM.Api.aihasthecapabilitytogetthemissingparametersfromtheuserthroughaconversation.Itwillcompletetheparametersforeachquery.ThisquerywillthenbepassedtotheGLOCOConsolidatedAPIGateway.

19

Example:Customersays,“Iwanttopurchaseapump.”Atthisstage,youonlyknowthecustomerID,nottheproductID.API.AIwillaskfurtherquestionstothecustomertogetotherdetailsrequiredtoprocesstheorder.

3.3.2AnalyticsandVisualization

TheexistingdashboardintheCRMwillcontinuetoshowalltherelevantinformationlikesalesthroughdifferentchannels(throughmobileapp,webapplication,orviaphonecall).

TheCUIRequestDatabasecapturesallthenaturallanguageverbalandtypedqueriestotheapplication,whichisanonymizedandrecordedforanalytics.TheinformationofthequeryissenttotheCRMviatheGLOCOConsolidatedGateway.AnalyticaltoolsaccesstheCUIRequestDatabaseandpresentthereportsonthedashboardandhelptrackthetrendofCUIuse.

3.4 DataDesign&Management

TheCUIenhancementforGLOCOwillattempttominimizetheintroductionofnewdata-at-rest.GLOCO’sexistingEnterpriseSystemswillcontinuetoserveastheSystemsofRecordforallimportantcustomerinformationandhistoricaltransactiondata.Instead,theCUIenhancementwillintroducenewentitiesprimarilyfordata-in-useinteractionsbetweentheCUIandtheexistingGLOCOAPIGateway.Theonlynewstoredentitieswillbetheanonymizednaturallanguagequeries.

3.4.1Entities

Inquiry:# FieldName Type Description1 Id String Identifierfortheinquiry2 Keyword String Keywordusedtosearchforaproduct3 Color String(optional) Desiredcolor4 Quantity Number(optional) Desirednumber5 Size String(optional) Desiredsize6 Full_text Memo Fullnaturallanguagetextofinquiry7 Success Boolean WasthisinquiryunderstoodbytheCUI

Table2

ThecustomerusesnaturallanguagetomakeproductinquiresthroughtheCUI.ThenaturallanguagerequestsareprocessedbyApi.aiandsentbacktotheController.TheControllertranslatestheinquiriesintoInquiryentitiesandforwardsthemtotheGLOCOAPIGateway.Thisdataisstored.

20

Product:# FieldName Type Description

1 Id String Identifierfortheitem2 Title String Titleoftheitem3 Description Memo Descriptionoftheitem4 Category String Categoryofitem5 Link String URLtoitem6 Availability String Availabilitystatus7 AvailabilityDate Date(optional) Dateavailable8 Price Number Priceofitem9 Currency String Currencyofitem10 Color String(optional) Colorofitem11 Gender String(optional) Genderitemisfor12 Age_Group String(optional) Agegroupitemisfor13 Material String(optional) Materialconstructedfrom14 Size String(optional) Sizeofitem15 Tax Number Applicabletax

Table 3

TheProductentityisusedinresponsetocustomerproductinquiries.InquiriesaremadethroughtheCUIandpassedthroughtotheGLOCOAPIGateway.TheGatewayqueriesGLOCOlegacyEnterprisesystemsandrespondswithaProductentityrepresentedasJSON.Thisdataisneverstored.

Order:# FieldName Type Description1 Id String Identifierfortheorder2 Customer_Id String Identifierofcustomerplacingorder3 Product_Id String Identifierofproductordered4 Quantity Number Numberpurchased5 Color(optional) String Colorofitem6 Size String Sizeofitem

Table 4

TheOrderentityisgeneratedbytheControlleronceacustomermakesanorderrequest.TheControllersendstheorderentrybacktotheCUIforconfirmation.Theorderdetailsarethenpresentedtotheuserverballyand/ortextually.Oncethecustomerhasconfirmedtheorderdetails,theorderispassedtotheGLOCOAPIGatewayforprocessing.Thisdataisneverstored.

21

Shipment:

# FieldName Type Description1 Id String Identifieroftheshipment2 Street String Streetaddress3 City String City4 State String Dateavailable5 Country String Priceofitem6 Zip String Currencyofitem7 Tracking_number(optional) String TrackingnumberfortheshipmentTable 5

TheShipmententityisreturnedfromtheGLOCOAPIGatewayonceanorderhasbeensuccessfullysubmitted,processed,andshippedbytheGLOCO’sexistinglegacyordermanagementsystem.TheShipmentisforwardedtotheControllerandtheControllerusesittohavetheCUIpresentthecompletedshipmentdetailstotheusereitherverballyortextually.Thisdataisneverstored.

3.4.2Storage&Analytics

TheCUIenhancementonlyaddsstorageforcustomerinquiries,boththosesuccessfullyprocessedbytheCUIandthosethatarenot.TheseInquiryentitieswillbestoredinaNoSQLkey-valuestoredatabaseusingMongoDB.Allstoreddatawillbeanonymizedandencrypted.

ThestoredInquiryentitiessupplementedwithregionandtimestampinformation.Theentitieswillbeusedtogainanalyticsaboutthetypesofqueriesmade,thelanguageandphrasesmostoftenusedwhenandwhere,andtoimprovetheuserexperiencebyfindingedgecasesandnewphrasestoprocess.

3.5 SolutionDemonstration

BelowisabasicdemoshowinghowtheGLOCOCUIwouldrespondtocustomerwhowantstoplaceanorder.Thisdemoagentcanperfectlyunderstanduser’sintent(Order),collectrequiredinformation,andcompleteorderthroughhuman-dialoguestylecommunication.Inordertoclearlyexplaintheworkingalgorithm,onlyonefunctionisdemonstratedhere.However,theactualproductwillbefarmorecomprehensiveandmulti-functionalthanthissimpledemo.

22

Screenshot1:GLOCO_EnglishAgentinterface.Usercaneasilystarttochatbyclickingthemicrophoneicon.

Screenshot2:Userwantstobuyaglucosemonitor.Theagentunderstoodtheuser’sintent,checkedalltheentitiesandfoundoutthattheparameter“address”ismissing.Therefore,agentasksusertoindicateaddress.

Screenshot3:Theuserindicatedtheaddress,agentcheckthroughentitiesagain,andfoundoutthatshippingmethodisstillmissing.Soagentaskedabouttheshippingmethod.

23

4. ImplementationPlan

4.1 SolutionDeliveryRoadmap

4.1.1DeliveryStrategy

TheCUIenhancementwillbedevelopedanddeployedusingagilepractices.Sprintswilltypicallybeonemonthinlength.Activitiesthatcanbereasonablydividedtooccurinparallelwillbesodivided.

Attheendofeachsprint,adeliverableproductwillbedemonstrated.Softwareandconfigurationproductswillbedemonstratedtoanaudience totheextentpossiblegiventhefunctionalityavailable.Serviceswillbebroughtonline,performancemetricscollected,andeventmonitoringwillcommence.Documentationwillbepresentedandsubjecttopeerreview.

4.1.2DeliverablesSprint# Deliverable Type Description Team(s)

1 CUIHostingEnvironments Service

Setupandtesthostingenvironmentsfordevelopment,test,andproduction

EnterpriseArchitectSolutionArchitect

2 CUIController Software Performuserrequests/requests Solution

Screenshot 4: agenthasalltheinformationneededtocompleteanordernow,itsentalltheinformationtotheCUIController,andrepliedtotheuserwithaconfirmation.

24

frommobileandwebtotheApi.ai ArchitectDevelopmentTeam

3MobileIntegrationComponent

Software AddCUItotheinterfaceoftheexistingGLOCOmobileapp

SolutionArchitectDevelopmentTeam

4 WebIntegrationComponent Software AddCUItotheinterfaceofthe

existingGLOCOwebsite

SolutionArchitectDevelopmentTeam

5 Conversationalscripts Documentation

ScriptsthatdetailsupportedconversationsthatmayhappenbetweenusersandtheCUI

AgentDevelopmentTeam

6 CUIApi.aiAgents ConfigurationCreateagentstoimplementtheconversationalscriptsineachtargetlanguage


7 ReorderScriptImplemented Configuration Createanddemonstratereorder

functionality

AgentDevelopmentTeamDevelopmentTeam

8 AvailabilityScriptImplemented Configuration Createanddemonstrateproduct

availabilityfunctionality


9 OrderScriptImplemented Configuration Createanddemonstrateneworder

functionality


10 CUIDatabaseandDashboards Software Createanddemonstratedashboard

andanalyticsfunctionality

DevelopmentTeamDatabaseManagementTeam

11 StaffOperationalSupportTrial Service Staffaretrainedanddemonstrate

CUIsupportscenarios

QATeamReliabilityEngineeringTeam

Table6

25

4.1.3Schedule

Figure 8 - Schedule

4.1.4ServiceTransition

TheCUIenhancementforGLOCOisagreenfielddeploymentthatislayeredontopofexistingsystems.Assuch,therenodowntimeorservicedegradationisrequiredforexistingGLOCOsystemstodeploythenewservice.

Thetransitionwilloccurseamlesslyandwillbecomeliveasthewebsiteandmobilecomponentsaremadeavailabletousers.Thecomponentswillbepilotedin4phases.Theuserpopulationwithaccesstotheenhancementwillbeexpandedateachphase.First,10%ofuserswillhaveaccess,then25%,50%,andinthefinalphase100%.Ateachphaseperformanceanduseradoptionrateoftheenhancementwillmeasured.

Ifanyproblemsareencounteredorrisksrealizedduringrollout,therollbackplanwillbetodisablecomponentavailabilityuntiltheissuescanberesolved.Onceresolved,componentswillbere-enabledandtherolloutpilotresumed.

4.1.5DataTransition

TheCUIenhancementforGLOCOdoesnotrequirethemigrationofdatafromexistingsystems.AllCUIenhancementcomponentswillpulldatafromsystems-of-recordthroughtheGLOCOConsolidatedGatewayAPI.

TheCUIdoes,however,requiretheintroductionofanewdatarepository,theCUIRequestDatabase.Atthetimeofdeployment,theCUIRequestDatabasewillcontainnoexistingdata.Thesystemwillbetestedtoensurethatthestorageofnewdataissuccessful,butnomigrationordatapopulationisrequiredatthetimeofdeployment.Subsequenttodeployment,allrequestswillberecordedthere.

26

4.1.6TransitionRisks# Risk Impact Likelihood Mitigation

1 Cloud-basedApi.aiisunabletoservicethefullvolumeofrequestsinamannerthatmeetsuserperformanceexpectations

High Low MigratetoahighperformancelocallyhostedApi.aiinstance

2 ChangestoApi.aiprogramminginterfacecausealossofservice

High Medium Withinthecontroller,asinglemoduleprovidestheinterfacetoApi.aiwithrigorousautomatedteststhatactivelyrunandmonitored

3 CUIenhancementcomponentstoexistingappsaredifficultforuserstofind/use

High Medium Usabilitytrialsareconducted.Componentsareinstrumentedsothatusagetelemetrycanbecollected

4 Api.aispeechrecognitionfallsshortofuserexpectationsforsomelanguages/users

Medium Low SamplesofuserqueriesaretakenfromCUIRequestDatabaseandusedforApi.aitraining

5 GLOCOConsolidatedGatewayAPIdoesnotsupportalltherequiredinterfacestoexistingsystems-of-record

Medium Medium EngageICTGLOCOConsolidatedGatewayAPIteamearlywithinformationonproposalandrequiredinterface.BepreparedwithfinancialresourcesinreserveincaseGatewayrequiresexpansion

6 ExistingnetworkbandwidthisunabletohandlethevolumeofaudiorequeststoApi.ai

Low Low IncorporatequerycachingandvolumethrottlingintoControllerandbepreparedtoupgradeexistingbandwidthtoprocessthevolumeofrequests

Table 7

4.2 Operationalization

AsthecentralfocusoftheCUIistoimprovetheoveralluserexperience,operationalizingthesystemwithoutcausingimpactisparamount.Fromtheuserperspective,GLOCOalreadymaintainsaglobalservicedeskwitha“followthesun”supportmodelsothatusersinallgeographicareasandtimezonereceiveequalsupport.ThetechnologiescomprisingtheCUIrelyheavilyonmachinelearningtofunctioncorrectly.Thishasthepotentialtocauseanon-trivialpercentageoferrorswhichcouldleadtouserfrustration.Wewillrelyonthepre-existing“livehelp”featurealreadyembeddedintheGLOCOmobileapplicationssothattheuserscangetsupportfromalivepersonifthesystemisn’tworkingcorrectly.

Allteamsinvolvedwillhaveatleasttenpeopleineachteam.ThedevelopmentoftheCUIsystemwillbemanagedbytheCUIDevelopmentTeam,anewteamwithinGLOCOICT.This

27

teamwillberesponsibleforinitialdevelopmentaswellasdesigningnewfeatures.Day-to-dayoperationswillrequiretheformationofanewReliabilityEngineeringteam.TheReliabilityEngineeringteamwillberesponsibleformanagingtheinternalinfrastructurerequiredtoruntheCUIaswellastheconnectionstothethirdpartyvendors.Thisteamwillberesponsibleforconfiguringthecorrectalerting,monitoring,andlogaggregations.

TheReliabilityEngineeringteamwillownthereleaseandmaintenanceprocessfortheCUI.TheywillworkcloselywiththeCUIDevelopmentTeamtodetermineanappropriatereleasecadenceandtounderstandallchangestothesystem.TheywillalsoberesponsibleforfollowingtheincidentmanagementprocessforanyissuesthatoccurwiththeCUIsystem.

Role/Team ResponsibilityProjectSponsor(GLOCOCTO) Ensuretheprojectdeliversthebusinessbenefits

ProjectManager Hasoverallresponsibilityforplanning,procurement,andexecutionofproject

EnterpriseArchitect IntegrationofCUIwithexistingsystemsSolutionArchitect Createblueprintofthesolution

DevelopmentTeam

• IncorporateadditionalcodeforCUIintoGLOCOapp• DevelopControllertohandleroutingtoandfromAPI.AIagent,CUI

RequestDatabase,andGLOCOapp• Developforfront-endofwebandmobileappstoincludeCUIoption

DatabaseManagementTeam BuildCUIRequestDatabase


• Designandproducetheagent,itsintentsandentities,contexts,etc.usingtheAPI.AIdeveloperconsole

• EmbedtheagentintoGLOCO’sapps• FinalizeintegrationoftheCUIwithexistingsystems

QATeam TesttheGLOCOappwiththeintegratedCUItoidentifyandpreventanyissues

ReliabilityEngineeringTeam ReleaseandmaintenanceprocessfortheCUI

Table 8

4.2.1SupportingNon-FunctionalComponents

Thenon-functionalcomponentslistedbelowwillhelpwiththeCUIimplementation.

• UserAdministrationConsole:Thiswillhelpglobaluseradministratorswithcertaintaskssuchasviewinginformationonusers,creatingnewusers,resettinguserpasswords,andsearchingforusersbyuserattributes

• CUIRequestDatabase:OneofourcorecomponentstheCUIRequestDatabasewhichstoresallrequestsandtransactions.Thisdatabasewillallowoursupportpersonnelto

28

seamlesslycontinueatransactionthathadbeenstartedduringaCUIsession.Oneofthepre-launchtaskswillbetoproviderobusttrainingfortheGLOCOsupportteamstobeabletoquicklysupportuserswhentheyarehavingissues.

4.2.2ChangeManagement

ThesamechangemanagementprocessGLOCOhaswillstillbefollowed.OncetheCUIhasbeenintegratedintoGLOCO’sapps,additionalchangesareexpectedtobemadefortheagentwhenitreceivesmorerequests.Theagentmaynotunderstandsomeofthoserequests.Alltherequests,includingtheunhandledrequestsarewithintheCUIRequestDatabase.Achangerequestmustbeplacedtobereviewedbythechangeadvisoryboard(CAB).IftheCABapproves,schedulesaremadeforthechangerequesttomovetoproductionforthenextrelease.Everymonth,additionalchangeswillfollowthestepsprovidedbelow:

1.IdentifyandextractunhandledCUIrequests

2.Traintheagentbyprovidingadditionalnecessaryintentsandentities

3.Performtestingwiththemodifiedagent

4.2.3ServiceLevelAgreements

HavingaservicelevelagreementisvitalasGLOCOisrelyingonApi.ai’sservicetointegratetheCUIfeaturestotheapps.GLOCOwillselecttheappropriatepricingplantohaveaservicelevelagreement(SLA)withApi.aitoestablishexpectationsintermsofApi.ai’sperformance.Theyprovidea99.99%guaranteeduptimeasstatedontheirwebsite.

4.3 UserEnablement

GLOCO’sCUIinterfacechangesthewaycustomersinteractwiththeGLOCOwebsiteandmobileapplicationtomakeneworders.Therefore,GLOCOICTdevelopedthebelowplantoensurethecustomers’adoptionofthenewCUIfeature:

• UserExperienceTesting:AheadoftheCUIfeaturerelease,GLOCOwillconductuserexperiencetestingwithalimitedgroupofusers.Theuserswillbeaskedtosharefeedbackabouttheirexperiencewiththetool.ThiswillhelpGLOCOidentifyissuesandareasofimprovementfromtheuser’sperspectiveearlyon.

• MarketingCampaign:ThecampaignisintendedtoinformGLOCOcustomersofthenewlyintroducedCUIfeatureandthebenefitsofusingit.Thecampaignwillconsistofthefollowing:

29

- VideoadvertisementdemonstratingtheCUIuse.ThevideowillbefeaturedonGLOCO’swebsite,mobileapplicationandsocialmediapages.

- Customerswillreceiveemailsandnotificationsfromthemobileapplicationintroducingthemtothenewfeature.

- Userincentives:GLOCOwillofferdiscountsforthefirst100usersthatusetheCUIfeaturetocompleteapurchaseorder.

• UserGuides:VideoandillustrateduserguideswilldemonstratetheuseofCUItomakeapurchase.Thesewillbemoreelaboratethantheadvertisement.TheywillbeavailableontheGLOCOwebsite,mobileapplicationandsocialmediapagesfortheuser’sreference.The“FrequentlyAskedQuestions”sectioninGLOCO’swebsitewillbeupdatedwithexpecteduserconcernsaroundtheCUIfeature.

• UserFeedbackthroughFeedbackRatingsandReports:CustomersthatusetheCUIfeaturewillberequestedtorateitonascalefrom1–5stars.Theywillalsohaveanoptiontosharecomments.GLOCOwillusethefeedbackreceivedfromthesedirectfeedbackchannelsaswellasfeedbackreceivedinemailsorcallstoGLOCO,toidentifyareasofimprovingtheCUI.

• TrackandMeasureMetricstoGainGreaterVisibility:GLOCOwillsettargetsforuseradoptionoftheCUItoolinthefirst6monthsofthereleaseanditwilltrackthenumberofordersplacedthroughtheCUIandcompareittothetargetstotrackuseradoptionofthefeature.Itwillalsotracknumbersofusersthatshowinterestintheadvertisement,refertotheuserguides,andsharefeedback.

Additionally,GLOCOalsotailoredtrainingfortheinternaluserswhowillbeprovidingsupportfortheCUI,aswellashigh-levelfamiliarizationtrainingforthemanagementteam.ThetablebelowshowstheGLOCOinternalfunctiongroupsthatwillreceivetraining,thetopictheywillbetrainedonandtheirtrainingmethodology.

UserGroup TrainingTopics TrainingMethod

ITHelpDeskAgents CUIsystembasictroubleshooting Formal2daytrainingcourse,Documentation.

CustomerServiceAgents BasicCUIfunctionality Documentation,UserGuides,Meeting.

CUISystemAdministrators

CUIadvancedconfiguration,API.AInaturallanguageprocessingandfulfillment,CUIcontrolleroperationandfulfillment.

Formal2weektrainingcourse,documentation.

ManagementTeam CUIDashboard Formalhalfdaytraininganduserguides.

30

4.4 SuccessMetrics

Thefollowingbusinessandtechnicalsuccessmetricstableisadjustedbasedoneachbusinessgoalsandtechnicalcategories.Duringthefirstyearofimplementation,theproductemphasiswillbeonCUIbasicfunctionalityanduserlearningprocessandadoption.Thusarelativelysmallincrease/decreaseinmetricsisexpected.Over2yearsofimplementation,theCUIproductismatureinmostfunctionality,andthekeyfocusshiftsfromlearningprocesstouserexpansion.Thus,higherbenefitsareexpected.

BusinessSuccessMetricsTable

BusinessGoal SuccessMetric-1Year SuccessMetric-2Years

IncreasecustomersatisfactionwhendealingwithGLOCO

Increasenetpromoterscoreby5%


Driveconsumersfromallmarketsegments(includingelderlyanddisabledconsumers)tousethemobileandwebapplicationcapabilities



Gaincompetitiveadvantageoverclosecompetitors

Increasemarketshareby10%


MaketoolseasiertouseforcustomersIncreasecustomersatisfactionsurveyscoresby5%

Increasecustomersatisfactionsurveyscoresby20%

Reducerequestsforcustomerservicesupport Reducecostforcustomerservicestaffby10%

Reducecostforcustomerservicestaffby15%

Increaseonlinesalesthroughthemobileappwithverbalrequests



Table10TechnicalSuccessMetricsTable

Category SuccessMetric-1Year SuccessMetric-2YearsIncreasefunctionsandtaskstheGLOCOAppsupports.

IncreasetheavailabilityoftheAppfunctionby5%

IncreasetheavailabilityoftheAppfunctionby20%

EnableeasierproductorderingandrelatedqueriesviaCUIfunctions.

Increasetheperformanceoforderingprocessby5%


IncreaseuserexperiencewhileusingtheAppbyconversationalandvisualfunctions.

IncreaseuserexperiencewiththeCUIby10%


Reducethetimespendingforuserstocompleteacertaintask.

Reduceaveragetimespendingpertaskby10%


Table11

31

GLOCOhasitsspecificmethodincollectingdataandmonitoringsuccessmetricstoensuresuccessfulimplementationoftheCUIproduct.Inthefollowingmeasurementsandevaluationtable,foreachsuccessmetricslisted,typeofdata/informationthatisneededisdescribed,andmethodsforcollectingsuchdataandanalyzing/evaluatingspecificmetricsarelisted.Asaresulteachmetriciscontinuouslyandcarefullymonitored,anyproperadjustmentsorchangescanbemade.

4.4.1MeasurementsandEvaluationsofSuccessMetrics

SuccessMetrics WhatNeedstobeMeasured Howitwillbemeasured


Customersatisfactionratingand+/-forcustomervolumenumber.

Customersatisfactionratingisobtainedfrommonthlycustomersurveyandmarketingsurvey.Customervolumenumberiscollectedfromuserdatabase.Trendforpotentialusersandusersatisfactioncanbepredictedfromabovedata.


Trendforapptrafficvolume.Customervolumebydifferentcategory.

Apptrafficvolumeiscontinuouslymonitoredbyexistingtrafficmonitorengine,anddetaileddataisstoredindatabase.Cananswerquestionslikeisthetrafficfromreturninguserornewuser,isthetrafficaddingvalue,andhowmuchtrafficiscontributedbycertainusergroup?


Currentmarketshareandtrendformarketsharechange.

Marketshareiscontinuouslymeasuredbymarketingdepartmentthroughresearchesandanalysis.Currentanalysisandpredictivetrendanalysisareavailable.

Increasecustomersatisfactionsurveyscoresby10%

Customersatisfactionsurveyscore

Customersatisfactionsurveyisprovidedonceamonth,andAppratingiscollectedrealtimeafterusersleavefeedback.Trendanalysisisdonewithabovedatatokeeptrackingcustomersatisfactionscore.

Reducecostforcustomerservicestaffby10%

Customerservicestaffnumberandcost.

Numberofuserqueriesandproductiveworkinghourforeachservicestaffisreported.Actualrequiredcustomerservicestaffnumberandservicehourscanbecalculated,andcostcanbecalculatedaswell.


NumberoftransactionsthatcompletedviaApp,droppedhalfway,orturnedtoservicestaff.

NumberoftransactioncompletedordroppedhalfwayviaAppisrecordedatthedatabase,andcustomerturnedfromApptoservicestaffisrecordedbyservicestaff.Conversionrateiscalculatedfromabovedata,andatrendcanbegeneratedbyanalysis.

Increasetheavailabilityofthe

NumberoffunctionsAppisabletoperform,andwhich

UsageofAppbyeachuserisrecordedinthedatabase.

32

Appfunctionby5% onesaremostutilizedwithusers.

Bycomparingavailablefunctionsandtheactualadoptionofeachfunctionbyusers,theavailabilitycanbeanalyzed,andpotentialissuecanbedetected.


UseradoptionlevelwithCUIorderingprocessandfeedback.

TimeandlengthofeachuserusageforCUIwillbecollected,andaveragelearningcurvedatawillbecollectedaswell.Feedbackwillalsobecollectedtomonitortheperformance.


UsersatisfactionsurveyscorewithnewCUIfunctionandfeedback.

UsersurveyspecificallyforCUIwillbegivenforeveryuseraftereachversionrelease.Customerfeedbackanduserexperienceratingwillbecollectedandanalyzed.


Averagetimeusersspendforcompletingtasks.(Categorizedbytask.)

Timeforeachcustomertocompleteeachdifferenttaskiscollectedandaveragetimeiscalculated.

Table12

Team4 Final v1 2016 Fall - Harvard University Velukumar Victor Viramontes 2 ... to first access...

Documents

Transcript of Team4 Final v1 2016 Fall - Harvard University Velukumar Victor Viramontes 2 ... to first access...