Quro: Facil i tat ing user symptom

check using a personalised chatbot -

oriented dialogue system

July 2018

2018

Shameek Ghosh , Sammi Bha t ia , Abh i Bha t ia

Outline

• Understanding a patient’s journey

• Background of the Problem

• Facilitation of symptom check using Quro

• Solution Description

• Experimental Results

• Conclusion

Feel Sick

Online Search

Appointment

Assessment

DiagnosisPatient jumps online,

starts searching about

symptoms, gets anxious

from incorrect generic

results.

Patient books an appointment or

walks in, describes the

symptoms.

Doctor asks about medical

history, conducts

examination.

Doctor offers a diagnosis

and explains context.

Patient experiences

symptoms, concern about

certain conditions.

Prescription

Doctor prescribes

medications, refers to

specialist, requests tests

or other responses

Driven By DoctorDriven By Patient Driven By PatientMedius – Technology Nurturing Humanity

A Patient’s Journey TODAY

1. https://www.healthdirect.gov.au/health-information-online-facts-or-fiction

2. http://6cpa.com.au/wp-content/uploads/National-trial-to-test-strategies-to-improve-medication-

compliance-in-a-community-pharmacy-setting-Full-Final-Report-.pdf

Pre Consultation

84% of users of the internet incorrectly search for health

related information online1

During Consultation

75% of all GP visits are for minor ailments, repeat

prescriptions, referrals - a heavy burden on the

healthcare system1.

Post Consultation

$1.2b per year is the cost to Australian healthcare

system due to non adherence of medication or treatment

regimes - triggering readmissions2.

Problem

Leading causes

of

death cancer

585,000

diagnostic

error

251,000

heart disease611,000

No of doctors per 1000

population

3rd Leading Cause of Death

0.59 0.641.1

2.15

3.313.59

3.9

4.78

India SouthAfrica

China UnitedKingdom

UnitedStates ofAmerica

Australia Germany Spain

Primary Healthcare is

crippled with myriad

problems

• Overburdened Doctors.

• Limited time with patients.

• Delayed Diagnosis.

• Unsatisfied Patients.

• Treatment Non-Adherence.

The problem with rule -based models

Symptom check by Quro• A persona l hea l th ass is tan t pow ered by

Ar t i f i c ia l In te l l igence and deve loped by

Med ius Hea l th .

• Quro is a goa l -d i rec ted conversa t iona l -

bot fo r p r imary care .

• Quro exp lores user ’s symptoms, ident i f i es

l i ke ly causes o f cond i t ions and he lps

them dec ide w hat to do nex t and w here to

go .

7M+ data points describing therelationships between…

• 8K+ Symptoms

• 2K+ Diseases

• 4K+ Causes & Risk Factors

Large-scale Data Extraction

and Careful Curation

Salient Features of the Quro Chatbot and Triage System

Online instant medical triage: Online users want to know if their conditions require going to emergency care, visiting the GP, or remain at home and rest

Improving engagement through an easy-to-use user interface and better user experience

Using natural language processing to make sense of the user’s demands followed by sequential symptom question answering using a medical knowledge graph

Ensemble models involving disease text embedding models for generation of a shareable pre-assessment report for a user for sharing with a GP

Overall architecture of Quro

Represent words in a continuous vector space

For each word, the vector could reflect syntactic and semantic patterns such as thedegree of similarity between words

Neural Network is used to generate a vector matrix for word text in the corpus

Visualizing and clustering medical text

Word Embedding Models

Continuous evaluation process for condition pre-assessment:

Current status

Evaluation Criteria

At least 1 of the top 3 reported conditions is a correct assessment

2 out of 3 reported conditions were expected conditions by our in-house clinical experts

Datasets used for testing

30 clinical vignettes curated by internal experts from primary case notes

On-going evaluations across 10 diseases

Evaluation Results for triage pre-assessment: Current status

Initial evaluation using 30 patient vignettes in two different test criteria, showed an accurate outcome in 25 out of 30 cases (83.3%) and in 20 out of 30 cases (66.6%).

Investigated Disease

Infectious Gastroenteristis (IG)

Cholecystitis (CTS)

Pelvic Inflammatory Disease (PID)

Benign Prostatic Hyperplasia (BHP)

Celiac Disease (CD)

Ulcerative Colitis (UC)

Menopause (MNP)

Gastroesophageal Reflux Disease (GERD)

Polycystic Ovarian Syndrome (PCOS).

Irritable Bowel Syndrome (IBS)

Urinary Tract Infectious (UTI)

On-going Study: List of diseases for word embeddings

Predict/True IG UTI IBS BPH GERD PCOS CTS PID CD UC MNP Precision Recall

IG 12 0 0 1 0 0 1 0 0 0 0 0.667 0.857

UTI 0 9 1 1 0 0 1 0 0 2 0 0.818 0.643

IBS 4 0 10 0 0 0 0 0 0 0 0 0.833 0.714

BPH 0 1 0 12 0 1 0 0 0 0 0 0.75 0.857

GERD 0 0 0 0 13 0 1 0 0 0 0 0.928 0.929

PCOS 0 1 0 0 0 10 0 1 1 0 1 0.909 0.714

CTS 1 0 0 0 0 0 12 0 0 1 0 0.706 0.857

PID 0 0 0 0 0 0 1 11 0 1 0 0.846 0.846

CD 1 0 0 0 0 0 0 1 10 2 0 0.833 0.714

UC 0 0 1 0 1 0 1 0 1 10 0 0.625 0.714

MNP 0 0 0 2 0 0 0 0 0 0 12 0.923 0.857

Accuracy 0.791

Initial Multi-class prediction results using Embedding Models

Future Work: Planned immediate improvements

Scale context embedding model to multi-hundred-disease prediction for 150 diseases activated in the Quro system

Integration of Quro knowledge graph with context embedding models

Further evaluations using expanded set of clinical vignettes across 150 diseases

Takeaways

Consumer focussed engagement for understanding user symptoms, their needs, and provide valuable informationto physicians for further inquiry

Proactive dynamic collection of illness narrative over time, prior to doctor appointments

Use of NLP entity recognition and relation extraction algorithms to determine initial entry points in knowledge graph

Graph reasoning engine for optimal sequential question answering

Automated data collection and expert driven primary case collection for development of disease prediction models

Thank you

Head Office

Level 25, Tower Three, International Towers Sydney,

Barangaroo

https://mediushealth.org/

Phone +61 430 450 204

shameek.ghosh@mediushealth.org