Data Science Popup Austin: Using lda and Structural Topic Modeling to Explore Trending Topics in a...

DATA SCIENCEPOP UP

AUSTIN

Using LDA and Structural Topic Modeling to Explore Trending Topics in a Call Center

Jordana HellerData Scientist, Mattersight

jheller

http://www.dominodatalab.com

DATA SCIENCEPOP UP

AUSTIN

#datapopupaustin

April 13, 2016Galvanize, Austin Campus

Lightning Talk: Using LDA and Structural Topic Modeling to Explore Trending Topics in a Call CenterJordana Heller @jhellerData Science Pop-up Austin, April 13, 2016

©2016 Mattersight Corporation. Mattersight Restricted Confidential Information.

What We Do


Our goal: Topic Trends

3/31/2016 4/30/2016 5/31/2016 6/30/2016 7/31/2016

Identifying contents and prevalence of multiword topics present in conversation in an unsupervised way

Unexpected Prevalence Critical Spikes Escalating Frequency


Our goals, continued

Manageable number of topics

Track expected and unexpected topics

Go deep: Contextualize topic usage


Short text: Keywords, hashtags, ngrams


Long text: Could use predetermined topics

Image credit: IBM Watson Concept Insights


Long text: Or discover themes

Image credit: Blei, 2012, Communications of the ACM

Latent Dirichlet Allocation (LDA) (Blei et al., 2003)


Great! How about contextualizing trends?

• Where are topics trending?• Structural Topic Modeling (Roberts et al., 2013)

– Instead of relying on post-hoc comparisons, includes covariates in LDA model• Specifies priors as GLMs• Word distribution determined by topic, covariates,

topic-covariate interaction– Authors’ implementation: R package stm (available

via CRAN; all code on GitHub!)

Ready to talk pipeline!


Data Collection and Preprocessing

Read Transcripts

Add Call-level Covariates

Preprocess text

• Collocations• -Stop words• Stem/completion• -Low freq terms

Create Term-Document

Matrix


Topic Model Creation

Retrieve last topic

model

• For comparison

Create current

topic model

•Detect number of topics, or specify

Create topic labels


Topic Model Comparison

Inspect overall topic prevalence

Compare overall topic prevalence across periods

• Topics change! Measure change in word probability distributions for each new topic wrt each old topic

• Match new to closest previous match below change threshold (otherwise new topic)

• Evaluate trends!

Estimate and inspect effects of

covariates

Compare effects of covariates

across periods

•Output can be interpreted similarly to regression

Example results: Hotel reservations Covariates: booking, caller distress


Trend Contextualization: Booking

� convention, center, mind, worry, philadelphia, inventory� NewÄ DecreasingÃ Increasing

Hit: > 1% of words on call assigned to a given topic



� school, college, graduate, medical, clinic

� NewÄ DecreasingÃ Increasing




Ã30% beach, balcony, ocean, view





Ä10% back, next, receive, listen, cash future





� back, minute, system, run, inconvenience





Ã42% confirm, email, arrival, local




Trend Contextualization: Caller Distress


Distress: > 30 seconds of linguistically-identified dissatisfaction or negative emotion



� square, city, price, hotel, manhattan, central





Ä12% online, website, cancel, purchase, advance� NewÄ DecreasingÃ Increasing



Our goals, revisited

Manageable number of topics

Track expected and unexpected topics

Go deep: Contextualize topic usage


Topic trends using structural topic models

Thank you!

DATA SCIENCEPOP UP

AUSTIN

@datapopup #datapopupaustin

Data Science Popup Austin: Using lda and Structural Topic Modeling to Explore Trending Topics in a...

Data & Analytics

Transcript of Data Science Popup Austin: Using lda and Structural Topic Modeling to Explore Trending Topics in a...