Rushin Shah, Engineering Manager, Facebook at MLconf SF 2017

Natural Language Understanding@ Facebook Scale

ENGINEERING MANAGER, FB

Rushin Shah

Our Goal:Understand textual content with near human accuracy at Facebook scale

Friends ML Conferences

¬

HockeyThe Daily Puck

The Canadiens ran through a high-intensity practice on Wednesday ahead of Thursday's game.

1 hr •

Ron Timpany, Dan Fell and 80 others

Like Comment Share

Montreal Hockey Insider

Canadiens on verge of clinching playoff spot, Lindgren called up.

1 hr •

Bill Russell, Joe Tony and 116 others

Like Comment Share

Benoit Dumoulin was watching Vancouver Canucks vs. Montreal Canadiens.

Yes! In OT!

1 hr •

Bill Russell, Joe Tony and 234 others

Like Comment Share

VAN 3 FINAL OT 4 MTL

NLU Tasks

• Text classification

• Word classification

• Content similarity

• Entity Resolution

Text Classification

Entity: Delicious Food

Jole Simmons1 hr •

Sarah Russell and 23 others 4 Comments

Like Comment Share

Topic: Cooking

I'm trying out this new recipe for a coconut curry tonight. It looks DELICIOUS!!!

Entity: Kaepernick

Jole Simmons1 hr •


Like Comment Share

Topic: Sports or Cooking ?

Last night's game was absolutely incredible. Once Curry gets cooking, there's no stopping that guy!

Word Classification

Content similarity

SIMILARITY0.75

MUCKBUCKET SUNSHINEMUSIC BAND

Parsing out Entities"DID YOU KNOW THAT MUCK BUCKET

SUNSHINE IS PERFORMING LIVE AT THE BOOM BOOM ROOM?"

TOPICLIVE EVENT

The Boom Boom Room1 hr •

Tonight before you Dance Dance Dance, join us for a special evening. Boom Boom Room Presents: Muckbucket Sunshine! Doors 8pm.


Like Comment Share

• Deep Learning For NLU

• ContinuousRepresentation

• Can Solve Hard NLP Problems

Natural Language Processing (Almost) From Scratch

DeepText

DeepText Features

Multiple Tasks Multiple Languages Multiple Architectures

Model Structure Learning Algorithm

Data Loader Tokenizer Feature Extraction

Model Training DeploymentFeat Prep

DeepText Platform architecture

DeepText Tasks

Sequence LabelingClassification

DeepText Document classification

DeepText Document classification - LSTM

Hidden

MLP

Messi todayscored

HiddenHidden

EmbeddingEmbeddingEmbedding

DeepText Document classification - CNN

Messi todayscored


MLP

CNN

DeepText Classification: FastText

Messi todayscored

Unigram Bigram Unigram Bigram Unigram

Output

Hidden

DeepText Word classification

MLP

CNN/LSTM

Messi

Embedding

today

Embedding

Classification(Messi)

MLP

Classification(today)

scored a hat-trick

DeepText Classification: Word classification

DeepText Content Similarity

Messi todayscored


CNN / LSTM

Messi todayscored


CNN / LSTM

MLP

Ranking loss

FC Barcelona

DeepText Entity recognition and linking

Real Madrid C.F.

Document

Candidate Selection

Mention Detection Disambiguation Entity

Annotations

Entity recognition and linking Architecture

Exploring Use Cases In Facebook

For Sale

Post for SaleNot Selling

Create a Sales post to sell your items faster. Only post as a discussion if you’re notselling something.

Are You Selling Something?

Social Recs

Scale MLExperts

Reuse Models

Optimize Labels

CLUE

CLUE

Label Efficiency

Active Learning

Democratize

Single entry point for NLU

Flexibility and Scale

Deep Text

Scale MLExperts

Reuse Models

Optimize Labels

ActiveLearning

Self Service With CLUE

TrainClassifier

LabelData

CollectData

(Search)

Review

Threshold

Prec

isio

n

POOLING

...

W1 W2 W3 W4 WN

CONVOLUTIO

I want some subway or burger king

I want some subway or burger kingSlots

[{

"name": "cu:restaurant""value": "subway""start": 12"end": 19"contextldx": 0

}{

"name": "cu:restaurant""value": "burger king""start": 22"end": 32"contextldx": 0

MODELS

200+

Rushin Shah, Engineering Manager, Facebook at MLconf SF 2017

Technology

Transcript of Rushin Shah, Engineering Manager, Facebook at MLconf SF 2017