DEANNA: Natural Language Questions for the Web of Data

Mohamed Yahya† Klaus Berberich† Shady Elbassiousni*

Maya Ramanath‡ Volker Tresp# Gerhard Weikum†

† Max Planck Institute for Informatics

*Qatar Computing Research Institute

‡IIT Delhi # Siemens

“Who played in Casablanca and was married to a writer born in Rome?”

“Classical” QA

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 2

“Who played in Casablanca and was married to a writer born in Rome?”

“Classical” QA

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 3

Ingrid Bergman (29 August 1915 – 29 August 1982) was a Swedish actress who starred in a variety of European and American films. She won three Academy Awards, two Emmy Awards, and the Tony Award for Best Actress. She is ranked as the fourth greatest female star of American cinema of all time by the American Film Institute. She is best remembered for her

roles as Ilsa Lund in Casablanca (1942), a World War II drama co-starring Humphrey Bogart and as Alicia Huberman in Notorious (1946), an Alfred Hitchcock thriller co-starring Cary Grant.

“Who played in Casablanca and was married to a writer born in Rome?”

“Classical” QA

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 4

Ingrid Bergman (29 August 1915 – 29 August 1982) was a Swedish actress who starred in a variety of European and American films. She won three Academy Awards, two Emmy Awards, and the Tony Award for Best Actress. She is ranked as the fourth greatest female star of American cinema of all time by the American Film Institute. She is best remembered for her

roles as Ilsa Lund in Casablanca (1942), a World War II drama co-starring Humphrey Bogart and as Alicia Huberman in Notorious (1946), an Alfred Hitchcock thriller co-starring Cary Grant.

In 1950, after a decade of stardom in American films, she starred in the Italian film Stromboli, which led to a love affair with director Roberto Rossellini while they were both already married. The

affair and then marriage with Rossellini created a scandal that forced her to remain in Europe until 1956, when she made a successful Hollywood return in Anastasia, for which she won her second Academy Award, as well as the forgiveness of her fans. Many of her personal and film documents can be seen in the Wesleyan University Cinema Archives.[4]

“Who played in Casablanca and was married to a writer born in Rome?”

“Classical” QA

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 5

Ingrid Bergman (29 August 1915 – 29 August 1982) was a Swedish actress who starred in a variety of European and American films. She won three Academy Awards, two Emmy Awards, and the Tony Award for Best Actress. She is ranked as the fourth greatest female star of American cinema of all time by the American Film Institute. She is best remembered for her

roles as Ilsa Lund in Casablanca (1942), a World War II drama co-starring Humphrey Bogart and as Alicia Huberman in Notorious (1946), an Alfred Hitchcock thriller co-starring Cary Grant.

In 1950, after a decade of stardom in American films, she starred in the Italian film Stromboli, which led to a love affair with director Roberto Rossellini while they were both already married. The

affair and then marriage with Rossellini created a scandal that forced her to remain in Europe until 1956, when she made a successful Hollywood return in Anastasia, for which she won her second Academy Award, as well as the forgiveness of her fans. Many of her personal and film documents can be seen in the Wesleyan University Cinema Archives.[4]

Roberto Gastone Zeffiro Rossellini (8 May 1906 – 3 June 1977)

was an Italian film director and screenwriter. Rossellini was one of the directors of the Italian neorealist cinema, contributing films such as Roma città aperta (Rome, Open City 1945) to the movement.

“Who played in Casablanca and was married to a writer born in Rome?”

“Classical” QA

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 6

Ingrid Bergman (29 August 1915 – 29 August 1982) was a Swedish actress who starred in a variety of European and American films. She won three Academy Awards, two Emmy Awards, and the Tony Award for Best Actress. She is ranked as the fourth greatest female star of American cinema of all time by the American Film Institute. She is best remembered for her

roles as Ilsa Lund in Casablanca (1942), a World War II drama co-starring Humphrey Bogart and as Alicia Huberman in Notorious (1946), an Alfred Hitchcock thriller co-starring Cary Grant.

In 1950, after a decade of stardom in American films, she starred in the Italian film Stromboli, which led to a love affair with director Roberto Rossellini while they were both already married. The

affair and then marriage with Rossellini created a scandal that forced her to remain in Europe until 1956, when she made a successful Hollywood return in Anastasia, for which she won her second Academy Award, as well as the forgiveness of her fans. Many of her personal and film documents can be seen in the Wesleyan University Cinema Archives.[4]

Roberto Gastone Zeffiro Rossellini (8 May 1906 – 3 June 1977)

was an Italian film director and screenwriter. Rossellini was one of the directors of the Italian neorealist cinema, contributing films such as Roma città aperta (Rome, Open City 1945) to the movement.

Rossellini was born in Rome. His mother, Elettra (née Bellan), was a housewife, and his father, Angiolo Giuseppe "Beppino" Rossellini, owned a construction firm. His mother was of part French descent, from immigrants who had arrived in Italy during the Napoleonic Wars. He lived on the Via Ludovisi, whereBenito Mussolini had his first Roman hotel in 1922 when Fascism obtained power in Italy.

QA, meet the (semantic) Web of Data

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 7

QA, meet the (semantic) Web of Data

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 8

QA, meet the (semantic) Web of Data

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 9

QA, meet the (semantic) Web of Data

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 10

Subject Predicate Object Rome isA city city subclassOf location Roberto_Rossellini mariedTo Ingrid_Bergman

WordNet

+

QA, meet the (semantic) Web of Data

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 11

Subject Predicate Object Rome isA city city subclassOf location Roberto_Rossellini mariedTo Ingrid_Bergman

WordNet

+

YAGO2: 120 million facts on 10 million entities

QA, meet the (semantic) Web of Data

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 12

Subject Predicate Object Rome isA city city subclassOf location Roberto_Rossellini mariedTo Ingrid_Bergman

WordNet

+

YAGO2: 120 million facts on 10 million entities

QA, meet the (semantic) Web of Data

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 13

Subject Predicate Object Rome isA city city subclassOf location Roberto_Rossellini mariedTo Ingrid_Bergman

WordNet

+

YAGO2: 120 million facts on 10 million entities

QA, meet the (semantic) Web of Data

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 14

Subject Predicate Object Rome isA city city subclassOf location Roberto_Rossellini mariedTo Ingrid_Bergman

WordNet

+

YAGO2: 120 million facts on 10 million entities

LOD: > 31 BILLION triples

Semantics Types

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 15

U.S. Cities

ITS LARGEST AIRPORT WAS NAMED FOR A WORLD

WAR II HERO; ITS SECOND LARGEST, FOR A WORLD

WAR II BATTLE

What is

Toronto???

Crash Course: Querying the Web of Data

“Who played in Casablanca and was married to a writer born in Rome?”

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 16

Crash Course: Querying the Web of Data

“Who played in Casablanca and was married to a writer born in Rome?”

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 17

SELECT ?p WHERE{ ?p type person . ?p actedIn Casablanca_(film). ?p isMarriedTo ?w . ?w type writer . ?w bornIn Rome }

SPARQL

Crash Course: Querying the Web of Data

“Who played in Casablanca and was married to a writer born in Rome?”

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 18

SELECT ?p WHERE{ ?p type person . ?p actedIn Casablanca_(film). ?p isMarriedTo ?w . ?w type writer . ?w bornIn Rome }

SPARQL

Projection Variable

Crash Course: Querying the Web of Data

“Who played in Casablanca and was married to a writer born in Rome?”

Dot: conjunction

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 19

SELECT ?p WHERE{ ?p type person . ?p actedIn Casablanca_(film). ?p isMarriedTo ?w . ?w type writer . ?w bornIn Rome }

SPARQL

Projection Variable

Crash Course: Querying the Web of Data

“Who played in Casablanca and was married to a writer born in Rome?”

Dot: conjunction ?p, ?w: variables to be bound

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 20

SELECT ?p WHERE{ ?p type person . ?p actedIn Casablanca_(film). ?p isMarriedTo ?w . ?w type writer . ?w bornIn Rome }

SPARQL

Projection Variable

Crash Course: Querying the Web of Data

“Who played in Casablanca and was married to a writer born in Rome?”

Dot: conjunction ?p, ?w: variables to be bound Same variable same binding (join)

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 21

SELECT ?p WHERE{ ?p type person . ?p actedIn Casablanca_(film). ?p isMarriedTo ?w . ?w type writer . ?w bornIn Rome }

SPARQL

Projection Variable

Crash Course: Querying the Web of Data

“Who played in Casablanca and was married to a writer born in Rome?”

Dot: conjunction ?p, ?w: variables to be bound Same variable same binding (join)

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 22

SELECT ?p WHERE{ ?p type person . ?p actedIn Casablanca_(film). ?p isMarriedTo ?w . ?w type writer . ?w bornIn Rome }

SPARQL

?p ?w

Projection Variable

What is DEANNA?

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al.

DEANNA

Question

SPARQL

KB

Answers 23

What is DEANNA?

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al.

“Who played in Casablanca and was married to a writer born in Rome?”

DEANNA

Question

SPARQL

KB

Answers 24

What is DEANNA?

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al.

?p type person. ?p actedIn Casablanca_(film). ?p isMarriedTo ?w. ?w type writer . ?w bornIn Rome .

“Who played in Casablanca and was married to a writer born in Rome?”

DEANNA

Question

SPARQL

KB

Answers 25

What is DEANNA?

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al.

?p type person. ?p actedIn Casablanca_(film). ?p isMarriedTo ?w. ?w type writer . ?w bornIn Rome .

“Who played in Casablanca and was married to a writer born in Rome?”

DEANNA

Question

SPARQL

KB

Answers 26

What is DEANNA?

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al.

?p type person. ?p actedIn Casablanca_(film). ?p isMarriedTo ?w. ?w type writer . ?w bornIn Rome .

“Who played in Casablanca and was married to a writer born in Rome?”

DEANNA

Question

SPARQL

KB

Answers 27

Inside DEANNA

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 28

DEANNA

Question

SPARQL

KB

Answers

Inside DEANNA

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 29

DEANNA

Question

SPARQL

KB

Answers

Phrase detection

Phrase mapping

Dependency detection

Joint Disambig.

Query Generation

Inside DEANNA

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 30

DEANNA

Question

SPARQL

KB

Answers

Phrase detection

Phrase mapping

Dependency detection

Joint Disambig.

Query Generation

Inside DEANNA

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 31

DEANNA

Question

SPARQL

KB

Answers

Phrase detection

Phrase mapping

Dependency detection

Joint Disambig.

Query Generation

Inside DEANNA

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 32

DEANNA

Question

SPARQL

KB

Answers

Phrase detection

Phrase mapping

Dependency detection

Joint Disambig.

Query Generation

Inside DEANNA

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 33

DEANNA

Question

SPARQL

KB

Answers

Phrase detection

Phrase mapping

Dependency detection

Joint Disambig.

Query Generation

Inside DEANNA

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 34

DEANNA

Question

SPARQL

KB

Answers

Phrase detection

Phrase mapping

Dependency detection

Joint Disambig.

Query Generation

Structured Query Generation

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 35

SELECT ?p WHERE { ?p type person. ?p actedIn Casablanca_(film). ?p isMarriedTo ?w. ?w type writer . ?w bornIn Rome }

q1

q2

q3

a writer

Casablanca

played in

Who

was married to

was born

Rome

c:writer

r:bornInPlace

e:Rome

e:Casablanca_(film)

r:actedIn

c:person

r:isMarriedTo

Outline

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 36

5 Experiments & Results

Disambiguation graph construction

Disambiguation graph processing

DEANNA

Question

SPARQL

KB

Phrase detection

Phrase mapping

Dependency detection

Joint Disambig.

Query Generation

1

2

3

4

Outline

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 37

5 Experiments & Results

DEANNA

Question

SPARQL

KB

Phrase detection

Phrase mapping

Dependency detection

Joint Disambig.

Query Generation

1

2

3

4

Phrase Detection

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 38

• Concepts: entities & classes: Dictionary-based

• Relations: Mainly use Reverb [Fader et al. EMNLP’11]: V | VP | VW*P … was/VBD married/VBN to/TO a/DT…

Concept Phrase

Casablanca Casablanca

Casablanca Casablanca, Morocco

Casablanca_(film) Casablanca the film

Casablanca_(film) Casablanca

… …

Phrase Detection

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 39

Casablanca

played

played in

Who

married

married to

was married to

a writer

• Concepts: entities & classes: Dictionary-based

• Relations: Mainly use Reverb [Fader et al. EMNLP’11]: V | VP | VW*P … was/VBD married/VBN to/TO a/DT…

Concept Phrase

Casablanca Casablanca

Casablanca Casablanca, Morocco

Casablanca_(film) Casablanca the film

Casablanca_(film) Casablanca

… …

Phrase Detection

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 40

Casablanca

played

played in

Who

married

married to

was married to

a writer

• Concepts: entities & classes: Dictionary-based

• Relations: Mainly use Reverb [Fader et al. EMNLP’11]: V | VP | VW*P … was/VBD married/VBN to/TO a/DT…

Concept Phrase

Casablanca Casablanca

Casablanca Casablanca, Morocco

Casablanca_(film) Casablanca the film

Casablanca_(film) Casablanca

… …

Outline

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 41

5 Experiments & Results

DEANNA

Question

SPARQL

KB

Phrase detection

Phrase mapping

Dependency detection

Joint Disambig.

Query Generation

1

2

3

4

Outline

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 42

5 Experiments & Results

DEANNA

Question

SPARQL

KB

Phrase detection

Phrase mapping

Dependency detection

Joint Disambig.

Query Generation

1

2

3

4

Phrase Mapping

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al.

Casablanca

played

played in

43

Phrase Mapping

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al.

Casablanca

played

played in

44

Concepts: entities & classes: Dictionary-based

Concept Phrase

Casablanca Casablanca

Casablanca Casablanca, Morocco

Casablanca_(film) Casablanca the film

Casablanca_(film) Casablanca

Played_(film) Played

Phrase Mapping

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al.

Casablanca

played

played in

e:White_House

e:Casablanca

e:Casablanca_(film)

e:Played_(film)

45

Concepts: entities & classes: Dictionary-based

Concept Phrase

Casablanca Casablanca

Casablanca Casablanca, Morocco

Casablanca_(film) Casablanca the film

Casablanca_(film) Casablanca

Played_(film) Played

Phrase Mapping

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al.

Casablanca

played

played in

e:White_House

e:Casablanca

e:Casablanca_(film)

e:Played_(film)

46

Concepts: entities & classes: Dictionary-based

• Relations: Dictionary -based

Concept Phrase

Casablanca Casablanca

Casablanca Casablanca, Morocco

Casablanca_(film) Casablanca the film

Casablanca_(film) Casablanca

Played_(film) Played

Relation Phrase

actedIn acted in

actedIn played in

hasMusicalRole plays

hasMusicalRole mastered

Phrase Mapping

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al.

Casablanca

played

played in

e:White_House

e:Casablanca

e:Casablanca_(film)

e:Played_(film)

r:actedIn

r:hasMusicalRole

47

Concepts: entities & classes: Dictionary-based

• Relations: Dictionary -based

Concept Phrase

Casablanca Casablanca

Casablanca Casablanca, Morocco

Casablanca_(film) Casablanca the film

Casablanca_(film) Casablanca

Played_(film) Played

Relation Phrase

actedIn acted in

actedIn played in

hasMusicalRole plays

hasMusicalRole mastered

Outline

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 48

5 Experiments & Results

DEANNA

Question

SPARQL

KB

Phrase detection

Phrase mapping

Dependency detection

Joint Disambig.

Query Generation

1

2

3

4

Outline

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 49

5 Experiments & Results

DEANNA

Question

SPARQL

KB

Phrase detection

Phrase mapping

Dependency detection

Joint Disambig.

Query Generation

1

2

3

4

Dependency Detection

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 50

Look for specific patterns in dependency parses [de Marneffe et al. LREC’06]

a writer

was born

born

Rome

c:writer

r:bornInPlace

r:bornOnDate

e:Max_Born

e:Born_(film)

e:Sydne_Rome

e:Rome

Dependency Detection

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 51

Look for specific patterns in dependency parses [de Marneffe et al. LREC’06]

a writer

was born

born

Rome

c:writer

r:bornInPlace

r:bornOnDate

e:Max_Born

e:Born_(film)

e:Sydne_Rome

e:Rome

Dependency Detection

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 52

Look for specific patterns in dependency parses [de Marneffe et al. LREC’06]

writer

in

born

Rome

partmod

prep

pobj

a writer

was born

born

Rome

c:writer

r:bornInPlace

r:bornOnDate

e:Max_Born

e:Born_(film)

e:Sydne_Rome

e:Rome

Dependency Detection

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 53

Look for specific patterns in dependency parses [de Marneffe et al. LREC’06]

writer

in

born

Rome

partmod

prep

pobj

q1

a writer

was born

born

Rome

c:writer

r:bornInPlace

r:bornOnDate

e:Max_Born

e:Born_(film)

e:Sydne_Rome

e:Rome

RESULT

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 54

Disambiguation Graph

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 55

q1

q2

q3

a writer

Casablanca

played

played in

Who

married

married to

was married to

was born

born

Rome

c:writer

r:bornInPlace

r:bornOnDate

e:Max_Born

e:Born_(film)

e:Sydne_Rome

e:Rome

e:White_House

e:Casablanca

e:Casablanca_(film)

e:Played_(film)

r:actedIn

r:hasMusicalRole

c:person

e:Married_(series)

c: married_person

r:isMarriedTo

q-nodes

Phrase-nodes Semantic nodes

Outline

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 56

5 Experiments & Results

DEANNA

Question

SPARQL

KB

Phrase detection

Phrase mapping

Dependency detection

Joint Disambig.

Query Generation

1

2

3

4

Outline

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 57

5 Experiments & Results

DEANNA

Question

SPARQL

KB

Phrase detection

Phrase mapping

Dependency detection

Joint Disambig.

Query Generation

1

2

3

4

Outline

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 58

5 Experiments & Results

DEANNA

Question

SPARQL

KB

Phrase detection

Phrase mapping

Dependency detection

Joint Disambig.

Query Generation

1

2

3

4

Joint Disambiguation - ILP

• ILP: Integer Linear Programming

• maximize α Σi,j wi,jYi,j + β Σk,l vk,l Zk,l + …

• Subject to:

No token in multiple phrases,

Triples observe type constraints, …

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 59

Joint Disambiguation – Objective

α Σi,j wi,jYi,j + β Σk,l vk,l Zk,l

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 60

Semantic nodes

q1

a writer

was born

born

Rome

c:writer

r:bornInPlace

r:bornOnDate

e:Max_Born

e:Born_(film)

e:Sydne_Rome

e:Rome

q-nodes

Phrase nodes

Coherence Edges

Similarity Edges

Prior

Joint Disambiguation – Objective

α Σi,j wi,jYi,j + β Σk,l vk,l Zk,l

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 61

Semantic nodes

Coherence

q1

a writer

was born

born

Rome

c:writer

r:bornInPlace

r:bornOnDate

e:Max_Born

e:Born_(film)

e:Sydne_Rome

e:Rome

q-nodes

Phrase nodes Similarity Edges

Coherence Edges

Joint Disambiguation – Constraints

A phrase node can be assigned to one semantic node:

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 62

Casablanca

e:White_House

e:Casablanca

e:Casablanca_(film)

Phrase nodes

Semantic nodes

a

1

2

3

Ya,1

Ya,2

Ya,3

α Σi,j wi,jYi,j + β Σk,l vk,l Zk,l

Joint Disambiguation – Constraints

A phrase node can be assigned to one semantic node:

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 63

Casablanca

e:White_House

e:Casablanca

e:Casablanca_(film)

Phrase nodes

Semantic nodes

a

1

2

3

Ya,1

Ya,2

Ya,3

α Σi,j wi,jYi,j + β Σk,l vk,l Zk,l

Joint Disambiguation – Constraints

Classes translate to type-constrained variables

Every semantic triple should have a class to join & project!

person actedIn Casablanca_(film)

▼

?x type person . ?x actedIn Casablanca_(film)

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 64

Joint Disambiguation – Constraints

Classes translate to type-constrained variables

Every semantic triple should have a class to join & project!

person actedIn Casablanca_(film)

▼

?x type person . ?x actedIn Casablanca_(film)

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 65

Joint Disambiguation – Constraints

Classes translate to type-constrained variables

Every semantic triple should have a class to join & project!

person actedIn Casablanca_(film)

▼

?x type person . ?x actedIn Casablanca_(film)

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 66

q1

a writer

was born

Rome

c:writer

r:bornInPlace

r:bornOnDate

e:Sydne_Rome

e:Rome

q-nodes

e:The_Writer (magazine)

Phrase nodes Semantic nodes

Joint Disambiguation – Constraints

Classes translate to type-constrained variables

Every semantic triple should have a class to join & project!

person actedIn Casablanca_(film)

▼

?x type person . ?x actedIn Casablanca_(film)

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 67

q1

a writer

was born

Rome

c:writer

r:bornInPlace

r:bornOnDate

e:Sydne_Rome

e:Rome

q-nodes

e:The_Writer (magazine)

Phrase nodes Semantic nodes

Joint Disambiguation – Constraints

Classes translate to type-constrained variables

Every semantic triple should have a class to join & project!

person actedIn Casablanca_(film)

▼

?x type person . ?x actedIn Casablanca_(film)

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 68

q1

a writer

was born

Rome

c:writer

r:bornInPlace

r:bornOnDate

e:Sydne_Rome

e:Rome

q-nodes

e:The_Writer (magazine)

Phrase nodes Semantic nodes

Outline

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 69

5 Experiments & Results

DEANNA

Question

SPARQL

KB

Phrase detection

Phrase mapping

Dependency detection

Joint Disambig.

Query Generation

1

2

3

4

Outline

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 70

5 Experiments & Results

DEANNA

Question

SPARQL

KB

Phrase detection

Phrase mapping

Dependency detection

Joint Disambig.

Query Generation

1

2

3

4

Structured Query Generation

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 71

SELECT ?p WHERE { ?w type writer . ?w bornIn Rome . ?p type person. ?p actedIn Casablanca_(film). ?p isMarriedTo ?w }

q1

q2

q3

a writer

Casablanca

played in

Who

was married to

was born

Rome

c:writer

r:bornIn

e:Rome

e:Casablanca_(film)

r:actedIn

c:person

r:isMarriedTo

Outline

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 72

5 Experiments & Results

DEANNA

Question

SPARQL

KB

Phrase detection

Phrase mapping

Dependency detection

Joint Disambig.

Query Generation

1

2

3

4

Outline

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 73

5 Experiments & Results

DEANNA

Question

SPARQL

KB

Phrase detection

Phrase mapping

Dependency detection

Joint Disambig.

Query Generation

1

2

3

4

Evaluation Methodology

• 3-stage evaluation for more insight:

1. Disambiguation

2. Query generation

3. Query answering

• We rely on human judges

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 74

Datasets

• QALD-1 YAGO2 27/50 questions within scope “Which software has been published by Mean

Hamster Software?”

• NAGA [Elbassuoni et al. CIKM’09] YAGO+IMDB 44/87 questions within scope “Which director has won the Academy Award for Best

director and is married to an actress that has won the Academy Award for Best Actress?”

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 75

Results 1/3: Disambiguation

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 76

Benchmark QALD-1 NAGA

covmacro 0.973 0.934

precmacro 1.000 0.934

covmicro 0.963 0.945

precmicro 1.000 0.941

cov = correct/ideal prec = correct / retrieved

Casablanca

e:White_House

e:Casablanca

e:Casablanca_(film)

Results 2/3: Query Generation

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 77

Benchmark QALD-1 NAGA

covmacro 0.975 0.894

precmacro 1.000 0.941

covmicro 0.963 0.847

precmicro 1.000 0.906

cov = #correct triples/#ideal triples prec = #correct triples/#retrieved triples

Issues: •Incorrect disambiguation •Incorrect dependencies.

Results 3/3: Answering

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 78

Benchmark QALD-1 NAGA

#questions 27 44

#queries 20 41

#satisfactory 10 15

#relaxed +3 +3

Results 3/3: Answering

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 79

Benchmark QALD-1 NAGA

#questions 27 44

#queries 20 41

#satisfactory 10 15

#relaxed +3 +3

unsatisfactory: main problem is empty result set

Results 3/3: Answering

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 80

Benchmark QALD-1 NAGA

#questions 27 44

#queries 20 41

#satisfactory 10 15

#relaxed +3 +3

unsatisfactory: main problem is empty result set

Relaxation: Keep type constraints, everything else to keywords ?x type writer . ?x bornIn Rome

?x type writer . ?x bornIn ?y [“Rome”]

Wrap-up

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al.

DEANNA

Question

SPARQL

KB

Answers 81

Wrap-up

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al.

“Who played in Casablanca and was married to a writer born in Rome?”

DEANNA

Question

SPARQL

KB

Answers 82

Wrap-up

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al.

“Who played in Casablanca and was married to a writer born in Rome?”

DEANNA

Question

SPARQL

KB

Answers 83

Wrap-up

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al.

?p type person. ?p actedIn Casablanca_(film). ?p isMarriedTo ?w. ?w type writer . ?w bornIn Rome

“Who played in Casablanca and was married to a writer born in Rome?”

DEANNA

Question

SPARQL

KB

Answers 84

Wrap-up

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al.

?p type person. ?p actedIn Casablanca_(film). ?p isMarriedTo ?w. ?w type writer . ?w bornIn Rome

“Who played in Casablanca and was married to a writer born in Rome?”

DEANNA

Question

SPARQL

KB

Answers 85

Rome isA city city subclassOf location Rossellini mariedTo Bergman

Wrap-up

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al.

?p type person. ?p actedIn Casablanca_(film). ?p isMarriedTo ?w. ?w type writer . ?w bornIn Rome

“Who played in Casablanca and was married to a writer born in Rome?”

DEANNA

Question

SPARQL

KB

Answers 86

Rome isA city city subclassOf location Rossellini mariedTo Bergman

QUESTIONS/COMMENTS?

Thank you.

EMNLP July 12, 2012

Natural Language Questions for the Web of Data - Yahya et al. 87

bit.ly/mpi-deanna