Never Ending Language Learning Carnegie Mellon University.

46
Never Ending Language Learning Carnegie Mellon University

Transcript of Never Ending Language Learning Carnegie Mellon University.

Page 1: Never Ending Language Learning Carnegie Mellon University.

Never Ending Language Learning

Carnegie Mellon University

Page 2: Never Ending Language Learning Carnegie Mellon University.

Tenet 1:

Understanding requires a belief system

We’ll never produce natural language understanding systems until we have systems that react to arbitrary sentences by saying one of:• I understand, and already knew that• I understand, and didn’t know, but accept it• I understand, and disagree because …

Page 3: Never Ending Language Learning Carnegie Mellon University.

Tenet 2:

We’ll never really understand learning until we build machines that • learn many different things, • over years, • and become better learners over time.

Page 4: Never Ending Language Learning Carnegie Mellon University.

NELL: Never-Ending Language LearnerInputs:• initial ontology • few examples of each ontology predicate• the web• occasional interaction with human trainers

The task:• run 24x7, forever• each day:

1. extract more facts from the web to populate the initial ontology

2. learn to read (perform #1) better than yesterday

Page 5: Never Ending Language Learning Carnegie Mellon University.

NELL today

Running 24x7, since January, 12, 2010

Inputs:• ontology defining >600 categories and relations• 10-20 seed examples of each• 500 million web pages • 100,000 web search queries per day • ~ 5 minutes/day of human guidance

Result:• KB with > 15 million candidate beliefs, growing daily• learning to reason, as well as read• automatically extending its ontology

Page 6: Never Ending Language Learning Carnegie Mellon University.

Globe and Mail

StanleyCup

hockey

NHL

Toronto

CFRB

Wilson

playhired

wonMaple Leafs

home town

city paper

league

Sundin

Milson

writer

radio

Maple Leaf Gardens

team stadium

Canada

city stadium

politician

country

Miller

airport

member

Toskala

Pearson

Skydome

Connaught

Sunnybrook

hospital

city company

skates helmet

uses equipment

wonRed

Wings

Detroit

hometown

GM

city company

competeswith

Toyota

plays in

league

Prius

Corrola

createdHino

acquired

automobile

economicsector

city stadium

NELL knowledge fragmentclimbing

football

uses equipment

Page 8: Never Ending Language Learning Carnegie Mellon University.

Semi-Supervised Bootstrap Learning

ParisPittsburghSeattleCupertino

mayor of arg1live in arg1

San FranciscoAustindenial

arg1 is home oftraits such as arg1

it’s underconstrained!!

anxietyselfishnessBerlin

Extract cities:

Page 9: Never Ending Language Learning Carnegie Mellon University.

hard (underconstrained)

semi-supervised learning problem

Key Idea 1: Coupled semi-supervised training of many functions

much easier (more constrained)semi-supervised learning problem

person

NP

Page 10: Never Ending Language Learning Carnegie Mellon University.

NP:

person

NP text context

distribution

__ is a friendrang the __

…__ walked in

f1(NP)

NP morphology

capitalized?ends with ‘...ski’?

…contains “univ.”?

f2(NP)

NP HTML contexts

www.celebrities.com:<li> __ </li>

f3(NP)

Type 1 Coupling: Co-Training, Multi-View Learning[Blum & Mitchell; 98][Dasgupta et al; 01 ][Ganchev et al., 08][Sridharan & Kakade, 08][Wang & Zhou, ICML10]

Page 11: Never Ending Language Learning Carnegie Mellon University.

team

personathlete

coachsport

NP

athlete(NP) person(NP)

athlete(NP) NOT sport(NP)

NOT athlete(NP) sport(NP)

Type 2 Coupling: Multi-task, Structured Outputs[Daume, 2008][Bakhir et al., eds. 2007][Roth et al., 2008][Taskar et al., 2009][Carlson et al., 2009]

Page 12: Never Ending Language Learning Carnegie Mellon University.

team

person

NP:

athletecoach

sport

NP text context

distribution

NP morphology

NP HTML contexts

Multi-view, Multi-Task Coupling

Page 13: Never Ending Language Learning Carnegie Mellon University.

coachesTeam(c,t)playsForTeam(a,t) teamPlaysSport(t,s)

playsSport(a,s)

NP1 NP2

Learning Relations between NP’s

Page 14: Never Ending Language Learning Carnegie Mellon University.

team

coachesTeam(c,t)playsForTeam(a,t) teamPlaysSport(t,s)

playsSport(a,s)

person

NP1

athlete

coach

sport

team

person

NP2

athlete

coach

sport

Page 15: Never Ending Language Learning Carnegie Mellon University.

team

coachesTeam(c,t)playsForTeam(a,t) teamPlaysSport(t,s)

playsSport(a,s)

person

NP1

athlete

coach

sport

team

person

NP2

athlete

coach

sport

playsSport(NP1,NP2) athlete(NP1), sport(NP2)

Type 3 Coupling: Argument Types

over 2500 coupled functions in NELL

Page 16: Never Ending Language Learning Carnegie Mellon University.

Continually Learning Extractors

Basic NELL ArchitectureKnowledge Base(latent variables)

Text Context patterns(CPL)

HTML-URLcontext patterns(SEAL)

Morphologyclassifier

(CML)

Beliefs

CandidateBeliefs

Evidence Integrator

Page 17: Never Ending Language Learning Carnegie Mellon University.

NELL: Learned reading strategiesPlays_Sport(arg1,arg2):

arg1_was_playing_arg2 arg2_megastar_arg1 arg2_icons_arg1 arg2_player_named_arg1 arg2_prodigy_arg1 arg1_is_the_tiger_woods_of_arg2 arg2_career_of_arg1 arg2_greats_as_arg1 arg1_plays_arg2 arg2_player_is_arg1 arg2_legends_arg1 arg1_announced_his_retirement_from_arg2 arg2_operations_chief_arg1 arg2_player_like_arg1 arg2_and_golfing_personalities_including_arg1 arg2_players_like_arg1 arg2_greats_like_arg1 arg2_players_are_steffi_graf_and_arg1 arg2_great_arg1 arg2_champ_arg1 arg2_greats_such_as_arg1 arg2_professionals_such_as_arg1 arg2_hit_by_arg1 arg2_greats_arg1 arg2_icon_arg1 arg2_stars_like_arg1 arg2_pros_like_arg1 arg1_retires_from_arg2 arg2_phenom_arg1 arg2_lesson_from_arg1 arg2_architects_robert_trent_jones_and_arg1 arg2_sensation_arg1 arg2_pros_arg1 arg2_stars_venus_and_arg1 arg2_hall_of_famer_arg1 arg2_superstar_arg1 arg2_legend_arg1 arg2_legends_such_as_arg1 arg2_players_is_arg1 arg2_pro_arg1 arg2_player_was_arg1 arg2_god_arg1 arg2_idol_arg1 arg1_was_born_to_play_arg2 arg2_star_arg1 arg2_hero_arg1 arg2_players_are_arg1 arg1_retired_from_professional_arg2 arg2_legends_as_arg1 arg2_autographed_by_arg1 arg2_champion_arg1 …

Page 18: Never Ending Language Learning Carnegie Mellon University.

If coupled learning is the key,how can we get new coupling constraints?

Page 19: Never Ending Language Learning Carnegie Mellon University.

Key Idea 2:

Discover New Coupling Constraints

• first order, probabilistic horn clause constraints:

– connects previously uncoupled relation predicates

– infers new beliefs for KB

0.93 athletePlaysSport(?x,?y) athletePlaysForTeam(?x,?z) teamPlaysSport(?z,?y)

Page 20: Never Ending Language Learning Carnegie Mellon University.

Example Learned Horn Clauses

athletePlaysSport(?x,basketball) athleteInLeague(?x,NBA)

athletePlaysSport(?x,?y) athletePlaysForTeam(?x,?z)

teamPlaysSport(?z,?y)

teamPlaysInLeague(?x,NHL) teamWonTrophy(?x,Stanley_Cup)

athleteInLeague(?x,?y) athletePlaysForTeam(?x,?z),

teamPlaysInLeague(?z,?y)

cityInState(?x,?y) cityCapitalOfState(?x,?y), cityInCountry(?y,USA)

newspaperInCity(?x,New_York) companyEconomicSector(?x,media)

generalizations(?x,blog)

0.95

0.93

0.91

0.90

0.88

0.62*

Page 21: Never Ending Language Learning Carnegie Mellon University.

Some rejected learned rules

teamPlaysInLeague{?x nba} teamPlaysSport{?x basketball}

0.94 [ 35 0 35 ] [positive negative unlabeled]

cityCapitalOfState{?x ?y} cityLocatedInState{?x ?y}, teamPlaysInLeague{?y nba}

0.80 [ 16 2 23 ]

teamplayssport{?x, basketball} generalizations{?x, university}

0.61 [ 246 124 3063 ]

Page 22: Never Ending Language Learning Carnegie Mellon University.

team

coachesTeam(c,t)playsForTeam(a,t) teamPlaysSport(t,s)

playsSport(a,s)

person

NP1

athlete

coach

sport

team

person

NP2

athlete

coach

sport

Learned Probabilistic Horn Clause Rules

0.93 playsSport(?x,?y) playsForTeam(?x,?z), teamPlaysSport(?z,?y)

Page 23: Never Ending Language Learning Carnegie Mellon University.

Key Idea 3: Automatically extend ontology

Page 24: Never Ending Language Learning Carnegie Mellon University.

Ontology Extension (1)

Goal:• Add new relations to ontology

Approach:• For each pair of categories C1, C2,

• co-cluster pairs of known instances, and text contexts that connect them

[Mohamed et al., EMNLP 2011]

Page 25: Never Ending Language Learning Carnegie Mellon University.

Example Discovered Relations

Category Pair Text contexts Extracted Instances Suggested Name

MusicInstrumentMusician

ARG1 master ARG2ARG1 virtuoso ARG2ARG1 legend ARG2ARG2 plays ARG1

sitar , George Harrisontenor sax, Stan Getz

trombone, Tommy Dorseyvibes, Lionel Hampton

Master

DiseaseDisease

ARG1 is due to ARG2ARG1 is caused by ARG2

pinched nerve, herniated disktennis elbow, tendonitis

blepharospasm, dystoniaIsDueTo

CellTypeChemical ARG1 that release ARG2

ARG2 releasing ARG1

epithelial cells, surfactantneurons, serotonin

mast cells, histomineThatRelease

MammalsPlant

ARG1 eat ARG2ARG2 eating ARG1

koala bears, eucalyptussheep, grassesgoats, saplings

Eat

RiverCity

ARG1 in heart of ARG2ARG1 which flows through

ARG2

Seine, ParisNile, Cairo

Tiber river, RomeInHeartOf

[Mohamed et al. EMNLP 2011]

Page 26: Never Ending Language Learning Carnegie Mellon University.

NELL: recently self-added relations

• athleteWonAward• animalEatsFood• languageTaughtInCity• clothingMadeFromPlant• beverageServedWithFood• fishServedWithFood• athleteBeatAthlete• athleteInjuredBodyPart• arthropodFeedsOnInsect• animalEatsVegetable• plantRepresentsEmotion• foodDecreasesRiskOfDisease

• clothingGoesWithClothing• bacteriaCausesPhysCondition• buildingMadeOfMaterial• emotionAssociatedWithDisease• foodCanCauseDisease• agriculturalProductAttractsInsect• arteryArisesFromArtery• countryHasSportsFans• bakedGoodServedWithBeverage• beverageContainsProtein• animalCanDevelopDisease• beverageMadeFromBeverage

Page 27: Never Ending Language Learning Carnegie Mellon University.

Key Idea 4: Cumulative, Staged Learning

1. Classify noun phrases (NP’s) by category

2. Classify NP pairs by relation

3. Discover rules to predict new relation instances

4. Learn which NP’s (co)refer to which concepts

5. Discover new relations to extend ontology

6. Learn to infer relation instances via targeted random walks

7. Learn to assign temporal scope to beliefs

8. Learn to microread single sentences

9. Vision: co-train text and visual object recognition

10. Goal-driven reading: predict, then read to corroborate/correct

11. Make NELL a conversational agent on Twitter

12. Add a robot body to NELL

Learning X improves ability to learn Y

Page 28: Never Ending Language Learning Carnegie Mellon University.

Globe and Mail

StanleyCup

hockey

NHL

Toronto

CFRB

Wilson

playhired

wonMaple Leafs

home town

city paper

league

Sundin

Milson

writer

radio

Maple Leaf Gardens

team stadium

Canada

city stadium

politician

country

Miller

airport

member

Toskala

Pearson

Skydome

Connaught

Sunnybrook

hospital

city company

skates helmet

uses equipment

wonRed

Wings

Detroit

hometown

GM

city company

competeswith

Toyota

plays in

league

Prius

Corrola

createdHino

acquired

automobile

economicsector

city stadium

Learning to Reason at Scaleclimbing

football

uses equipment

Page 29: Never Ending Language Learning Carnegie Mellon University.

Inference by KB Random Walks [Lao et al, EMNLP 2011]

If: x1 competeswith

(x1,x2)

x2 economic sector (x2, x3)

x3

Then: economic sector (x1, x3)

Page 30: Never Ending Language Learning Carnegie Mellon University.

Inference by KB Random Walks [Lao et al, EMNLP 2011]

KB:

Random walkpath type:

Trained logistic function for R, where ith feature is probability of arriving at node y when starting at node x, and taking a random walk along path type i

Infer Pr(R(x,y)):

? competeswith

? economic sector

?

Page 31: Never Ending Language Learning Carnegie Mellon University.

Feature = Typed Path CityInState, CityInstate-1, CityLocatedInCountry 0.8 0.32

Pittsburgh

Feature Value

Logistic Regresssion

Weight

CityLocatedInCountry(Pittsburgh) = ? [Lao et al, EMNLP 2011]

Page 32: Never Ending Language Learning Carnegie Mellon University.

Feature = Typed Path CityInState, CityInstate-1, CityLocatedInCountry 0.8 0.32

Pittsburgh

Pennsylvania

CityIn

State

Feature Value

Logistic Regresssion

Weight

CityLocatedInCountry(Pittsburgh) = ? [Lao et al, EMNLP 2011]

Page 33: Never Ending Language Learning Carnegie Mellon University.

Feature = Typed Path CityInState, CityInstate-1, CityLocatedInCountry 0.8 0.32

Pittsburgh

Pennsylvania

CityIn

State

CityInState-1

CityInS

tate-1

PhiladelphiaHarisburg

…(14)

Feature Value

Logistic Regresssion

Weight

CityLocatedInCountry(Pittsburgh) = ? [Lao et al, EMNLP 2011]

Page 34: Never Ending Language Learning Carnegie Mellon University.

Feature = Typed Path CityInState, CityInstate-1, CityLocatedInCountry 0.8 0.32

Pittsburgh

Pennsylvania

CityIn

State

CityInState-1

CityInS

tate-1

PhiladelphiaHarisburg

…(14)

U.S.

Feature Value

Logistic Regresssion

Weight

CityLocatedInCountry

CityLocatedInCountry(Pittsburgh) = ? [Lao et al, EMNLP 2011]

Page 35: Never Ending Language Learning Carnegie Mellon University.

Feature = Typed Path CityInState, CityInstate-1, CityLocatedInCountry 0.8 0.32

Pittsburgh

Pennsylvania

CityIn

State

CityInState-1

CityInS

tate-1

PhiladelphiaHarisburg

…(14)

U.S.

Feature Value

Logistic Regresssion

Weight

CityLocatedInCountry

CityLocatedInCountry(Pittsburgh) = ?

Pr(U.S. | Pittsburgh, TypedPath)

[Lao et al, EMNLP 2011]

Page 36: Never Ending Language Learning Carnegie Mellon University.

Feature = Typed Path CityInState, CityInstate-1, CityLocatedInCountry 0.8 0.32 AtLocation-1, AtLocation, CityLocatedInCountry 0.20

Pittsburgh

Pennsylvania

CityIn

State

CityInState-1

CityInS

tate-1

PhiladelphiaHarisburg

…(14)

U.S.

Feature Value

Logistic Regresssion

Weight

CityLocatedInCountry

CityLocatedInCountry(Pittsburgh) = ? [Lao et al, EMNLP 2011]

Page 37: Never Ending Language Learning Carnegie Mellon University.

Feature = Typed Path CityInState, CityInstate-1, CityLocatedInCountry 0.8 0.32 AtLocation-1, AtLocation, CityLocatedInCountry 0.20

Pittsburgh

Pennsylvania

CityIn

State

CityInState-1

CityInS

tate-1

PhiladelphiaHarisburg

…(14)

U.S.

Feature Value

Logistic Regresssion

Weight

CityLocatedInCountry

DeltaPPG

AtLocation -1

CityLocatedInCountry(Pittsburgh) = ? [Lao et al, EMNLP 2011]

Page 38: Never Ending Language Learning Carnegie Mellon University.

Feature = Typed Path CityInState, CityInstate-1, CityLocatedInCountry 0.8 0.32 AtLocation-1, AtLocation, CityLocatedInCountry 0.20

Pittsburgh

Pennsylvania

CityIn

State

CityInState-1

CityInS

tate-1

PhiladelphiaHarisburg

…(14)

U.S.

Feature Value

Logistic Regresssion

Weight

CityLocatedInCountry

DeltaPPG

AtLocation -1 AtLocationAtlanta

DallasTokyo

CityLocatedInCountry(Pittsburgh) = ? [Lao et al, EMNLP 2011]

Page 39: Never Ending Language Learning Carnegie Mellon University.

Feature = Typed Path CityInState, CityInstate-1, CityLocatedInCountry 0.8 0.32 AtLocation-1, AtLocation, CityLocatedInCountry 0.6 0.20

Pittsburgh

Pennsylvania

CityIn

State

CityInState-1

CityInS

tate-1

PhiladelphiaHarisburg

…(14)

U.S.

Feature Value

Logistic Regresssion

Weight

CityLocatedInCountry

DeltaPPG

AtLocation -1 AtLocationAtlanta

DallasTokyo

Japan

CityLocatedInCountry(Pittsburgh) = ?

City

Loca

tedI

nCou

ntry

[Lao et al, EMNLP 2011]

Page 40: Never Ending Language Learning Carnegie Mellon University.

Feature = Typed Path CityInState, CityInstate-1, CityLocatedInCountry 0.8 0.32 AtLocation-1, AtLocation, CityLocatedInCountry 0.6 0.20 … … …

Pittsburgh

Pennsylvania

CityIn

State

CityInState-1

CityInS

tate-1

PhiladelphiaHarisburg

…(14)

U.S.

Feature Value

Logistic Regresssion

Weight

CityLocatedInCountry(Pittsburgh) = U.S. p=0.58

CityLocatedInCountry

DeltaPPG

AtLocation -1 AtLocationAtlanta

DallasTokyo

Japan

CityLocatedInCountry(Pittsburgh) = ?

City

Loca

tedI

nCou

ntry

[Lao et al, EMNLP 2011]

Page 41: Never Ending Language Learning Carnegie Mellon University.

Random walk inference: learned path types

CityLocatedInCountry(city, country):

8.04 cityliesonriver, cityliesonriver-1, citylocatedincountry

5.42 hasofficeincity-1, hasofficeincity, citylocatedincountry

4.98 cityalsoknownas, cityalsoknownas, citylocatedincountry

2.85 citycapitalofcountry,citylocatedincountry-1,citylocatedincountry

2.29 agentactsinlocation-1, agentactsinlocation, citylocatedincountry

1.22 statehascapital-1, statelocatedincountry

0.66 citycapitalofcountry . . . 7 of the 2985 paths for inferring CityLocatedInCountry

Page 42: Never Ending Language Learning Carnegie Mellon University.

Random Walk Inference: ExampleRank 17 companies by probability competesWith(MSFT,X):

NELL/PRA ranking

GoogleOracleIBMAppleSAPYahoo

FacebookRedhatLenovoFedExSAS

BoeingHondaDupontLufthansaExxonPfizer

Page 43: Never Ending Language Learning Carnegie Mellon University.

Random Walk Inference: ExampleRank 17 companies by probability competesWith(MSFT,X):

NELL/PRA ranking Human Ranking (9 subjs)

AppleGoogleYahooIBMRedhatOracle

FacebookSAPSAS

LenovoBoeingHondaFedExDupontExxon

LufthansaPfizer

GoogleOracleIBMAppleSAPYahoo

FacebookRedhatLenovoFedExSAS

BoeingHondaDupontLufthansaExxonPfizer

1. Tractable (bounded length)

2. Anytime

3. Accuracy increases as KB grows

4. Addresses question of how to combine probabilities from different horn clauses

demo

Page 44: Never Ending Language Learning Carnegie Mellon University.

Random walk inference: learned path types

CompetesWith(company, company):

5.29 companyAlsoKnownAs, competesWith

2.12 companyAlsoKnownAs, producesProduct, agentInvolvedWith-1

0.77 companyAlsoKnownAs, subj_offer_obj, subj_offer_obj-1

0.65 companyEconomicSector, companyEconomicSector-1

- 0.19 companyAlsoKnownAs

- 0.38 companyAlsoKnownAs, companyAlsoKnownAs

. . .

KB graph augmented by Subj-Verb-Obj

corpus statistics

6 of the 7966 path types learned for CompetesWith

Page 45: Never Ending Language Learning Carnegie Mellon University.

Summary

Key ideas:• Coupled semi-supervised learning• Learn new coupling constraints (Horn clauses)• Automatically extend ontology• Learn progressively more challenging types of K• Scalable random walk probabilistic inference

– Integrating symbolic extracted beliefs,

+ subsymbolic corpus statistics

Page 46: Never Ending Language Learning Carnegie Mellon University.

thank you

and thanks to: Darpa, Google, NSF, Intel, Yahoo!, Microsoft, Fullbright