DiscourseandSummarization

Prof.SameerSinghCS295:STATISTICALNLP

WINTER2017

March16,2017

BasedonslidesfromDanJurafsky,JacobEisenstein,andeveryoneelsetheycopiedfrom.

Upcoming…

• Finalreportdueinaweek:March20,2017• Instructionsup:ACLstyle,5pages(+references)Project

2CS295:STATISTICALNLP(WINTER2017)

Outline

Discourse

3

Summarization

Wrapup

CS295:STATISTICALNLP(WINTER2017)

Outline

Discourse

4

Summarization

Wrapup

CS295:STATISTICALNLP(WINTER2017)

Discourse

5

Coreference

Coherence

Relations

Resolvingentities andevents.

Whatmakesthetextcoherent?

Rhetorical andnarrative linksbetweenunits

CS295:STATISTICALNLP(WINTER2017)

Discourse

6

Coreference

Coherence

Relations

Resolvingentities andevents.

Whatmakesthetextcoherent?

Rhetorical andnarrative linksbetweenunits

CS295:STATISTICALNLP(WINTER2017)

Coherence

7CS295:STATISTICALNLP(WINTER2017)

Coherence

8CS295:STATISTICALNLP(WINTER2017)

CoherencevsSemantics

9

Ameaninglesssentencecanbegrammatical..

Colorlessgreenideassleepfuriously

Thediscourseequivalentofgrammaticalityiscoherence

Canacoherenttextbewithoutmeaning?

CS295:STATISTICALNLP(WINTER2017)

ExampleEssay

10CS295:STATISTICALNLP(WINTER2017)

ExampleEssay

11

Thesecondreasonforthefive-paragraphthemeisthatitmakesyoufocusonasingletopic.Somepeoplestartwritingontheusualtopic,likeTVcommercials,andtheywindupallovertheplace,talkingaboutwhereTVcamefromorcapitalismorhealthfoodsorwhatever.Butwithonlyfiveparagraphsandonetopicyou’renottemptedtogetbeyondyouroriginalidea,likecommercialsareagoodsourceofinformationaboutproducts.Yougiveyourthreeexamples,andzap!you’redone.Thisisanotherwaythefive-paragraphthemekeepsyoufromthinkingtoomuch.

CS295:STATISTICALNLP(WINTER2017)

Detecting“Coherency”

12CS295:STATISTICALNLP(WINTER2017)

DiscourseConnectors

13CS295:STATISTICALNLP(WINTER2017)

LexicalChains

14CS295:STATISTICALNLP(WINTER2017)

DiscourseRelations

15

1.Intoday’ssociety,collegeisambiguous.2.Weneedittolive,3.butwealsoneedittolove.4.Moreover,withoutcollegemostoftheworld’slearningwouldbeegregious.5.College,however,hasmyriadcosts.6.Oneofthemostimportantissuesfacingtheworldishowtoreducecollegecosts.7.Somehavearguedthatcollegecostsareduetotheluxuriesstudentsnowexpect.8.Othershavearguedthatthecostsarearesultofathletics.9.Inreality,highcollegecostsaretheresultofexcessivepayforteachingassistants

CS295:STATISTICALNLP(WINTER2017)

DiscourseRelations

16

1.Intoday’ssociety,collegeisambiguous.2.Weneedittolive,3.butwealsoneedittolove.4.Moreover,withoutcollegemostoftheworld’slearningwouldbeegregious.5.College,however,hasmyriadcosts.6.Oneofthemostimportantissuesfacingtheworldishowtoreducecollegecosts.7.Somehavearguedthatcollegecostsareduetotheluxuriesstudentsnowexpect.8.Othershavearguedthatthecostsarearesultofathletics.9.Inreality,highcollegecostsaretheresultofexcessivepayforteachingassistants

CS295:STATISTICALNLP(WINTER2017)

DiscourseRelations

17

1.Intoday’ssociety,collegeisambiguous.2.Weneedittolive,3.butwealsoneedittolove.4.Moreover,withoutcollegemostoftheworld’slearningwouldbeegregious.5.College,however,hasmyriadcosts.6.Oneofthemostimportantissuesfacingtheworldishowtoreducecollegecosts.7.Somehavearguedthatcollegecostsareduetotheluxuriesstudentsnowexpect.8.Othershavearguedthatthecostsarearesultofathletics.9.Inreality,highcollegecostsaretheresultofexcessivepayforteachingassistants

CS295:STATISTICALNLP(WINTER2017)

DiscourseRelations

18

1.Intoday’ssociety,collegeisambiguous.2.Weneedittolive,3.butwealsoneedittolove.4.Moreover,withoutcollegemostoftheworld’slearningwouldbeegregious.5.College,however,hasmyriadcosts.6.Oneofthemostimportantissuesfacingtheworldishowtoreducecollegecosts.7.Somehavearguedthatcollegecostsareduetotheluxuriesstudentsnowexpect.8.Othershavearguedthatthecostsarearesultofathletics.9.Inreality,highcollegecostsaretheresultofexcessivepayforteachingassistants

CS295:STATISTICALNLP(WINTER2017)

DiscourseRelations

19

1.Intoday’ssociety,collegeisambiguous.2.Weneedittolive,3.butwealsoneedittolove.4.Moreover,withoutcollegemostoftheworld’slearningwouldbeegregious.5.College,however,hasmyriadcosts.6.Oneofthemostimportantissuesfacingtheworldishowtoreducecollegecosts.7.Somehavearguedthatcollegecostsareduetotheluxuriesstudentsnowexpect.8.Othershavearguedthatthecostsarearesultofathletics.9.Inreality,highcollegecostsaretheresultofexcessivepayforteachingassistants

CS295:STATISTICALNLP(WINTER2017)

DiscourseRelations

20

1.Intoday’ssociety,collegeisambiguous.2.Weneedittolive,3.butwealsoneedittolove.4.Moreover,withoutcollegemostoftheworld’slearningwouldbeegregious.5.College,however,hasmyriadcosts.6.Oneofthemostimportantissuesfacingtheworldishowtoreducecollegecosts.7.Somehavearguedthatcollegecostsareduetotheluxuriesstudentsnowexpect.8.Othershavearguedthatthecostsarearesultofathletics.9.Inreality,highcollegecostsaretheresultofexcessivepayforteachingassistants

CS295:STATISTICALNLP(WINTER2017)

CoherenceStructure

21

Segmentation

Zoning/Ordering

Centering/Salience

1.Intoday’ssociety,collegeisambiguous.2.Weneedittolive,3.butwealsoneedittolove.4.Moreover,withoutcollegemostoftheworld’slearningwouldbeegregious.5.College,however,hasmyriadcosts.6.Oneofthemostimportantissuesfacingtheworldishowtoreducecollegecosts.7.Somehavearguedthatcollegecostsareduetotheluxuriesstudentsnowexpect.8.Othershavearguedthatthecostsarearesultofathletics.9.Inreality,highcollegecostsaretheresultofexcessivepayforteachingassistants

CS295:STATISTICALNLP(WINTER2017)

CoherenceStructure

22

Segmentation`

Zoning/Ordering

Centering/Salience

1.Intoday’ssociety,collegeisambiguous.2.Weneedittolive,3.butwealsoneedittolove.4.Moreover,withoutcollegemostoftheworld’slearningwouldbeegregious.5.College,however,hasmyriadcosts.6.Oneofthemostimportantissuesfacingtheworldishowtoreducecollegecosts.7.Somehavearguedthatcollegecostsareduetotheluxuriesstudentsnowexpect.8.Othershavearguedthatthecostsarearesultofathletics.9.Inreality,highcollegecostsaretheresultofexcessivepayforteachingassistants

CS295:STATISTICALNLP(WINTER2017)

CoherenceStructure

23

Segmentation

Zoning/Ordering

Centering/Salience

1.Intoday’ssociety,collegeisambiguous.2.Weneedittolive,3.butwealsoneedittolove.4.Moreover,withoutcollegemostoftheworld’slearningwouldbeegregious.5.College,however,hasmyriadcosts.6.Oneofthemostimportantissuesfacingtheworldishowtoreducecollegecosts.7.Somehavearguedthatcollegecostsareduetotheluxuriesstudentsnowexpect.8.Othershavearguedthatthecostsarearesultofathletics.9.Inreality,highcollegecostsaretheresultofexcessivepayforteachingassistants

1.Intoday’ssociety,collegeisambiguous.2.Weneedittolive,3.butwealsoneedittolove.4.Moreover,withoutcollegemostoftheworld’slearningwouldbeegregious.5.College,however,hasmyriadcosts.6.Oneofthemostimportantissuesfacingtheworldishowtoreducecollegecosts.7.Somehavearguedthatcollegecostsareduetotheluxuriesstudentsnowexpect.8.Othershavearguedthatthecostsarearesultofathletics.9.Inreality,highcollegecostsaretheresultofexcessivepayforteachingassistants

CS295:STATISTICALNLP(WINTER2017)

CoherenceStructure

24

Segmentation

Zoning/Ordering

Centering/Salience

1.Intoday’ssociety,collegeisambiguous.2.Weneedittolive,3.butwealsoneedittolove.4.Moreover,withoutcollegemostoftheworld’slearningwouldbeegregious.5.College,however,hasmyriadcosts.6.Oneofthemostimportantissuesfacingtheworldishowtoreducecollegecosts.7.Somehavearguedthatcollegecostsareduetotheluxuriesstudentsnowexpect.8.Othershavearguedthatthecostsarearesultofathletics.9.Inreality,highcollegecostsaretheresultofexcessivepayforteachingassistants

1.Intoday’ssociety,collegeisambiguous.2.Weneedittolive,3.butwealsoneedittolove.4.Moreover,withoutcollegemostoftheworld’slearningwouldbeegregious.5.College,however,hasmyriadcosts.6.Oneofthemostimportantissuesfacingtheworldishowtoreducecollegecosts.7.Somehavearguedthatcollegecostsareduetotheluxuriesstudentsnowexpect.8.Othershavearguedthatthecostsarearesultofathletics.9.Inreality,highcollegecostsaretheresultofexcessivepayforteachingassistants

1.Intoday’ssociety,collegeisambiguous.2.Weneedittolive,3.butwealsoneedittolove.4.Moreover,withoutcollegemostoftheworld’slearningwouldbeegregious.5.College,however,hasmyriadcosts.6.Oneofthemostimportantissuesfacingtheworldishowtoreducecollegecosts.7.Somehavearguedthatcollegecostsareduetotheluxuriesstudentsnowexpect.8.Othershavearguedthatthecostsarearesultofathletics.9.Inreality,highcollegecostsaretheresultofexcessivepayforteachingassistants

CS295:STATISTICALNLP(WINTER2017)

ApplicationsofCoherence

25

SentenceOrdering

Whengeneratingsummaries,reordertillsentencesarecoherent.

ReadabilityAssessment Isapieceoftexteasilyreadable?

CS295:STATISTICALNLP(WINTER2017)

Discourse

26

Coreference

Coherence

Relations

Resolvingentities andevents.

Whatmakesthetextcoherent?

Rhetorical andnarrative linksbetweenunits

CS295:STATISTICALNLP(WINTER2017)

DiscourseRelations

27CS295:STATISTICALNLP(WINTER2017)

UseinSentimentAnalysis

28CS295:STATISTICALNLP(WINTER2017)

Outline

Discourse

29

Summarization

Wrapup

CS295:STATISTICALNLP(WINTER2017)

TextSummarizationGoal:produceanabridgedversionofatextthatcontainsinformationthatisimportantorrelevanttoauser.

SummarizationApplications◦ outlinesorabstractsofanydocument,article,etc◦ summariesofemailthreads◦ actionitemsfromameeting◦ simplifyingtextbycompressingsentences

30CS295:STATISTICALNLP(WINTER2017)

Whattosummarize?Single-documentsummarization◦ Givenasingledocument,produce◦ abstract◦ outline◦ headline

Multiple-documentsummarization◦ Givenagroupofdocuments,producea“gist”:◦ aseriesofnewsstoriesonthesameevent◦ asetofwebpagesaboutsometopicorquestion

31CS295:STATISTICALNLP(WINTER2017)

Query-focusedvsGenericGenericsummarization:◦ Summarizethecontentofadocument

Query-focusedsummarization:◦ summarizeadocumentwithrespecttoaninformationneedexpressedinauserquery.

◦ akindofcomplexquestionanswering:◦ Answeraquestionbysummarizingadocumentthathastheinformationtoconstructtheanswer

32CS295:STATISTICALNLP(WINTER2017)

Extractivesummarization&AbstractivesummarizationExtractivesummarization:◦ createthesummaryfromphrasesorsentencesinthesourcedocument(s)

Abstractivesummarization:◦ expresstheideasinthesourcedocumentsusing(atleastinpart)differentwords

33CS295:STATISTICALNLP(WINTER2017)

Summarization:ThreeStages1. contentselection:choosesentencestoextractfromthedocument

2. informationordering:chooseanordertoplacetheminthesummary

3. sentencerealization:cleanupthesentences

34

DocumentSentence

SegmentationSentenceExtraction

All sentencesfrom documents

Extracted sentences

Information Ordering

Sentence Realization

Summary

Content Selection

Sentence Simplification

CS295:STATISTICALNLP(WINTER2017)

Simplifyingsentences

appositives Rajam,28,anartistwhowaslivingatthetimeinPhiladelphia,foundtheinspirationinthebackofcitymagazines.

attribution clauses Rebelsagreedtotalkswithgovernmentofficials,internationalobserverssaidTuesday.

PPswithoutnamed entities

ThecommercialfishingrestrictionsinWashingtonwillnotbeliftedunlessthesalmonpopulationincreases[PPtoasustainablenumber]]

initialadverbials “Forexample”,“Ontheotherhand”,“Asamatteroffact”,“Atthispoint”

CS295:STATISTICALNLP(WINTER2017) 35Zajic etal.(2007),Conroyetal.(2006),Vanderwende etal.(2007)

Simplestmethod:parsesentences,userulestodecidewhichmodifierstoprune(morerecentlyawidevarietyofmachine-learningmethods)

ROUGE(RecallOrientedUnderstudyforGistingEvaluation)Intrinsicmetricforautomaticallyevaluatingsummaries◦ BasedonBLEU(ametricusedformachinetranslation)◦ Notasgoodashumanevaluation(“Didthisanswertheuser’squestion?”)◦ Butmuchmoreconvenient

GivenadocumentD,andanautomaticsummaryX:◦ HaveNhumansproduceasetofreferencesummariesofD◦ Runsystem,givingautomaticsummaryX◦ WhatpercentageofthebigramsfromthereferencesummariesappearinX?

36

ROUGE − 2 =min(count(i,X),count(i,S))

bigrams i∈S∑

s∈{RefSummaries}∑

count(i,S)bigrams i∈S∑

s∈{RefSummaries}∑

CS295:STATISTICALNLP(WINTER2017)

Outline

Discourse

37

Summarization

Wrapup

CS295:STATISTICALNLP(WINTER2017)

WordoutofContext

38CS295:STATISTICALNLP(WINTER2017)

WordsinContext

39CS295:STATISTICALNLP(WINTER2017)

Sentences

40CS295:STATISTICALNLP(WINTER2017)

InformationExtraction

CS295:STATISTICALNLP(WINTER2017) 41

MachineTranslation

42CS295:STATISTICALNLP(WINTER2017)

Other“Applications”

43CS295:STATISTICALNLP(WINTER2017)

DocumentDocumentDocument

DocumentDocume

ntDocumentDocume

ntDocument

Question Processing

PassageRetrieval

Query Formulation

Answer Type Detection

Question

Passage Retrieval

Document Retrieval

Answer Processing

Answer

passages

Indexing

RelevantDocs

DocumentDocumentDocument

DocumentSentence

SegmentationSentenceExtraction

All sentencesfrom documents

Extracted sentences

Information Ordering

Sentence Realization

Summary

Content Selection

Sentence Simplification

Wrapup oftheCourse

CS295:STATISTICALNLP(WINTER2017) 44

AndNow!

CS295:STATISTICALNLP(WINTER2017) 45

DoresearchinNLP!

46CS295:STATISTICALNLP(WINTER2017)