POStaggingCMSC723/LING723/INST725

MarineCarpuat

PartsofSpeech

• “Equivalenceclass”oflinguisticentities• “Categories”or“types”ofwords

• StudydatesbacktotheancientGreeks• DionysiusThrax ofAlexandria(c. 100BC)• 8partsofspeech:noun,verb,pronoun,preposition,adverb,conjunction,participle,article

• Remarkablyenduringlist!

2

HowcanwedefinePOS?

• Bymeaning?• Verbsareactions• Adjectivesareproperties• Nounsarethings

• Bythesyntacticenvironment• Whatoccursnearby?• Whatdoesitactas?

• Bywhatmorphologicalprocessesaffectit• Whataffixesdoesittake?

• Typicallycombinationofsyntactic+morphology

PartsofSpeech

• Openclass• Impossibletocompletelyenumerate• Newwordscontinuouslybeinginvented,borrowed,etc.

• Closedclass• Closed,fixedmembership• Reasonablyeasytoenumerate• Generally,shortfunctionwordsthat“structure”sentences

OpenClassPOS

• FourmajoropenclassesinEnglish• Nouns• Verbs• Adjectives• Adverbs

• Alllanguageshavenounsandverbs...butmaynothavetheothertwo

Nouns

• Openclass• Newinventionsallthetime:muggle,webinar,...

• Semantics:• Generally,wordsforpeople,places,things• Butnotalways(bandwidth,energy,...)

• Syntacticenvironment:• Occurringwithdeterminers• Pluralizable,possessivizable

• Othercharacteristics:• Massvs.countnouns

Verbs

• Openclass• Newinventionsallthetime:google,tweet,...

• Semantics• Generally,denoteactions,processes,etc.

• Syntacticenvironment• E.g.,Intransitive,transitive

• Othercharacteristics• Mainvs.auxiliaryverbs• Gerunds(verbsbehavinglikenouns)• Participles(verbsbehavinglikeadjectives)

AdjectivesandAdverbs

• Adjectives• Generallymodifynouns,e.g.,tall girl

• Adverbs• Asemanticandformalhodge-podge…• Sometimesmodifyverbs,e.g.,sangbeautifully• Sometimesmodifyadjectives,e.g.,extremely hot

ClosedClassPOS

• Prepositions• InEnglish,occurringbeforenounphrases• Specifyingsometypeofrelation(spatial,temporal,…)• Examples:on theshelf,before noon

• Particles• Resemblesapreposition,butusedwithaverb(“phrasalverbs”)• Examples:findout,turnover,goon

Particlevs.Prepositions

Hecameby theofficeinahurryHecameby hisfortunehonestly

Weranup thephonebillWeranup thesmallhill

Heliveddown theblockHeneverliveddown thenicknames

(by=preposition)(by=particle)

(up=particle)(up=preposition)

(down=preposition)(down=particle)

MoreClosedClassPOS

• Determiners• Establishreferenceforanoun• Examples:a,an,the (articles),that,this,many,such,…

• Pronouns• Refertopersonorentities:he,she,it• Possessivepronouns:his,her,its• Wh-pronouns:what,who

ClosedClassPOS:Conjunctions

• Coordinatingconjunctions• Jointwoelementsof“equalstatus”• Examples:catsand dogs,salador soup

• Subordinatingconjunctions• Jointwoelementsof“unequalstatus”• Examples:We’llleaveafter youfinisheating.While Iwaswaitinginline,Isawmyfriend.

• Complementizers areaspecialcase:Ithinkthat youshouldfinishyourassignment

BeyondEnglish…ChineseNoverb/adjectivedistinction!

RiauIndonesian/MalayNoArticlesNoTenseMarking3rdpersonpronounsneutraltobothgenderandnumberNofeaturesdistinguishingverbsfromnouns

漂亮:beautiful/tobebeautiful

Ayam (chicken) Makan (eat)

The chicken is eatingThe chicken ate

The chicken will eatThe chicken is being eatenWhere the chicken is eatingHow the chicken is eating

Somebody is eating the chickenThe chicken that is eating

POStagging

POSTagging:What’sthetask?

• Processofassigningpart-of-speechtagstowords

• Butwhattagsarewegoingtoassign?• Coarsegrained:noun,verb,adjective,adverb,…• Finegrained:{proper,common}noun• Evenfiner-grained:{proper,common}noun± animate

• Importantissuestoremember• Choiceoftagsencodescertaindistinctions/non-distinctions• Tagsets willdifferacrosslanguages!

• ForEnglish,PennTreebankisthemostcommontagset

PennTreebankTagset:45Tags

PennTreebankTagset:Choices

• Example:• The/DTgrand/JJjury/NNcommmented/VBDon/INa/DTnumber/NNof/INother/JJtopics/NNS./.

• Distinctionsandnon-distinctions• Prepositionsandsubordinatingconjunctionsaretagged“IN”(“Although/INI/PRP..”)

• Exceptthepreposition/complementizer “to”istagged“TO”

WhydoPOStagging?

• OneofthemostbasicNLPtasks• NicelyillustratesprinciplesofstatisticalNLP

• Usefulforhigher-levelanalysis• Neededforsyntacticanalysis• Neededforsemanticanalysis

• SampleapplicationsthatrequirePOStagging• Machinetranslation• Informationextraction• Lotsmore…

Tryyourhandattagging…

• Theback door• Onmyback• Winthevotersback• Promisedtoback thebill

Tryyourhandattagging…

• Ihopethat shewins• That daywasnice• Youcangothat far

WhyisPOStagginghard?

• Ambiguity!

• AmbiguityinEnglish• 11.5%ofwordtypesambiguousinBrowncorpus• 40%ofwordtokensambiguousinBrowncorpus• AnnotatordisagreementinPennTreebank:3.5%

POStagging:howtodoit?

• GivenPennTreebank,howwouldyoubuildasystemthatcanPOStagnewtext?

• Baseline:pickmostfrequenttagforeachwordtype• 90%accuracyiftrain+test setsaredrawnfromPennTreebank

• Canwedobetter?

HowtoPOStagautomatically?

HowcanwePOStagautomatically?

• POStaggingasmulticlassclassification• Whatisx?Whatisy?

• POStaggingassequencelabeling• Modelssequencesofpredictions

LinearModelsforClassification

Featurefunction

representation

Weights

Multiclassperceptron

POStaggingSequencelabelingwiththeperceptron

Sequencelabelingproblem

• Input:• sequenceoftokensx=[x1 … xK]• VariablelengthK

• Output(akalabel):• sequenceoftagsy=[y1 … yK]• Sizeofoutputspace?

StructuredPerceptron• Perceptronalgorithmcanbeusedforsequencelabeling

• Buttherearechallenges• Howtocomputeargmax efficiently?• Whatareappropriatefeatures?

• Approach:leveragestructureofoutputspace

Featurefunctionsforsequencelabeling

• Examplefeatures?• Numberoftimes“monsters”istaggedasnoun

• Numberoftimes“noun”isfollowedby“verb”

• Numberoftimes“tasty”istaggedas“verb”

• Numberoftimestwoverbsareadjacent• …

Featurefunctionsforsequencelabeling

• StandardfeaturesofPOStagging

• Unaryfeatures: #timeswordwhasbeenlabeledwithtaglforallwordswandalltagsl

• Markovfeatures: #timestaglisadjacenttotagl’inoutputforalltagslandl’

• Sizeoffeaturerepresentationisconstantwrtinputlength

Solvingtheargmax problemforsequences

• Efficientalgorithmspossibleifthefeaturefunctiondecomposesovertheinput

• Thisholdsforunaryandmarkovfeatures

Solvingtheargmax problemforsequences

• Trellissequencelabeling• Anypathrepresentsalabelingofinputsentence

• Goldstandardpathinred

• Eachedgereceivesaweightsuchthataddingweightsalongthepathcorrespondstoscoreforinput/ouputconfiguration

• Anymax-weightmax-weightpathalgorithmcanfindtheargmax

• e.g.ViterbialgorithmO(LK2)

POStaggingCMSC723/LING723/INST725

MarineCarpuat