Proceedings of the Society for Computation in Linguistics

Volume 2 Article 33

2019

Quantifying Structural and Lexical Constraints inPP Ordering TypologyZoey LiuUniversity of California, Davis, yiliu@ucdavis.edu

Follow this and additional works at: https://scholarworks.umass.edu/scil

Part of the Computational Linguistics Commons

This Extended Abstract is brought to you for free and open access by ScholarWorks@UMass Amherst. It has been accepted for inclusion in Proceedingsof the Society for Computation in Linguistics by an authorized editor of ScholarWorks@UMass Amherst. For more information, please contactscholarworks@library.umass.edu.

Recommended CitationLiu, Zoey (2019) "Quantifying Structural and Lexical Constraints in PP Ordering Typology," Proceedings of the Society for Computationin Linguistics: Vol. 2 , Article 33.DOI: https://doi.org/10.7275/ch62-ft92Available at: https://scholarworks.umass.edu/scil/vol2/iss1/33

Quantifying Structural and Lexical Constraints in PP Ordering Typology

Zoey LiuDepartment of Linguistics

University of California, DavisOne Shields Avenue, Davis, CA

yiliu@ucdavis.edu

1 Introduction

Recent research has demonstrated across lan-guages that the overall or average dependencylengths tend to be minimized by their grammarsas a whole (Futrell et al., 2015). Other experi-ments looking at specific syntactic constructionsof individual languages have shown that if theconstituents in a sentence have flexible orderings,there is a preference for shorter constituents to ap-pear closer to their syntactic heads (Jaeger andNorcliffe, 2009). Nevertheless, whether there iscrosslinguistic preference for dependency lengthminimization (DLM) in specific syntactic struc-tures with alternative constituent orderings re-mains unclear. Additionally, previous studieshave shown contrary evidence and suggested thatwhether DLM exists relies on the structural char-acteristics of the language and the particular order-ing constructions (Lohmann and Takada, 2014).Besides, what other linguistic factors can serveas effective typological determinants of word or-der preferences and their relationship with depen-dency length await further experimentation.

This study starts to bridge the gap by inves-tigating prepositional and postpositional phrase(PP) order typology. Previous experiments re-garding PP ordering have mainly focused on En-glish (Hawkins, 1999; Liu and Sagae, 2018). Us-ing multilingual corpora from the Universal De-pendencies project (Nivre et al., 2017), we presentthe first large-scale crosslinguistic exploration ofPP orderings across 31 languages. More specif-ically, we focus on sentences with verb phrases(VP) that contain exactly two PPs, the orderingof which in some contexts allows certain flexi-bility. In other words, the placement of the twoPPs is either less or not constrained by the gram-mar of the language and hence language usershave a choice towards the relative ordering of the

PPs. We examine the effects of four constraintsthat have been suggested to affect constituent or-derings and/or language processing: dependencylength, semantic closeness, lexical frequency andword co-occurrence information.

2 Related Work & Motivation

Dependency length Previous studies have shownthat PP orderings in English demonstrate DLMwhere shorter PP dependents tend to appear closerto the head verb. We investigate whether such ten-dency holds for PP ordering typology. To estimatethe effect of dependency length, for each instancewe measured and compared the length1 of the PPcloser to the verb and of the PP farther from theverb.

Semantic closeness Though it has been sug-gested constituents that are semantically relatedshould occur together syntactically (Wasow andArnold, 2003), the precise meaning of semanticcloseness as well as the ideal way to computeit have not been explored much. We investigatewhether the PP which is semantically closer tothe head verb would appear closer. We approxi-mate semantic closeness as how semantically sim-ilar the head verb and the nominal head of eachPP are, measured with cosine similarity and wordembeddings2. We consider the PP with a nomi-nal head that is semantically more similar with itshead verb to be the PP that is semantically closer.

Lexical frequency Few studies have looked atthe role of lexical frequency in syntactic orderings.Morgan and Levy (2015) demonstrated that themore frequent word tends to appear first in bino-mial expressions. Liu and Sagae (2018) presentedsimilar effects for lexical frequency in PP ordersin English. Following Liu and Sagae (2018), we

1The length of each PP is measured as the number of to-kens it has following the annotation scheme of the corpora.

2https://github.com/bheinzerling/bpemb

306Proceedings of the Society for Computation in Linguistics (SCiL) 2019, pages 306-309.

New York City, New York, January 3-6, 2019

investigate whether there is crosslinguistic prefer-ence for the PP with more frequent words to occurfirst. We approximate lexical frequency of eachPP with estimations from language unigram modelfor each language using a Wikipedia dump3.

Word Co-occurrence Information Recentstudies indicate that words that are more likely toco-occur with each other prefer to appear closer(Futrell and Levy, 2017). We investigate whetherthe PP that is more likely to co-occur with thehead verb would be placed closer. We use point-wise mutual information (PMI) to measure the co-occurrence information of the head verb and thenominal head within each PP. We regard the PPwith a nominal head that has higher PMI with thehead verb to be the PP that is more likely to co-occur with the verb.

3 Experiments & Results

Our results4 show four different PP orders as de-scribed in Table 1.

3.1 Languages with head-initial PP afterhead verb

(a) He talked⇥

with friends⇤ ⇥

about the new movie⇤

.

(b) He talked⇥

about the new movie⇤ ⇥

with friends⇤

.

For consistently head-initial languages like En-glish, when a VP contains two PP dependentson the same side of the head verb, it will havehead-initial PPs after the head verb, where theheadedness of the PPs matches that of the VP.Based on our initial conjectures regarding theeffects of the four factors, it is clear that de-pendency length, semantic closeness and wordco-occurrence information always pull in the samedirection regardless of the PP ordering structureof the language. If we assume that shorter PPstend to be more frequent, a preference for shorterdependencies will place the shorter PP closer tothe verb, which also means that the presumablymore frequent PP will occur first. In this case,

3https://dumps.wikimedia.org/.4All results from Monte Carlo permutation test with

1,000,000 iterations.

lexical frequency will be co-operating with theother three factors in the same direction as well.

Eleven languages demonstrate the same PP or-dering pattern as English, all favoring DLM. Thenumber of sentences with the shorter PP closer tothe verb is on average 4 times that of sentenceswith the longer PP closer to the verb. There doesseem to be a general preference for PPs with morefrequent words to appear first; nonetheless, the ef-fect is not quite as pronounced as that of depen-dency length. On the other hand, semantic close-ness and word-co-occurrence information do notappear to affect PP orderings overall.

3.2 Languages with head-final PP beforehead verb

For consistently head-final languages likeJapanese, the headedness consistency between thePPs and the VP also exists, except that both arehead-final. When the shorter PP is closer to theverb, which reduces overall dependency length, itis the second in the sequence of the two PPs andthe longer PP appears first. However, if the shorterPP is in fact more frequent, that will violate theexpectation that the more frequent PP shouldoccur first. In this case, lexical frequency willbe pulling in different directions from the otherthree factors. Thus we expect effects based ondependency length, semantic closeness, and wordco-occurrence information to be weaker than inlanguages like English, where the four constraintsare in agreement with regard to PP ordering.

(c)✏/⇥�⌅ :S '

⇤ ⇥ró '

⇤�⌅�

He [ a red pen with ] [ himself by ] wrote.

(d)✏/⇥ró '

⇤ ⇥�⌅ :S '

⇤�⌅�

He [ himself by ] [ a red pen with ] wrote.

‘He wrote by himself with a red pen.’

In our dataset, Japanese and Hindi show VP in-stances with exactly two head-final PP dependentsbefore the head verb. Though there appears tobe a preference for DLM in Japanese, the effect

307

Word order features PP ordering pattern LanguagesConsistently head-initial head-initial PP after head verb English, Arabic, Danish, Galician, Greek, Hebrew,

Indonesian, Norwegian, Serbian, Slovak, Swedish

Consistently head-final head-final PP before head verb Japanese, Hindi

Relatively free word order head-initial PP before German, Bulgarian, Croatian, Czech, Dutchor after head verb Latin, Russian, Slovenian, Ukranian

Catalan, French, Italian, Portuguese, Romanian, SpanishAfrikaans, Persian

Typologically mixed head-initial PP before head verb Chinese

Table 1: PP ordering patterns of four types of languages based on our dataset.

of dependency length is much weaker comparedto consistently head-initial languages. The ratiobetween the number of VPs with the shorter PPcloser to the verb and the number of VPs with thelonger PP closer to the verb is 1.7, which is sub-stantially lower than the 4.0 average ratio for lan-guages as English, and even substantially lowerthan the lowest of these ratios, 2.4. The contrastbetween Hindi and consistently head-initial lan-guages is even more pronounced, with a slightoverall preference (1.14 ratio) for longer PPs tobe closer to the verb. No effects have been ob-served for either semantic closeness or word co-occurrence information. Despite this, lexical fre-quency seems to play a role for PP ordering inJapanese. The overall patterns for the four factorsspeak to our initial conjecture that in consistentlyhead-final languages, lexical frequency is partiallycompeting against the other three factors.

3.3 Languages with PPs before or after headverb

(e) dass er⇥

auf dem Tisch⇤ ⇥

mit ihr⇤

isst

that he [ on the table ] [ with her ] eats

(f) dass er⇥

mit ihr⇤ ⇥

auf dem Tisch⇤

isst

that he [ with her ] [ on the table ] eats

‘that he eats with her on the table.’

Whether the PPs and the VP are consis-tently head-initial or head-final, the Englishand Japanese examples demonstrate that havingmatching headedness between the VP and the PPs

results in optimal overall dependency length whenthe shorter PP is closer to the verb. However,for some languages with relatively free word or-der, particularly those with a mix of verb-initialand verb-final clausal structures such as Germanor Czech, the PP ordering pattern is more compli-cated. These languages have predominantly head-initial PPs, which show the same orderings as En-glish. Besides, they also exhibit cases where twohead-initial PPs appear before the head verb, as in(e) and (f). However, when head-initial PPs appearbefore the verb, the cost of the longest dependencybetween the head verb and the first PP is incurredno matter what the order of the PPs is. Hence inlanguages with inconsistent headedness betweenPPs and VPs, there may not be an as pronouncedpreference for DLM. Nevertheless, since semanticcloseness and word co-occurrence information areless subject to the cost of the long dependency, it ispossible that these two factors are able to compar-atively account for more of PP ordering patternsin these languages. Similar to Japanese, since thetwo PPs appear before the head verb, lexical fre-quency will be working in opposition to the otherthree factors as well.

Fifteen languages in our dataset have head-initial PPs appearing both before or after the headverb. When the PPs appear after the head verb,ordering preferences for DLM are closely alignedwith those that are consistently head-initial. Whenthe PPs appear before the head verb, however, theobservations for DLM are substantially weaker.The six Romance languages in particular (Cata-lan, French, Italian, Portuguese, Romanian, Span-ish) exhibit uniform patterns that are against DLM,where longer PPs appear closer to the head verb.Lexical frequency appears to play a role when thePPs are after the head verb, with results compa-rable to those for the consistently head-initial lan-

308

guages. Whereas we observe mixed patterns whenthe PPs occur before the verb as well. No signif-icant effects have been found for semantic close-ness and word co-occurrence information regard-less of where the PPs appear.

Besides the fifteen, two other languages withrelatively free word order, Afrikaans and Per-sian, present only cases where two head-initialPPs occur before the head verb. Both languagesshow a preference for shorter dependencies, yetboth present the opposite ordering observationsfor lexical frequency. Semantic closeness doesnot seem to be effective. On the other hand,word co-occurrence information seems to play thestrongest role among the four factors for the twolanguages.

3.4 Languages with head-initial PP beforehead verb

(g)Ÿõ⇥å ÷ Ñ Ñ�

⇤ ⇥( hb

⇤ ˙e

These (comments) [ with his prophecy ] [ on thesurface ] have differences.

(h)Ÿõ⇥( hb

⇤ ⇥å ÷ Ñ Ñ�

⇤ ˙e

These (comments) [ on the surface ] [ with hisprophecy ] have differences.

‘These comments have differences on the surfacewith Virgil’s prophecy.’

For typologically mixed languages as MandarinChinese, the inconsistency of headedness betweenthe PPs and the VP is also observed. In the Chi-nese data, all found VP instances have head-initialPPs before the head verb. While the PP orderspresent opposite preferences for DLM, the otherthree factors show positive effects. This suggeststo our initial conjecture that when inconsistentheadedness exists, that when the long dependencyis already incurred, the other three factors may po-tentially have more pronounced effects.

In sum, we demonstrate empirically that thefour factors are co-operating as well as competing

motivations for PP orderings, and that the effectsof these constraints are not necessarily language-universal, but dependent on the structural featuresof different language types. The fact there is cer-tain amount of data where none of the four con-straints are effective indicates there are other lin-guistic motivations, possibly involving discoursestructure, that come into play. We leave that forfuture work.

ReferencesRichard Futrell and Roger Levy. 2017. Noisy-

context surprisal as a human sentence processingcost model. In Proceedings of the 15th Conferenceof the European Chapter of the Association for Com-putational Linguistics (EACL). page 688–698.

Richard Futrell, Kyle Mahowald, and Edward Gibson.2015. Large-scale evidence of dependency lengthminimization in 37 languages. Proceedings ofthe National Academy of Sciences 112(33):10336–10341.

John A. Hawkins. 1999. The relative order of prepo-sitional phrases in english: Going beyond manner–place–time. Language variation and change11(3):231–266.

T. Florian Jaeger and Elisabeth J. Norcliffe. 2009. Thecross-linguistic study of sentence production. Lan-guage and Linguistics Compass 3(4):866–887.

Zoey Liu and Kenji Sagae. 2018. Dependency lengthminimization and lexical frequency in prepositionalphrase ordering in english. Proceedings of the Soci-ety for Computation in Linguistics 1(1):189–192.

Arne Lohmann and Tayo Takada. 2014. Order in npconjuncts in spoken english and japanese. Lingua152:48–64.

Emily Morgan and Roger Levy. 2015. Modelingidiosyncratic preferences: How generative knowl-edge and expression frequency jointly determinelanguage structure. In Proceedings of the 37th An-nual Meeting of the Cognitive Science Society. page1649–1654.

Joakim Nivre, Zeljko Agic, Lars Ahrenberg, andet al. 2017. Universal dependencies 2.1. LIN-DAT/CLARIN digital library at the Institute ofFormal and Applied Linguistics (UFAL), Facultyof Mathematics and Physics, Charles University.http://hdl.handle.net/11234/1-2515.

Thomas Wasow and Jennifer Arnold. 2003. Post-verbal constituent ordering in english. Topics in En-glish Linguistics 43:119–154.

309