From Web n-grams to collocation learning

Shaoqun Wu

University of Waikato

strong smoker

powerful tea

big rain

look book

one sex product

heavy smoker

strong smoker

heavy rain

read a book

disposable product

What is collocation?

Choice of one word conditions the choice of the next, and of the next again

John McHardy Sinclair

Learning collocation is important

Existing collocation resources

What are you doing tonight ?

I am going to look TV.

Watch TV?

Z Z Z

Building a collocation database from the Web text

Google n-gram collection

word_1 <space> word_2 <space>… word_n <tab> count

tokens 1,024,908,267,229 1012

sentences 95,119,665,584 0.95109

unigrams 13,588,391 0.014109

bi-grams 314,843,401 0.3109

trigrams 977,069,902 1.0109

four-grams 1,313,818,354 1.3109

five-grams 1,176,470,663 1.2109

Building a Web collocation collection

• clean up n-grams,

• assign syntactic tags to the words of n-grams,

• match tagged n-grams with the syntactic patterns, and

• discard ones that occur less than 100 times.

http://flax.nzdl.org

Parents have a wide range of options to cultivate their children with a good art understanding

cultivate their children

http://flax.nzdl.org

Evaluation

• language learners from the education school86% vs 67%

• myself: diary recording …

• and you?

Questions

specific questions

serious questions

tough questions

difficult questions

hard questions

critical question

simple questions

general questions

interesting questions

various questions

common questions

no further questions

no stupid questions

THERE ARE