As Measured by Performance on
Tasks We're Getting Better at*
* tasks that would be easy if we were good at natural language understanding and that we therefore use to measure our progress toward natural language understanding
Winograd Schemas
The conference organizer disinvited the speaker because he feared a boring talk.
The conference organizer disinvited the speaker because he proposed a boring talk.
conference organizer
speaker
(I Am Being Unfair)
Each of these tasks is valuable on its own merits
Likely they are getting us closer to actual natural language understanding
Grammars
SS -> NP VP
NP VPVP -> VBZ ADJP
NP VBZ ADJPNP -> JJ NN
JJ NN VBZ ADJPADJP -> JJ
JJ NN VBZ JJJJ -> "Artificial"NN -> "intelligence"VBZ -> "is"JJ -> "dangerous"
Artificial intelligence is dangerous
0 0 0 0 0 0 0 0 0 1 0 0 0 0 ... 0
.01 0 0 .9 0 0 0 0 0 .05 0 0 0 0 ... 0
.3 .6 .1 .2 2.3
Joel is attending an artificial intelligence conference.
artificial
intelligence
embedding
prediction
"NLP's ImageNet moment"
Task 1: Masked Language Modeling
Joel is giving a [MASK] talk at a [MASK] in San Francisco
interestingexcitingderivativepedestriansnooze-worthy...
conferencemeetupravecoffeehouseWeWork...
Task 2: Next Sentence Prediction[CLS] Joel is giving a talk. [SEP] The audience is enthralled. [SEP]
[CLS] Joel is giving a talk. [SEP] The audience is falling asleep. [SEP]
99% is_next_sentence1% is_not_next_sentence
1% is_next_sentence99% is_not_next_sentence
In Conclusion● NLP is cool● Modern NLP is solving really hard
problems● (And is changing really really quickly)● Lots of really smart people with lots of
data and lots of compute power have trained models that you can just download and use
● So take advantage of their work!
I'm fine-tuning a transformer model!
Thanks!● I'll tweet out the slides: @joelgrus
○ read the speaker notes○ they have lots of links
● I sometimes blog: joelgrus.com● AI2: allenai.org● AllenNLP: allennlp.org● GPT-2 Explorer: gpt2.apps.allenai.org● podcast: adversariallearning.com
Referenceshttp://ruder.io/a-review-of-the-recent-history-of-nlp/https://ankit-ai.blogspot.com/2019/03/future-of-natural-language-processing.htmlhttps://lilianweng.github.io/lil-log/2019/01/31/generalized-language-models.html#openai-gpt
Top Related