Practical Applications
Machine translation
Database access
Information Retrieval
Query-answering
Text categorization
Summarization
Data extraction
Machine Translation
Proposals for mechanical translators of languages pre-date the invention of the digital computer
First was a dictionary look-up system at Birkbeck College, London 1948
American interest started by Warren Weaver, a code breaker in WW2, was popular during cold war, but alas, rather unsuccessful
Machine Translation: Working Systems
Taum-Meteo – Translates Weather reports from English to French in Montreal. Works because language used in reports is stylized and regular.
Xerox Systram – Translates Xerox manuals from English to all languages that Xerox deals in. Utilized pre-edited texts
Machine Translation: Difficulties
Need a big Dictionary with Grammar rules in both (or all) languages, large start-up cost
Direct word translation often ambiguousLexicons (words that aren’t in a dictionary, but made
of common parts)(ex. Lebensversicherungsgesellschaftsangestellter,
a life insurance company employee)
Ambiguity even in primary languageElements of language are different
Machine Translation: Difficulties
Essentially requires a good understanding of the text, and finding a corresponding text in the target language that does a good job of describing the same (or similar) situation.
Requires computer to “understand”.
Machine Translation: Successes
Limited Domain allows for limited vocabulary, grammar, easier disambiguation and understandingJournal article: Church, K.W. and E.H. Hovy. 1993. Good Applications for Crummy Machine Translation. Machine Translation 8 (239--258)
MATmachine-aided translation, where a machine starts, and a real person proof-reads for clarity. (Sometimes doesn’t require bi-lingual people).
Example of MAT (page 692)
The extension of the coverage of the health services to the underserved or not served population of the countries of the region was the central goal of the Ten-Year Plan and probably that of greater scope and transcendence. Almost all the countries formulated the purpose of extending the coverage although could be appreciated a diversity of approaches for its attack, which is understandable in view of the different national policies that had acted in the configuration of the health systems of each one of the countries. (Translated by SPANAM: Vasconcellos and Leon, 1985).
Database Access
The first major success for NLP was in the area of database access
Natural Language Interfaces to Databases were developed to save mainframe operators the work of accessing data through complicated programs.
Database Access:Working Systems
LUNAR (by Woods for NASA, 1973)
allowed queries of chemical analysis data of lunar rock and soil samples brought back by Apollo missions
CHAT (Pereira, 1983)
allows queries of a geographical database
Database Access: Difficulties
Limited VocabularyUser must phrase question correctly – system doesn’t understand everything
Context detectionallowing questions that implicitly refer to previous questions
Becomes Text Interpretation question
Database Access: Conclusion
Worked well for a time
Now more information is stored in text, not in databases (ex. email, news, articles, books, encyclopedias, web pages)
The problem now is not to find information, it’s to sort through the information that’s available.
Information Retrieval
Now the main focus of Natural Language Processing
There are four types: 1. Query answering2. Text categorization3. Text summary 4. Data extraction
Information Retrieval: The task
Choose from some set of documents ones that are related to my query
Ex. Internet search
Information RetrievalMethods
Boolean: “(Natural AND Language) OR (Computational AND Linguistics)”
• too confusing for most users
Vector: Assign different weights to each term in query. Rank documents by distance from query and report ones that are close.
Information Retrieval
Mostly implemented using simple statistical models on the words only
More advanced NLP techniques have not yielded significantly better results
Information in a text is mostly in its words
Text Categorization
Once upon a time… this was done by humansComputers are much better at it (and more
consistent)Best success for NLP so far (90+ % accuracy)Much faster and more consistent than humans.
Automated systems now perform most of the work.
NLP works better for TC than IR because categories are fixed.
Text Summarization
Main task: understand main meaning and describe in a shorter way
Common Systems: Microsoft
How: – Sentence/paragraph extraction (find the most
important sentences/paragraphs and string them together for a summary)
– Statistical methods are more common
Data extraction
Goal: Derive from text assertions to store in a database
Example: SCISOR, Jacobs and Rau 1990
Summarizes Dow Jones News stories, and adds information to a database.
NLP Goals
Have (or feign) some understanding based on communication with Natural Language
In order to receive and send information in ways easily understandable by human users
How to get there
NLP applications are all similar in that they require some level of understanding.
Understand the query, understand the document, understand the data being communicated…
Understanding Sentences: Overview
Parsing and GrammarHow is a sentence composed?
LexiconsHow is a word composed?
Ambiguity
Parsing Requirements
Requires a defined Grammar
Requires a big dictionary (10K words)
Requires that sentences follow the grammar defined
Requires ability to deal with words not in dictionary
Parsing (from Section 22.4)
Goal: Understand a single sentence by syntax analysis
Methods – Bottom-up– Top-down
More efficient (and complicated) algorithm given in 23.2
A Parsing Example
Rules:
The Sentence: The boy went home.
S NP VP
NP Article N | Proper
VP Verb NP
N home | boy | store
Proper Betty | John
Verb go|give|see
Article the | an | a
Lexicons
The current trend in parsing
Goal: figure out this word
Method: 1. Tokenize with morphological analysis
Inflectional, derivational, compound
2. Dictionary lookup on each token
3. Error recovery (spelling correction, domain-dependent cues)
Lexicons in Practice
10,000 – 100,000 root word forms
Expensive to develop, not readily shared
Wordnet (George Miller, Princeton)
clarity.princeton.edu
Ambiguity
More extensive Language more Ambiguity
Disambiguation: task of finding correct interpretation
Evidence: • Syntactic • Lexical• Semantic • Metonymy• Metaphor
Disambiguation Tools
Syntaxmodifiers (prepositions, adverbs) usually attach to nearest possible place
Lexicalprobability of a word having a particular meaning, or being used in a particular way
Semanticdetermine most likely meaning from context
Semantic Disambiguation Example: “with”
Sentence RelationI ate spaghetti with meatballs. (ingredient of spaghetti)
I ate spaghetti with salad. (side dish of spaghetti)
I ate spaghetti with abandon. (manner of eating)
I ate spaghetti with a fork. (instrument of eating)
I ate spaghetti with a friend. (accompanier of eating)
Disambiguation is probabilistic!
More Disambiguation Tools
Metonymy
“Chrysler announced” doesn’t mean companies can talk.
Metaphor
more is up: confidence has fallen, prices have sky-rocketed.
Beyond Sentences: Discourse understanding
Sentences are nice but…
Most communication takes place in the form of multiple sentences (discourses)
There’s lots more to the world than parsing and grammar!
Discourse Understanding: Goals
Correctly interpret sequences of sentences
Increase knowledge about world from discourse (learn)– Dependent on facts as well as new knowledge
gained from discourse.
Discourse Understanding: an example
John went to a fancy restaurant.He was pleased and gave the waiter a big tip.He spent $50.
What is a proper understanding of this discourse?
What is needed to have a proper understanding of this discourse?
General world knowledge
• Restaurants serve meals, so a reason for going to a restaurant is to eat.
• Fancy restaurants serve fancy meals, $50 is a typical price for a fancy meal. Paying and leaving a tip is customary after eating meals at restaurants.
• Restaurants employ waiters.
General Structure of Discourse
“John went to a fancy restaurant. He was pleased…”
Describe some steps of a plan for a character
Leave out steps that can be easily inferred from other steps.
From first sentence: John is in the eat-at-restaurant plan. Inference: eat-meal step probably occurred – even if it wasn’t mentioned.
Syntax and Semantics
“...gave the waiter a big tip.”
“the” used for objects that have been mentioned before
OR
Have been implicitly alluded to; in this case, by the eat-at-restaurant plan
Specific knowledge about situation
“He spent $50”
• “He” is John.
• Recipients of the $50 are the restaurant and the waiter.
Structure of coherent discourse
Discourses comprised of segmentsRelations between segments
(more in Mann and Thompson, 1983)(coherence relation)
– Enablement– Evaluation– Causal– Elaboration– Explanation
Speaker Goals (Hobbs 1990)
The Speaker does 4 things:1) wants to convey a message
2) has a motivation or goal
3) wants to make it easy for the hearer to understand.
4) links new information to what hearer knows.
A Theory of “Attention”
Grosz and Sidner, 1986
Speaker or hearer’s attention is focused
Focus follows a stack model
Explains why order is important.
Order is important
What’s the difference?
I visited Paris. I visited Paris.
I bought you some Then I flew home.
expensive cologne.
Then I flew home. I went to Kmart.
I went to Kmart. I bought you some expensive cologne.
I bought some underwear. I bought some underwear.
Summary
• NLP have practical applications, but none do a great job in an open-ended domain
• Sentences are understood through grammar, parsing and lexicons
• Choosing a good interpretation of a sentence requires evidence from many sources
• Most interesting NLP comes in connected discourse rather than in isolated sentences
Current NLP Crowd
– Originally, mostly mathematicians. – Now Computer Scientists (computational
linguists= linguists, stasticians, computer science folk).
– Big names are Perrault, Hobbs, Pereira, Grosz and Charniak
Current NLP conferences
Association for Computational Linguistics
Coling
EACL (Europe Association for Computational Linguistics)
USA Schools with NLP Grad.Brown UniversityBuffalo, SUNY atCalifornia at Berkeley, University ofCalifornia at Los Angeles, University ofCarnegie-Mellon UniversityColumbia UniversityCornell UniversityDelaware, University ofDuke UniversityGeorgetown UniversityGeorgia, University ofGeorgia Institute of TechnologyHarvard UniversityIndiana UniversityInformation Sciences Institute (ISI) at the University
of Southern CaliforniaJohns Hopkins University
Massachusetts at Amherst, University ofMassachusetts Institute of TechnologyMichigan, University ofNew Mexico State UniversityNew York UniversityOhio State UniversityPennsylvania, University ofRochester, University ofSouthern California, University ofStanford UniversityUtah, University ofWisconsin - Milwaukee, University ofYale University
Current NLP Journals
Computational Linguistics
Journal of Natural Language Engineering (JLNE)
Machine Translation
Natural Language and Linguistic Theory
Industrial NLP Research Centers
AT&T Labs - ResearchBBN Systems and Technologies CorporationDFKI (German research center for AI)General Electric R&DIRST, ItalyIBM T.J. Watson Research, NYLucent Technologies Bell Labs, Murray Hill, NJMicrosoft Research, Redmond, WAMITRENEC CorporationSRI International, Menlo Park, CASRI International, Cambridge, UKXerox, Palo Alto, CAXRCE, Grenoble, France
Speaker Goals (Hobbs 1990)
The Speaker does 4 things:1) wants to convey a message
2) has a motivation or goal
3) wants to make it easy for the hearer to understand.
4) links new information to what hearer knows.
Discourse comprehensionThe procedure is actually quite simple. First you arrange things into different groups. Of course, one pile may be sufficient depending on how much there is to do. If you have to go somewhere else due to lack of facilities that is the next step, otherwise you are pretty well set. It is important not to overdo things. That is, it is better to do too few things at once than too many. In the short run this may not seem important but complications can easily arise. A mistake is expensive as well. At first the whole procedure will seem complicated. Soon however, it will become just another facet of life. It is difficult to foresee any end to the necessity of this task in the immediate future, but then one can never tell. After the procedure is completed one arranges the material into different groups again. Then they can get put into their appropriate places. Eventually they will be used once more and the whole cycle will have to be repeated. However, this is a part of life.
Now: What do you remember?
What are the four steps mentioned?
What step is left out?
What is the “material” mentioned?
What kind of mistake would be expensive?
Is it better to do too few or too many?
Why?
Top Related