Natural Language Processing

• Overview• Basic issues (Problems)

– Ambiguity– Anaphora– Metaphors

• The need for knowledge and inference• 5 stages of processing

Natural Language Processing

Speech

signal

Word

string

Syntactic

representation

Semantic

representation

Intended

message

Natural language understanding

Recognition Parsing Interpretation Contextualisation+reasoning

Synthesis sentence construction+lexical selection planning

Natural Language Generation

Applications

• Machine translation• Speech-to-speech translation

• Information extraction • Information retrieval

• Database querying • Instruction following

• Authoring aids (word processing tools)

• …….

Machine Translation• Machine translation = understanding + generation

The spirit is willing but the flesh is weak.EnglishRussian EnglishThe vodka is good but the meat is rotten.

English: She put the bottle/plate on the table.German: stehen/liegen

English: A friend stayed over last nightFrench: un ami/une amie

English: He is clever/sick.Spanish: Él es listo/ Él está enfermo

Engish: a book/ a chair/ a desk Chinese: yi ben shu yi ba yi zi yi zhang shu zhuo

Basic Issues, Problem 1: Ambiguity

Lexical ambiguity

She went to the bank (to get some water/to get some money out)

The pen is in the box

The box is in the pen

The bats were hanging on the wall. They looked spooky in the moonlight.

The bats were hanging on the wall. He picked one and went out to play.

Ambiguity ctd.

Structural ambiguity:– I saw the man on the hill with the telescope– The lamb is ready to eat– Flying planes can be dangerous

– The horse raced past the barn fell.– The old train the next generation

– IRAQI head seeks arms– Two ships collide, one dies.

Basic Issues, Problem 2: Anaphora• The use of a word which refers to, or is a substitute for, a preceding

word or group of words. “they”, “he”, “she”, “it”, etc. – John gave Jill the gun. It was making him nervous.– John met Bill after work. He’d been doing overtime.

• Heuristics:– Check agreement in number and gender– Topic– Last-mentioned entity; Last subject NP; Last animate entity; Last entity before an

interruption

• But having problems with– The city council refused the demonstrators a permit because they

feared/advocated violence.– Harry went to visit Sally, but the door was locked and there was no light.

Basic Issues, Problem 3: Metaphors

• John is a lion.• Yes

– John is brave– John is fierce– John is proud– John has shaggy tawny hair

• No– John is carnivorous– John is feline– John comes from Africa/India

More Problems

• The same expression means different things in different contexts– Where is the water? (drinking water; in chemistry lab, it must be

pure; dealing with a leaky roof, it can be filthy)• There are lots of ways to say the same thing

– Mary was born on October 11.– Mary’s birthday is October 11.

• Sentences are incomplete descriptions of the information that they are intended to convey

– Most students are working very hard

More Problems

• No natural language program can be complete because new word, expressions and meanings can be generated quite freely.

• What is said /== what it meant– Do you know what the time is? – Can you pass the book?

• “non-compositional” interpretations – An old cat/ an old city– A red flag/ a fake gun/ a potential suicide

The Need for Knowledge

• Linguistic knowledge:– lexical: words, their parts of speech (noun, verb, adj,

etc.) and grammatical features (singular, plural, subjective, objective, past, present, future)

– syntactic: grammar rules; rules concerning number and case agreement, etc.

– semantic: knowledge about the entities that can exist in the world, and the entities that can be involved in particular kinds of events

The Need for Knowledge, ctd.

• Non-linguistic knowledge:– General Knowledge (Common sense) : about the

world and the ways entities in it behave. Knowledge about people & their goals & behaviour is particularly important.

Sally and Sarah are sisters

Sally and Sarah are mothers

– Specific facts: about individual entities in the world, e.g., Wellington is the capital city of NZ, the United Nations is an organization with representatives from governments of countries.

The Need for Inference

The system needs to make inferences from its internal representationProcessing the text itselfExample: Disambiguation• Structural: I saw the man on the hill with the telescope• Lexical:

– The pen is in the box– The box is in the pen

• Referential: The city council refused the demonstrators a permit because they feared/advocated violence.

Answering questionsExample: John went to a store. He picked up some toothpaste and went to

the check-out counter. He paid the cashier and left.Question: What did John buy?

Five Stages of Processing

Natural language processing can be broken down into roughly five stages (which may overlap or be combined):

(1) Morphological processing: find root form and category, eg., “rose”: past tense of “rise” or singular noun.

– Involves lexical and morphological knowledge (how to make plurals, past tense, present continuous tense, adverbs from adjectives, etc).

(2) Syntactic processing: parse sequence of words into constituent structure.

– Involves lexical and syntactic knowledge: grammar, number and case agreement, etc.

– Syntactic constraints may resolve some lexical or syntactic ambiguities.

Five Stages of Processing, ctd.

(3) Semantic processing: assign meanings to syntactic structures. Constraints about entities and actions are applied.

(4) Discourse processing: interpret sentence in context of discourse. Entities referred to must have been introduced, and overall discourse structure must be consistent. Can help to resolve ambiguities.

(5) Pragmatic analysis: reinterpret structure representing what was said to determine what was meant. Can use a wide range of knowledge about the world, and in particular, knowledge about peoples beliefs and goals and conversational conventions.– Do you know what the time is?– Can you pass the salt?

Summary

• NLP is full of hard problems• NLP is full of interesting problems