Is Question Answering an Acquired Skill?

Ramakrishnan, Chakrabarti, Paranjpe, Bhattacharyya

Paper presentation: Vinay Goel

Introduction

• Question Answering (QA) system• Most QA systems are substantial team efforts

– Difficult to reproduce a well tuned QA system from scratch, gauge the benefit of new algorithmic ideas, new corpora and new languages

• QA systems– Complex building blocks like taggers and parsers– Lashed together with customized “glue”– Crucial knobs best preset by QA specialists

Goal

• Decompose the QA task cleanly into discovering features and learning to score answer snippets

• QA system– Performs fast, shallow processing of corpus– Structures the scoring task using features and learners– Trains its scoring algorithm from a past history of

questions and vetted answers– Can include side information (Wordnet etc.)– Reuses expertise accumulated from one corpus to a

new corpus

Noisy simulation perspective

• In a structured database with a suitable schema and a structured query language, information needs can be expressed clearly

• See QA as a transformation of this process by adding natural language from an unknown generative process, for both query and data

• Given a question, discover structured fragments in it– Extract selectors which will appear almost unchanged in an

answer passage– Extract atype clues, which tell what else to look for in a

passage that satisfied all selectors

Typical connections between a question and answer

Atype

• Minimal subclass of entities which will answer a question

• Two representations important to factoid QA– Atype as synset– Atype as surface patterns

Atype as synset

• Q: Name an animal that sleeps upright

• A: horse

• Wordnet helps recognize that horse is an instance of animal

• Most answers which are common nouns are assisted by this representation

Atype as surface patterns

• Infinite or very large domains such as numbers, person names, place names etc. cannot be covered by Wordnet

• Logically augment Wordnet to add connections from synsets to pattern matchers such as “at DD:DD” or “Xx+ said” etc.

From the question to an atype

• Set of common “wh-words”

• Questions starting with when, where and who immediately reveal their expected atypes

• Word after how is almost always a clue

• Questions using using what and which mention atype directly

Shallow parsing to extract atype

• Shallow parsing involves finding noun phrases, modifiers and attachments between phrases

• Purely based on POS tags• Strategy for locating atype clues from “what”

and “which” questions:– Head of NP appearing before the auxiliary / main

verb if it is not a wh-word– Otherwise, head of NP appearing after the verb

Learning to map atype

• When, where, who and how do not directly use a term that describes a synset

• Augmented synsets based on surface patterns (DDDD) may come handy

• Devised a learning module to help compile mappings between short token sequences in questions to atypes

Selectors

• Second blank in the SQL query select…where…

• In QA, simply a set of words in the question that are expected to appear unchanged in the answer passage

Identifying selectors

• Choice of features– POS– POS assigned to left and right neighbors– Whether the word starts with an uppercase letter

– Whether the word is a stopword– Some version of IDF– How many senses the word has in isolation– For a given sense, how many other words

describe the sense

How to use selectors

• Two places– Pad the initial keyword query– Rerank the candidate phases

• Experiments insist that the response by the word search engine (Lucene)– Contains all selectors– Use OR over other question words

Learning to score passages

• If (q,r) is a positive instance, it is expected that– All selectors match between q and r– r has an answer zone a which does not

contain selectors– The linear distance between a and

matched selectors in r, tend to be small– a has strong Wordnet-based similarity with

the atype of q

Overall architecture

Experiments

• TREC QA track• Picked sliding windows of three

sentences as passages• Questions and passages were

tokenized using GATE• For learning tasks, used J48 decision

tree and the logistic regression packages in WEKA

Extracting atypes from shallow parses

Spotting selectors in questions

Passage Reranking performance

MRR improvement via reranking

Training on corpus of another year

Conclusion

• QA system built by wrapping logic around text indexers, taggers, shallow parsers and classifiers

• Simple assembly of building blocks

• Future work involves improving performance of different blocks