Open-Source Implementation of Document Structuring Algorithm
for NLTK
Nicholas FitzGerald
Natural Language Generation
Generate coherent text outputs to express information
Express the right information Express information in the right order
NLG Tasks
1. Document Structuring - most important and relevant information selected from knowledge base (Content Determination), then ordered and structured in such a way as to maximize coherence and informativeness (Text Planning)
2. Micro-Planning – specifics of word selection, referring expressions, and the finalization of ordering are determined
3. Realization – internal representations of the above decisions are realized in actual text output
Document Structuring
Given a set of information to be expressed, determine the order and grouping of this information
Texts cannot be simply a random bag of sentences Order of message presentation has significant effect on
meaning [Hovy 1993]: One way:
1 - “Maria was diagnosed with cancer some months ago.” 2 - “Maria and Zurab had a fight last night.” 3 - “She was found dead this morning.”
Vs. 1 - “Maria was diagnosed with cancer some months ago.” 2 - “She was found dead this morning.” 3 - “Maria and Zurab had a fight last night.”
Document Structuring
Ordering also effects coherence: “John was hungry. John went to the store. He bought
some bread to make a sandwich.” “John bought some bread to make a sandwich. He went
to the store. John was hungry.”
Discourse relations
Relationship between a message or group of messages
Elaboration(m1,m2) I love jazz music(m1). My favourite album is Oscar
Peterson's “Night Train” (m2). Contrast(m1, m3)
I love jazz music (m1). However, my favourite album is The Beatles' “White Album” (m3).
Cue word - However
Rhetorical Structure Theory
Mann and Thompson 1988 A text is coherent by virtue of relationships that
hold between messages in the text A small number of relations (~25) can explain
relationships between messages in a wide range of text
Project Proposal
Implement these general algorithms for inclusion in NLTK
Provide a sample Data Set and DR schema for testing and illustration
based on hypothetical WeatherExplainer from [Reiter and Dale 2000]
Experiment utilizing these new tools as part of Abstractive Summarization System for Evaluative Statement Summarization (ASSESS)
Implementation 1: Schemas
Top-Down Approach Output document structure is predictable and
stereotyped Schemas are patterns of expansion, similar to CFG Ie:
CompareAndContrast → DescribeRelationship CompareProperties.
CompareProperties → CompareProperty CompareProperties.
CompareProperties → . “John is much bigger than Kate (DR). He is five inches
taller (CP) and weighs almost twice as much (CP).” Specify rules for choosing if multiple expansions exist
Top-Down Problems
Hypothesis-Driven Content selection done “on-line” Not easily pipelined
Therefore, Bottom-Up used
Implementation 2: Bottom-Up
Output document structure is not predictable
POOL = messages to be expressedwhile( size(pool) > 1)):
find all pairs of elements in pool which can be joined by a DRassign a desirability score to each potential DRfind pair E
i and E
j with highest score and combine with E
k
remove Ei and E
j from POOL, replace with E
k
end while
Implementation
Used nltk.featstruct for Messages and DocPlans A mapping from feature identifiers to feature values, where each
feature value is either a basic value (such as a string or an integer), or a nested feature structure.
TotalRainfallMsg period year 1996 month 06 attribute type 'RelativeVariation' magnitude unit 'inches' number 4 direction '+'
[ *msgType* = 'TotalRainfallMsg' ][ ][ [ direction = '+' ] ][ [ ] ][ attribute = [ magnitude = [ number = 4 ] ] ][ [ [ unit = 'inches' ] ] ][ [ ] ][ [ type = 'RelativeVariation' ] ][ ][ period = [ month = 6 ] ][ [ year = 1996 ] ]
Implementation
nltk.featstruct.FeatStruct unify(other):
Unify fstruct1 with fstruct2, and return the resulting feature structure. This unified feature structure is the minimal feature structure that: contains all feature value assignments from both fstruct1 and
fstruct2. preserves all reentrance properties of fstruct1 and fstruct2.
If no such feature structure exists (because fstruct1 and fstruct2 specify incompatible values for some feature), then unification fails, and unify returns None.
Unification
TotalRainfallMsg period year 1996 Month 06 attribute type 'RelativeVariation' direction '+'
+
TotalRainfallMsg period year 1996 month 06 attribute type 'RelativeVariation' magnitude unit 'inches' number 4
=
TotalRainfallMsg period year 1996 month 06 attribute type 'RelativeVariation' magnitude unit 'inches' number 4 direction '+'
Implementation
nltk.featstruct.FeatStruct subsumes(other):
True if self subsumes other. I.e., return true if unifying self with other would result in a feature structure equal to other.
Subsumes
TotalRainfallMsg period year 1996 Month 06
subsumes
TotalRainfallMsg period year 1996 month 06 attribute type 'RelativeVariation' magnitude unit 'inches' number 4 direction '+'
TotalRainfallMsg period year 1996 month 06 attribute type 'RelativeVariation' magnitude unit 'inches' number 4
Does notsubsume
TotalRainfallMsg period year 1996 month 06
Using Subsumes
”Select from messages all DocPlans whose with a relType of Contrast and a nucleus which is a message of msgType ('TotalRainfallMsg')”
d = DocPlan(relType = 'Contrast', nucleus = Message('TotalRainfallMsg'))return = filter(lambda msg: d.subsumes(msg), messages)
Implementation: Input Formats
Messages:
TotalRainfallMsg period year 1996 month 06 attribute type 'RelativeVariation' magnitude unit 'inches' number 4 direction '+'
Input Formats
Rules:
Elaboration(Message('MonthlyRainfallMsg') M1, Message('TotalRainfallMsg') M2)(M1.attribute.direction == M2.attribute.direction) : ConstituentSet('Elaboration', M1, M2) : 3
inputs
conditions return heuristic
Example Usage
with open('msg_file', 'r') as f:msg_string = f.read()
with open('rule_file', 'r') as f:rule_string = f.read()
messages = read_messages(msg_string)rules = read_rules(rule_string)
plan = bottom_up_plan(messages, rules)
Data Set - WeatherExplainer
Simple example provided in [Reiter and Dale 2000]
Created 3 messages and 3 rules in the input format
WeatherExplainer MessagesTotalRainfallMsg
periodyear 1996month 06
attributetype
'RelativeVariation'magnitude
unit 'inches'
number 4direction '+'
MonthlyRainfallMsgperiod
year 1996month 06
attributetype
'RelativeVariation'magnitude
unit 'inches'
number 2direction '+'
MonthlyTemperatureMsgperiod
year 1996month 06
temperaturecategory 'hot'
WeatherExplainer Messages
Elaboration(Message('MonthlyRainfallMsg') M1, Message('TotalRainfallMsg') M2)(M1.attribute.direction == M2.attribute.direction) : ConstituentSet('Elaboration', M1, M2) : 3
Contrast(Message('MonthlyRainfallMsg') M1, Message('TotalRainfallMsg') M2)(M1.attribute.direction != M2.attribute.direction) : ConstituentSet('Contrast', M1, M2) : 2
Sequence(Message('MonthlyTemperatureMsg')|ConstituentSet(nucleus=Message('MonthlyTemperatureMsg')) M1, Message('MonthlyRainfallMsg')|ConstituentSet(nucleus=Message('MonthlyRainfallMsg')) M2)
() : ConstituentSet(Sequence, M1, M2) : 1
WeatherExplainer Result[ *type* = 'DPDocument' ][ ][ [ [ [ *msgType* = 'TotalRainfallMsg' ] ] ] ][ [ [ [ ] ] ] ][ [ [ [ [ direction = '+' ] ] ] ] ][ [ [ [ [ ] ] ] ] ][ [ [ [ attribute = [ magnitude = [ number = 4 ] ] ] ] ] ][ [ [ *aux* = [ [ [ unit = 'inches' ] ] ] ] ] ][ [ [ [ [ ] ] ] ] ][ [ [ [ [ type = 'RelativeVariation' ] ] ] ] ][ [ [ [ ] ] ] ][ [ [ [ period = [ month = 6 ] ] ] ] ][ [ [ [ [ year = 1996 ] ] ] ] ][ [ [ ] ] ][ [ *aux* = [ [ *msgType* = 'MonthlyRainfallMsg' ] ] ] ][ [ [ [ ] ] ] ][ [ [ [ [ direction = '+' ] ] ] ] ][ [ [ [ [ ] ] ] ] ][ children = [ [ [ attribute = [ magnitude = [ number = 2 ] ] ] ] ] ][ [ [ *nucleus* = [ [ [ unit = 'inches' ] ] ] ] ] ][ [ [ [ [ ] ] ] ] ][ [ [ [ [ type = 'RelativeVariation' ] ] ] ] ][ [ [ [ ] ] ] ][ [ [ [ period = [ month = 6 ] ] ] ] ][ [ [ [ [ year = 1996 ] ] ] ] ][ [ [ ] ] ][ [ [ *relType* = "'Elaboration'" ] ] ][ [ ] ][ [ [ *msgType* = 'MonthlyTemperatureMsg' ] ] ][ [ [ ] ] ][ [ *nucleus* = [ period = [ month = 6 ] ] ] ][ [ [ [ year = 1996 ] ] ] ][ [ [ ] ] ][ [ [ temperature = [ category = 'hot' ] ] ] ][ [ ] ][ [ *relType* = 'Sequence' ] ][ ][ title = [ text = None ] ][ [ type = None ] ]
WeatherExplainer Result
Roughly:
”This has been a hot month. Average rainfall this month is greater than usual. So far, rainfall is four inches above average.”
ASSESS
Summarization of Evaluative Opinions
An Abstractive Summarization Pipeline
Determine most relevant information
and generate summary
Extract all Information from
input corpus
InputReviews Data Summary
ASSESS Testing
Input: Review sentences tagged with crude-feature
evaluations Crude-Feature to User-Defined-Feature mapping
Simple content selection Group evaluations by UDF Calculate average evaluation Also include info on UDF-parent in hierarchy, number
of evaluations
Example Message
[ *msgType* = 'AverageOpinionMessage' ][ numOpinions = 17 ][ polarity = '-' ][ udf = 'Universal Remote Control' ][ udf_parent = 'Extra Features' ][ valence = 1.1764705882352942 ]
12 messages generated
RulesConjunction(Message('AverageOpinionMessage') M1, Message('AverageOpinionMessage') M2) (M1.udf_parent == M2.udf_parent and M1.polarity == M2.polarity):ConstituentSet(Conjunction,M1,M2):(2,M1.numOpinions+M2.numOpinions)
Contrast(Message('AverageOpinionMessage') M1, Message('AverageOpinionMessage') M2) (M1.udf_parent == M2.udf_parent and M1.polarity != M2.polarity):ConstituentSet(Contrast,M1,M2):(3,M1.numOpinions+M2.numOpinions)
Explanation(Message('AverageOpinionMessage') M1, Message('AverageOpinionMessage') M2) (M1.udf == M2.udf_parent and M1.polarity == M2.polarity):ConstituentSet(Explanation,M1,M2):(5,0)
Explanation(Message('AverageOpinionMessage') M1, ConstituentSet(relType = 'Conjunction', nucleus=Message('AverageOpinionMessage')) M2) (M1.udf == M2.nucleus.udf_parent and M1.polarity == M2.nucleus.polarity):ConstituentSet(DExplanation,M1,M2):(10:0)
Sequence(Message('AverageOpinionMessage')|ConstituentSet() M1, Message('AverageOpinionMessage')|ConstituentSet() M2) ():ConstituentSet(Sequence,M1,M2):(1,0)
ASSESS Result
It works! Evaluation of resulting DocPlan would say more about
Rules and Content Selection than Document Structuring Algorithm
Was able to handle larger number of messages and rules
4 of 5 rules used Still, only one message type used
Future Improvements
Investigate whether this simple framework can be used to develop more “intelligent” rules for more sophisticated domain models
[Carenini 2008] – SEA May require changes to implementation
Complete comprehensive documentation and user-manual
Submit to NLTK
References
Bird, Steven; Ewan Klein; Edward Loper (2009). Natural Language Processing with Python. O'Reilly Media Inc. Print and online.
Carenini, G., Moore, J.D., (2006) Generating and evaluating evaluative arguments. Artificial Intelligence, 170(11): 925-952
Carenini, G., Ng, R., and Pauls, A. (2006) Multi-Document Summarization of Evaluative Text. Proc. of the Conf. of the European Chapter of the Association for Computational Linguistics.
FitzGerald, N. (2009) A Complete Pipeline for Semantic Evaluation Summarization. Unpublished Project ReportLester, J. And Porter, B., (1997). Developing and empirically testing robust explanation generators: the KNIGHT
experiments. Computational Linguistics, 23(1):65-101Mann, W. and Thompson, S. (1988) Rhetorical structure theory: toward a functional theory of text organization. Text 3:
243-281.Marcu, D. (1997) From local to global coherence: A bottom-up approach to test planning. Proceedings of Fourteenth
National Conference on Artificial Intelligence (AAAI-1997), 629- 635.Pitler, Emily et al (2008). Easily Identifiable Discourse Relations. University of Pennsylvania Department of Computer
and Information Science Technical Report No. MS-CIS-08-24.Reiter, E. and Dale, R. (1997) Building applied natural language generation systems. Natural Language Engineering 3
(1): 57-87.Reiter, E., and Robert Dale. Building Natural Language Generation Systems (Studies in Natural Language Processing) .
New York: Cambridge UP, 2000. Print.Young, R.M., Moore, J.D. DPOCL: A principled approach to discourse planning, in: Proceedings of the 7th International
Workshop on Natural Language Generation, Kennebunkport, ME, June 17–21, 1994, pp. 13–20.
Top Related