Ştefan Trăuşan-Matu
University Politehnica of Bucharest
Computer Science Department
Polyphonic Analysis of Discourse in Texts and
in Collaborative Learning Chats
Contents
The Polyphonic Model
Polyphonic Analysis
Implementations of the Polyphonic Analysis
The PolyCAFe Analysis System
Other Applications
Conclusions
12/14/2013 2 Stefan Trausan-Matu
Polyphony- An Unitary Model of
Human Communication
Mediation:
Using natural language (words) in
texts
hypertexts
discussion forums
conversations
but also
Non verbal communication (e.g.
gestures)
12/14/2013 3 Stefan Trausan-Matu
Polyphony- An Unitary Model of
Human Group Communication
Considering rather a dialogistic, post-
structuralist (Bakhtin, Kristeva) than a
mechanistic perspective on communication
Rather a socio-cultural (Vygotsky) approach
than a cognitivist one (like in Artificial
Intelligence) but considering the both
Ethnomethodology (Garfinkel), Conversation
Analysis (Sacks, Schegloff, Jefferson)
12/14/2013 4 Stefan Trausan-Matu
Polyphony- An Unitary Model of
Group Communication
12/14/2013 5 Stefan Trausan-Matu
Applicable to:
Small groups (e.g. virtual teams
collaborating by chat or forums – the
INTER-ANIMATION phenomenon
appears)
Large groups – social networks
Global level - intertextuality
Polyphony- An Unitary Model of
Group Communication and
Intertextuality
It may be used in IT implementations
using
Natural language Processing
Machine learning
Social Network analysis
Specific techniques (inter-animation
and collaboration analysis)
12/14/2013 6 Stefan Trausan-Matu
The Polyphonic Model
12/14/2013 7 Stefan Trausan-Matu
Polyphony
Appears in music (e.g. J.S.Bach) and in texts (Bakhtin)
The Polyphonic
Model (Trausan-Matu, Handbook of Hybrid Learning, 2010)
Analysis method (Trausan-Matu and Rebedea, 2010)
Computer support tools for the polyphonic analysis of F2F, online and offline conversations: The “Polyphony” system (Trausan-Matu and all, 2007)
ASAP (Dascalu, Chioasca and Trausan-Matu, 2008)
PolyCAFe (Trausan-Matu, Rebedea and Dascalu, 2011; Rebedea, Dascalu, Trausan-Matu and all, 2010)
Collaboration regions detection (Banica, Trausan-Matu and Rebedea, 2011)
Detection of the Important moments (Chiru and Trausan-Matu, 2012)
12/14/2013 8 Stefan Trausan-Matu
Polyphony
A group of participants that, each of them
keeps their individuality, personality, creativity,
but also collaborate to achieve a common
goal, trying to solve dissonances
A merge of:
Unity vs. Difference
Melody (longitudinal) and Harmony (transversal)
Dissonance – Consonance Cycles –
centrifugal/centripetal forces
Inter-animation of voices – inter-animation
patterns 12/14/2013 9 Stefan Trausan-Matu
The Polyphonic Model Polyphony = Model of collaboration and interaction
(Trausan-Matu, Stahl and Zemel, 2005)
Human communication in knowledge construction and collaboration are processes in which words and other utterances are linked in parallel threads which interact similarly to voices in polyphonic music
Repetition and rhythm are essential
12/14/2013 10 Stefan Trausan-Matu
12/14/2013 11 Stefan Trausan-Matu
Polyphony
12/14/2013 12 Stefan Trausan-Matu
Dialogism and Polyphony (Bakhtin) Mikhail Bakhtin:
• Utterances (not sentences) should be the unit of analysis
• “These are different voices singing variously on a single theme. This is indeed 'multivoicedness,' exposing the diversity of life and the great complexity of human experience. 'Everything in life is counterpoint, that is, opposition,' “ (Bakhtin, 1984)
• “… Any true understanding is dialogic in nature” (Voloshinov-Bakhtin, 1973)
• Speech genres
• Polyphony Inter-animation of voices
• Basis for the CSCL paradigm (Koschman, 1999)
• Opposed to de Saussure ideas: • Real life dialog should be the focus, not written text
• Words are not arbitrary
12/14/2013 13 Stefan Trausan-Matu
Bakhtin’s Polyphony
Everything is a dialog (applying not only to speech and text)
Utterances
Voices Inter-animations among voices
12/14/2013 14 Stefan Trausan-Matu
Utterances Utterances (not sentences, as in ‘classical’ linguistics) should
be the unit of analysis (Bakhtin) Utterances are acts An utterances may be a:
Word Turn, a reply in a conversation, chat or forum Sentence Text Image (picture, diagramatic representation, etc.) Gesture (individual or group) Thought – inner utterances – inner speech
Utterances should be considered at different granularities
Utterances are linked in threads formed by: Explicit links (VMT chat environment; forum’s replies) - uptakes
(Suthers, 2010) Implicit links, detected by Natural Language Processing techniques
– contingencies, uptakes (Suthers, 2010)
Utterances may become voices
12/14/2013 15 Stefan Trausan-Matu
Voices Distinctive presences in a group, influencing the
other voices
Generated by utterances (singular or repeated) Correspond to:
participants (may also be inner voices) groups of participants (e.g. collective or collaborative
utterances) chains or threads of words or concepts: repeated words lexical chains co-references reasoning or argumentation rhetorical schemas
Each utterance may contain multiple voices Voices continue and influence each other through
explicit or implicit links.
12/14/2013 16 Stefan Trausan-Matu
Inter-animation patterns
Longitudinal
Adjacency pairs
Repetitions
Elaboration
Convergence
Cumulative talk
Repair
Transversal, differential
Dissonance
12/14/2013 17 Stefan Trausan-Matu
The Polyphonic Method applications
Chat conversations with multiple participants for:
CSCL: K-12 students solving mathematics problems both individually and collaboratively in the VMT project at Drexel University, Philadelphia, US
CS students at University Politehnica of Bucharest , Romania at
o CHI course in Romanian and French – role playing and debate
o Natural Language Processing - role playing and debate
o Algorithm Design – problem solving
Fostering creativity – brainstorming, synectics
F2F collaborative learning (Suthers & all, 2011)
Analysis of Rhythm
Metacognition (conversation & essays)
OpenSimDeveloper dataset
Intertextuality 12/14/2013 18 Stefan Trausan-Matu
Computer Support for the
Polyphonic Analysis
12/14/2013 19 Stefan Trausan-Matu
LTfLL - EU FP7 Project (2008-2011) and
NSF Virtual Math Teams Project http://www.ltfll-project.org/ http://mathforum.org
Language Technologies for Lifelong Learning Netherlands, France, United Kingdom, Germany, Ausria, Romania,
Bulgaria
PolyCAFe system (Polyphony-based Collaboration
Analysis and Feedback generation)
The system has been validated with students and tutors in
University of Manchester, UK
Politehnica University of Bucharest, Romania
12/14/2013 20 Stefan Trausan-Matu
Chat-based CSCL
K-12 students solving mathematics problems both individually and collaboratively in the Virtual Math Teams (VMT) project at Drexel University, Philadelphia, US (Directed by Gerry Stahl)
Computer Science students at University Politehnica of Bucharest (UPB), Romania at Human-Computer Interaction course in Romanian and
French – role playing and debate Natural Language Processing - role playing and debate Algorithm Design – problem solving
12/14/2013 21 Stefan Trausan-Matu
Theories for analysing
multi-parties conversation
Discourse analysis (Tannen)
Conversation analysis (Sacks, Jefferson, Schegloff)
Accountable talk (Resnick)
Transactivity (Teasley, Berkowitz & Gibbs, Joshi &
Rose)
Events/contingencies, coordinations/uptakes
(Suthers)
Inter-animation (Bakhtin, Wegerif, Trausan-Matu)
Polyphony (Bakhtin, Trausan-Matu)
12/14/2013 22 Stefan Trausan-Matu
Analysis methods
TF-IDF
Latent Semantic Analysis
Naïve Bayes
Social Network Analysis
WordNet (wordnet.princeton.edu)
Support Vector Machines
Collin’s perceptron
TagHelper environment
12/14/2013 23 Stefan Trausan-Matu
Analyis methods TF-IDF
Latent Semantic Analysis Almost all are based also on
Naïve Bayes a two interlocutors
Social Network Analysis model, in which
WordNet (wordnet.princeton.edu) one person speaks
Support Vector Machines at a time, resulting
Collin’s perceptron one discussion thread
TagHelper environment
12/14/2013 24 Stefan Trausan-Matu
12/14/2013 25 Stefan Trausan-Matu
12/14/2013 26 Stefan Trausan-Matu
NLP pipe
spelling correction, stemmer, tokenizer, Named Entity
Recognizer, POS tagger and parser, and NP-chunker.
Stanford NLP software
(http://nlp.stanford.edu/software)
Spellchecker : Jazzy
http://www.ibm.com/developerworks/java/library/j-
jazzy/
Alternative NLP pipes are under development,
GATE (http://gate.ac.uk)
LingPipe (http://aliasi.com/lingpipe/).
12/14/2013 27 Stefan Trausan-Matu
Social network analysis
Consider explicit and implicit referencing as arcs
between participants, which are the nodes
A kind of page-rank algorithm – an utterance is
important if it is referred by important utterances;
The strength of a voice (of an utterance)
depends on the strength of the utterances that
refer to it
Determines if a person is central/peripheral
12/14/2013 28 Stefan Trausan-Matu
Polyphony, Inter-animation and
Collaboration analysis
Assign an importance value for each utterance
considering several indicators of inter-animation
(collaboration) Detection of voices (chains) inter-animation patterns
(Trausan-Matu) in the chat
Consider several criteria such as the presence in the
chat of questions, agreement, disagreement
Presence of others’ voices
Social Networks metrics
Machine learning approach (genetic algorithms and
neural networks) for tuning the 12/14/2013 29 Stefan Trausan-Matu
Computational details (Trausan-Matu, Dascalu and Dessus, ITS 2012;
Dascalu, Trausan-Matu and all, 2010, 2011))
12/14/2013 30 Stefan Trausan-Matu
Representations:
Conversation graph
For each participant there is a separate horizontal line in the representation
Each utterance is placed in the line corresponding to the issuer of that utterance, according to the emission time, alligned from left to right The explicit references among utterances are
depicted using connecting lines distinctively colored
The implicit references (deduced by the system) are represented using other color that the explicit ones
An estimation of the strength of each utterance (when available) is represented as a bar chart
12/14/2013 31 Stefan Trausan-Matu
Representations:
Weaving of Voices
Voices in the conversation graph
Participants = horizontal lines
Threads of repeated words or phrases = differently
colored threads
12/14/2013 32 Stefan Trausan-Matu
12/14/2013 33 Stefan Trausan-Matu
12/14/2013 34 Stefan Trausan-Matu
12/14/2013 35 Stefan Trausan-Matu
12/14/2013 36 Stefan Trausan-Matu
37 12/14/2013 37 Stefan Trausan-Matu
38 12/14/2013 38 Stefan Trausan-Matu
39 12/14/2013 39 Stefan Trausan-Matu
40 12/14/2013 40 Stefan Trausan-Matu
41 12/14/2013 41 Stefan Trausan-Matu
Validation (Rebedea, Dascalu, Trausan-Matu and all, ECTEL 2010;
Rebedea, Dascalu, Trausan-Matu and all, ECTEL 2011)
12/14/2013 42 Stefan Trausan-Matu
Other applications
12/14/2013 43 Stefan Trausan-Matu
Analysis Dimensions (types of voices)
in Face-to-Face Settings (Trausan-Matu, in Suthers & all, (eds.) 2013)
Spoken dialog
Body language
Individual
Collective
The visual dimension
Visual data on the blackboard
What others participants do
Others’ body language
Internal dialogue (at an intra-mental level)
Echoes
12/14/2013 44 Stefan Trausan-Matu
Inner utterances, inner speech
“Mead (1934) called thought a <<conversation
with the generalized other,>> implying that when
we think individually we attempt to respond-
internally and vicariously-to the imagined
responses of others to our ideas and arguments.”
(Resnick & all, 1993)
“There are no ontological differences between
inner and outer speech” (Clark and Holquist,
1984).
12/14/2013 45 Stefan Trausan-Matu
Detecting important moments (Chiru and Trausan-Matu, ITS 2012)
12/14/2013 46 Stefan Trausan-Matu
Graphical representation of topic’s
rhythmicity (Chiru, Cojocaru, Trausan-Matu and Rebedea, ISMIS 2011)
High
rhythmicity for
all topics –
these were
debated in
parallel as it
can be seen by
the lack of flat
lines near the
left side of the
representation.
Low rhythmicity
– flat lines on
the left side of
the graphic
showing that
the topic that
they represent
has not been
debated in
those parts of
the chat. 12/14/2013 47 Stefan Trausan-Matu
Determination of collaborative
regions (Banica, Trausan-Matu and Rebedea, CSCL 2011)
12/14/2013 48 Stefan Trausan-Matu
Metacognition (Trausan-Matu, Dascalu and Dessus, ITS 2012)
Combining Chats and Texts
Intertextuality
12/14/2013 49 Stefan Trausan-Matu
ReaderBench (Dascalu, Trausan-Matu, Dessus, 2012)
12/14/2013 50 Stefan Trausan-Matu
Creativity fostering
(e.g. brainstorming)(Trausan-Matu, 2011)
12/14/2013 51 Stefan Trausan-Matu
Opinion mining (Musat, Velcin, Rizoiu, and Trausan-Matu,
2011)
12/14/2013 52 Stefan Trausan-Matu
Topic Modeling (Musat and Trausan-Matu, 2011)
No generally accepted definition for a “topic” Document clusters Abstractions based on document clusters
Labels; Centroids, etc
(Word, Probability) pairs
Bayesian statistical models Topics – distributions over words Documents – distributions over topics Generative model Topic Intertwining Conceptually similar to the ideas of Mikhail Bakhtin
Topics and voices
12/14/2013 53 Stefan Trausan-Matu
Topic Modeling (Musat and Trausan-Matu, 2011)
LDA/pLSA/hLDA/CTM
Each newer version corrects some flaws of the earlier
ones
However the traditional means of testing the accuracy
have been proven wrong
Even more reason to look into the problem of evaluating
the models
LDA
Readily available
Mallet
Easily reproducible experiments
Well known topic model;
12/14/2013 54 Stefan Trausan-Matu
Intertextuality analysis (Ghiban & Trausan-Matu, 2012)
Voice I
Voice II
Voice III
Voice I
Voice II
Voice III
In dialog
Text 1 Text 2 Text 3
Text 4
Text 1
Text 2
Text 3
In dialog in text 4
12/14/2013 55 Stefan Trausan-Matu
Intertextuality analysis (Ghiban & Trausan-Matu, 2012)
Theme 2 and Theme
3 may have the
same words but only
different concepts
Section 1 and 6 are dialogical or
polyphonical. They may present a
higher force of expresivity. 12/14/2013 56 Stefan Trausan-Matu
Analysis of interethnic discourse
(Trausan-Matu, 2012)
Needed a corpus of texts with time stamps
Extract recurrent concepts in texts
Identify historical events
Generate time series
Analysis of correlations between time series
Analysis of the polyphonic structure
12/14/2013 57 Stefan Trausan-Matu
Time Series Analysis of News (Badea & Trausan-Matu, 2013)
12/14/2013 58 Stefan Trausan-Matu
Music Composition at K-Teams Laboratory (Master and Bachelor Thesis coordinated by Prof. Trausan-Matu)
Genetic Algorithms
Celular automata
Artificial chemistry
Constraint-based systems
Accompaniments generation with Markov Models
Random generation
Automatic counterpoint generation according to Fux rules
Chat sonification
12/14/2013 59 Stefan Trausan-Matu
Chat sonification (Stefan Trausan-Matu and Alexandru Calinescu)
12/14/2013 60 Stefan Trausan-Matu
Chat sonification
12/14/2013 61 Stefan Trausan-Matu
12/14/2013 62 Stefan Trausan-Matu
Chat sonification
(Orchestration by Serban Nichifor)
12/14/2013 63 Stefan Trausan-Matu
Conclusions
The polyphonic model may apply to non-verbal
collaboration and intra-subjective (inner thinking)
as well as inter-subjective levels
A combination of Conversational Analysis with
Natural Language Processing is possible
(cognitive and socio-cultural)
Learning analytics tools that combine the two
perspectives and the Polyphonic Model may be
developed
12/14/2013 64 Stefan Trausan-Matu
Thank you!
http://www.racai.ro/~trausan
Questions?
12/14/2013 65 Stefan Trausan-Matu
Top Related