Writing Studies and Innovation in Designing
NLP Educational Applications:
A Multidisciplinary PerspectiveWorkshop 3: 9th Workshop on Innovative Use of NLP
for Building Educational Applications (BEA)Norbert Elliot
Professor of EnglishJune 26, 2014
Outline
• Part 1: Automated Writing Evaluation—Perspectives from Rhetoric and Composition/Writing Studies
• Part 2: Case Study, 2010 to 2014• Part 3: Directions for Multidisciplinary,
Collaborative Research
Part 1: Automated Writing Evaluation—Perspectives from Rhetoric and Composition/Writing Studies
What is the field of Writing Studies?What is the relationship between Writing
Studies and AWE?
Writing Studies: A Chronology
• Modern Language Association Founded: 1883
• National Council of Teachers of English Founded: 1911
• Conference on College Composition and Communication Founded: 1948
• The Visibility Project: 2004-2010– 23 English Language and
Literature/Letters– 23.13 Rhetoric and
Composition/Writing Studies– 23.1301 Writing, General– 23.1302 Creative Writing– 23.1303 Professional,
Technical, Business and Scientific Writing
– 23.1304 Rhetoric and Composition
– 23.1399 Rhetoric and Composition/Writing Studies, Other
Writing Studies: Traditional Beliefs
• Aristotle (335 BCE)– “Rhetoric may be
defined as the faculty of observing in any given case the available means of persuasion. This is not a function of any other art.” (Rhetoric, Book 1, Chapter 2, p. 24)
• Roberta Camp (1992)– Writing is “a rich,
multifaceted, meaning-making activity that occurs over time and in a social context, an activity that varies with purpose, situation, and audience and is improved by reflection on the written product and on the strategies used in creating it.” (p. 135)
Writing Studies: Emerging Beliefs
• White, Elliot, Peckham (forthcoming)– writing within a digital
environment is an activity that emphasizes audiences within and beyond the classroom, an activity that resonates with the writer’s use of image and voice and improves through collaboration and experimentation.
• Council of Writing Program Administrators, National Council of Teachers of English, and National Writing Project (2011)– Habits of mind– Writing, reading, and
critical analysis experiences
Writing Studies: Construct Span
Communication in Digital, Print, Blurred Environments
Writing and Reading, Visualization, and Speaking
Rhetorical Conceptualization
Cognitive Domain: Genre, Task, Audience, Writing Process, Problem Solving, Information Literacy, Conventions, Metacognition
Interpersonal Domain: Collaboration, Social Networking, Leadership, Diversity, Ethics
Intrapersonal Domain: Openess, Conscientiousness, Extraversion, Agreeableness, Stability
Neurological, Attention, and Visual Domain
Digression: Multidisciplinary Research• Fish (1989)
– “The blurring of existing authoritative disciplinary lines and boundaries will only create new lines and new authorities; the interdisciplinary impulse finally does not liberate us from the narrow confides of academic ghettos to something more capacious; it merely redomiciles us in enclosures that do not advertise themselves as such” (237).
• Elliot & Klobucar (2013)– Importance of
multidisciplinary research– CIP Codes: Teams for
Multidisciplinary AWE Research
• 11: Computer and Information Sciences
• 13: Education• 16: Foreign Languages,
Literatures, and Linguistics• 23: English Language and
Literature• 27: Mathematics and
Statistics• 42: Psychology• 52: Business, Management,
and Marketing
Determining Directions for Future Research
• Not in our control– Interpretations
inherent in technological determinism
• Within our control– Patterns of
adoption associated with technological transfer
Part 2: Case Study, 2010 to 2014
What work has been done to bridge the gap between Writing Studies and AWE?
Klobucar, Deane, Elliot, Ramineni, Dees, Rudniy (2012)
Portfolio and course grade: r = .43Combined AWE score and portfolio relationship: r = .39Combined AWE score and course grade: r = .29
Klobucar, Elliot, Dees, Rudniy, Joshi (2013)
Statistically significant relationships
Variation in group predictions
Burstein & Elliot, Holtzman, Lentini, Molloy, Shore, Steinberg, & Vezzu, 2014
Commonality = black Disjuncture = red (lower)/green (higher)
Two-proportion Z-test used to evaluate statistically significance (p < .000)
Alvarez, Biswal, Elliot, Klobucar (in process, IRB F 183-13); ETS partnership
• Subject: A neural basis of writing
• Aim: Examination of modes of discourse scored by e-rater®
• Sampling plan: High performing writers and emerging writers
• Analysis: Resting-state functional MRI (R-fMRI)
Part 3: Directions for Multidisciplinary, Collaborative Research
What might the future hold for multidisciplinary collaboration?
Future Direction 1: A Multidisciplinary Program of Research• Identify consequence as a
primary category of evidence• Distinguish between mode
(exposition, description, narration, and argumentation [EDNA]) and aim
• Reconsider hierarchy of the variables of writing ability
• Engage in robust construct modeling, with machines and humans serving designated roles based on known assessment limits.
Top Related