Questions Arising From the Assessment of EFL Narrative Writing

7/23/2019 Questions Arising From the Assessment of EFL Narrative Writing

1/10

Questions arising from theassessment of EFL narrative writing

Yong Yi

This article questions how narrative writing is assessed, seeking to understandwhat we test, what we value, and why. It uses a single anomalous case thatarose in the course of my recent PhD thesis to highlight the issues, asking

if sufficient attention is being given to the value of emotional content in apiece of writing in comparison to its technical merit. In doing so, the natureand purpose of teaching narrative writing as a subject in EFL is questioned.Given the diversity of English-speaking cultures, has too little attentionbeen given to the importance of narrative writing as a key subject area? Isthere unrealized potential for integrating narrative writing into languageacquisition? A new dialogue between academics and practitioners is called

for to develop new approaches in understanding and enhancing our conceptsof narrative writing.

Introduction I had been a well-regarded English teacher in a university in Chinabefore I spent a year in Australia as a visiting scholar, followed by a totalof five years studying for MA and PhD degrees in the United Kingdom.However, I found that I was not well prepared for the English languageI encountered on my travels. This was a curiosity at first. I realizedthat this was not simply what is often referred to as cross-culturalcommunication issues; there was more to it than simply learning aboutthe customs and mores of another people group.

What I believe I was experiencing was that, despite using the same

language framework, communication and understanding was difficultbecause we told different stories in different ways. I began to reflecton the nature of English teaching in China and how well it serves ourstudents, especially if they are to be employed in English speakingcountries or work with native English speakers in China or elsewherein the globalized world. It seemed to me that we needed to considerrevising our approach away from focusing on technical forms ofEnglish towards a more pragmatic, evolved, and relevant engagementwith the language as it is used in different cultures that use Englishnatively. The majority of everyday communication basically consists of

engagement with an ongoing narrative: we are story-bound creatures, asexpressed so well by Hardy (1968: 5):

The Author 2012. Published by Oxford University Press; all rights reserved.ELTJournal Volume 67/1 January 2013; doi:10.1093/elt/ccs062

Advance Access publication November 20, 2012

70

atUniversidadAutnomadeTlaxcalaonMarch23,2015

http://eltj.oxfordjournals.org/

Downloadedfrom
http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/


2/10

For we dream in narrative, day-dream in narrative, remember,anticipate, hope, despair, believe, doubt, plan, revise, criticize,construct, gossip, learn, hate, and love by narrative. In order reallyto live, we make up stories about ourselves and others, about thepersonal as well as the social, past and future.

Hardys reminder that narrative is an indispensable part of our daily lifeled me to speculate that it perhaps should be a more important elementin EFL teaching and learning than it currently is.

The curious case ofNumber 20

The perspective I outline stemmed from issues that I encountered inmy doctoral research (Yi 2011) concerning the effects of visualizationtraining on intermediate Chinese EFL learners narrative writing. Inthis research, I involved two groups of participants: the Experimentalgroup which was given developmental training in narrative writing withspecific training in visualization, and the Control group who receiveddevelopmental training without the visualization elements.

To assess students pre- and post-test written narratives, four assessorswere engaged: two native speakers of English (D1 and D2) and twonative speakers of Chinese (Q and Y) who taught English in a Chineseuniversity. They were asked to rate the narratives by applying thefollowing five-element criteria:

reader involvement imagery provoking/stimulating communicative effectiveness fluency

intrinsic value.

These criteria were not conventional, as I was attempting to finda way to differentiate writing quality in line with criteria related todevelopment through visualization techniques.

The results proved inconclusive in terms of differentiating the testnarratives. At one level, therefore, this test section was devoid of usefuldata. There was, however, one piece of work in the post-test set for theExperimental group that presented a hugely anomalous result (seeAppendix A). All the assessors marked it as the highest in the whole

tranche, and the score was significantly higher than this participantreceived in her pre-test piece (see Appendix B). As has been noted inother situations throughout history, it is the anomaly, the exception,that has provided people with the stimulus to explore more deeply.Thus, the post-test piece by Number 20 became one of the mostcurious and stimulating outcomes of my research.

In order to explore the assessors responses to this piece, I needed toidentify the issues at stake. There were multiple variables in the matrix;most significantly and obviously I wondered if the result stemmed fromproblems in the rating criteria, the assessors use of the criteria, or was

it simply a response to the nature of the topic? Thus, I formulated twofollow-up questions for the assessors to discover more about how theyconducted their assessment of Number 20s contributions:

Questions arising from the assessment of EFL narrative writing 71



Downloadedfrom


3/10

1 In what way(s) is the second piece better than the first? Pleasecomment.

2 Please feel free to add any other comment that you consider may beof value.

I purposely kept the questions very open so that the assessors could saywhatever they had in mind when recording their thoughts about theprocess. This was mainly to explore the assessors internal processes:their subjective reality. I was not seeking to justify or critique this butto see where it led me in terms of developing my conceptual thinkingabout narrative writing. The following section is a summary of theassessors responses.

Responses tofollow-up questions

Assessor D1 emphasized the fact that the technical use of words stillhas to be good enough not to detract from the story, but that they[the words] are very much secondary in importance to the intrinsicpower and value of the story itself. He placed strong emphasis on the

relationship between writer and reader in co-creating the story and itsmeaning through the power of shared and individual imagery:

I found Number 20s first piece to be quite visual and vaguelyinteresting, nominally engaging, but it was not fluent and full ofgrating language errors. The emotional content and intrinsic valuedid not raise itself up high enough to compensate for the frustrationin reading between the lines that was necessary to follow the story.In contrast, the second piece, objectively, could be said to be littlemore fluent or imagery-stimulating, but its emotional content wasdeep, sincere and raw and the power of the story transcended thetechnical aspects, drawing the reader in and almost making one gaspwith the intrinsic value and emotionality of the piece: it was not justgood, it was searingly, heart-rendingly beautiful. I had to fight totry and find a place to be objective and mark down the fluency andcommunicative effectiveness, as I just wanted to give it full marksfor making me feel so much.

Assessor D2 questioned whether Number 20 had someone in thefamily background with a fair competence in English, raising thequestion of how exposure to competent English usage outside the

learning environment affects storytelling ability.

It is not easy to pin down five criteria for assessing a short piece ofwriting. They cant be mutually exclusive, for certain. The criteriareader involving and communicative effectiveness are both good,but tend to overlap. Imagery provoking is difficult, as some studentstend to concentrate on emotions without much reference to thingsseen and heard. Would reactions or inner feelings cover theground? Fluency is a good requirement to sort out the A-studentsfrom the rest. Command of a natural English style is needed,

allowing the reader undivided attention.Assessor Qs responses were written in Chinese and the following is aclose translation, which was confirmed by Q as being accurate.

72 Yong Yi



Downloadedfrom


4/10


5/10

childrens approach to writing narrative, I found that their emphasison communication was relevant and useful for my study. They made astrong case for communication being the core aim of narrative writing,associated with the transmission of meaning, emphasizing that theyfelt a need to upset the applecart of traditional notions about writingassessment, where convention is more important than communication(ibid.: 68). This supports the general direction of my belief that

assessment of narrative writing needs to focus on meaning, context,and effective communication over simple reliance upon linguisticcompetence.

After my work was completed, I discovered the High Plains RegionalTechnology in Education Consortiums (2010) six-trait model designedto contribute to the quality of narrative writing through an assessmentrubric, which supports the Gearhart and Wolf approach. This comprisesfocus on topic (ideas and content); sequencing (organization); addingpersonality (voice); word choice; sentence structure (fluency); andgrammar and spelling (conventions). Within this, for instance, theelement ideas and content states that effective and appropriate detailscreate a vivid picture showing knowledge and insight, which attemptsto blend holistic and analytical concerns. Similarly, with the criterionvoice, the rubric incorporates such elements as

the writers personality is expressed; confidence and feeling areapparent;

individual, powerful commitment to the topic is obvious; connection to audience and purpose is excellent; writing evokes strong emotion in the reader.

The High Plains assessment rubric thus differs from others in that, inaddition to conventional areas such as grammar and spelling, it involvesthe writers personality, feelings, and connection to the audience.The inclusion of rating criteria concerning the stimulation of readersemotions encourages the expression and assessment of communicativecompetence in broader ways that meet more effectively the demands ofa twenty-first century globalized world.

Discussion In my wide experience of rating students writing, no matter whatrubrics or criteria for assessment are designed beforehand, assessorsappear to be more inclined to mark according to their holistic, overallresponse to the writing. According to my observations and dialoguewith teaching colleagues, it is the same throughout the process ofassessing students writing. Sometimes teachers/assessors might evenwork backwards, using the framework in such a way as to justify themarks awarded according to their original, intuitive, or experiencedresponses. In order to take account of this phenomenon, it is necessaryto address the issue as it is, not as we might like it to be or assumeit has been. The time assessors have to report is also a factor andmight predispose towards holistic assessment, citing experience

as the justification. The assessment of narrative writing has to meetjustifiable criteria for consistency in order to ensure the students workis assessed fairly, but the qualitative nature of the assessment needs

74 Yong Yi



Downloadedfrom


6/10

acknowledgement. The development of an appropriate scheme needsto be considered anew. Good narrative tends to appeal to criteria thatare not easily defined, but are inherently subjective, and thus resistantto categorization. Phenomenologically, assessors derive a significantproportion of their assessment using their instinct, responding totheir heart, especially in the face of an emotionally powerful story. Butis this kind of assessment reliable? In the learning and development

context, is it desirable? Is the emphasis upon effect right to hold overtechnique?

In the context of these questions, it seems that a methodologyrepresenting a new approach needs to be developed, based on apragmatic reflection of the need for engagement with the skill of tellingstories that really mean something and telling them well. This is thereason for my wanting to develop narrative writing in EFL specifically.For me, storytelling is the universal form of communication thatpromotes contextual understanding of a language in dynamic use and

thus it becomes a potential force in language acquisition, not justan exercise in technical competence. Would a new form of dialoguebetween teaching practitioners and research academics be a usefuldevelopment?

To illustrate this, I re-present the following assessor comments made inresponse to Number 20s work:

the reader can feel the true love through the language (Assessor Y); grammatical mistakes did not affect the meaning or the conveyance

of emotions (Assessor Q);

the power of the story transcended the technical aspects (AssessorD1).

These comments appear to support the approach engendered by theGearhart and Wolf and High Plains Consortium rubrics, as previouslydescribed. By its very nature, narrative writing is subjective. Itspurpose is to appeal to readers emotions and sensitivities; to conveyinformation of course, but not to just do that. Therefore, my questionsare about the evolving nature of narrative and just what we are seekingfrom it. In the past, the tendency was for a technical approach, suchas that reported in Zamel (1982), which points out that many studies

had sought to prove the efficacy of one grammar over another (thusperpetuating the belief that a better pedagogical approach, particularlyone that focused on usage, structure, or correct form, would improvewriting), concluding that this approach, while relevant, was inadequate.

Johnson and VanBrackle (2012) show that previous research hasexamined reactions by raters to errors in narrative writing pieces. Theyreported that Connors and Lunsford (1988) conducted an historicalstudy on rater reactions to errors. They concluded that educators inthe early twentieth century were more concerned about errors thancontemporary educators due to the predominance of process writing

pedagogies. Paradoxically, they found that while contemporary educatorsclaim to be less error driven, they, in fact, mark errors more than theycomment on other aspects of essays (for example development or




Downloadedfrom


7/10

content); I speculate that this is because it is easier to mark in this wayunder time pressures, but I need to test this view through engagementwith others. Most significantly, Connors and Lunsford contend thatdefinitions of errors and views of what constitutes a severe error changeover time and that teachers are influenced by current trends. There is,in other words, no absolute standard for the judgment of writing. Thisagain emphasizes the questions concerning what it is that we seek

and why; this is a dynamic, fluid field of work and, one may argue,a field that is changing more rapidly now than ever before. Whetherthe advancement of new criteria will help to address the concernsabout meaningless subjectivity for the objectivists or the concernsof subjectivists about false objectivity or not, may depend on thepotential for gaining agreement on progress towards large-scale practicalapplication. But to do this, we return once again to the question of justwhat narrative writing is hoping to achieve in an EFL context.

Concluding thoughts The curious case of Number 20 indicated that grammatical accuracy

does not necessarily affect the impact of narrative writing on readers.Although it was a singular, anomalous result acquired tangentially fromother work, it points to findings and discussions elsewhere and opens avalid area of enquiry.

Rarely will researchers want to take much account of single instancesif we assume a general empirical approach to development, but it is mycontention that we may gain much from the anomalous if we learn toask the right questions. My focus is on analysing what happened andwhy, and what it might mean in terms of liberating the deeper potential

inherent in effective narratives to express meaningful, emotionallyintelligent writing, in balance with legitimate requirements forgrammatical accuracy and appropriate vocabulary, as expressed clearlyin another of Assessor D1s comments:

narrative writing is evidence of the power of a story to transcend itscomponent writing elements and calls into question purely analyticalapproaches. The value of a narrative lies in its ability to communicatemeaning holistically; and that the meaning should be a blend of whatthe writer found meaningful and what the writer wants to offer tostimulate the readers own meaning.

In commenting on what to teach and how to assess, D1 adds:

Thus the job of the teacher is to teach effective, engaging, sociallyvaluable communication with meaning: not to produce an assemblyline of perfect grammar that means nothing much.

Is the job of the assessor to value the right elements using head andheart, engaged emotionally and tuned to hear the story, and not just toappreciate the apostrophe rule and correctly hanging indents? I think so,but it would be useful for researchers and practitioners engaged in the

work of teaching and assessing EFL narrative writing to share practice.Without any methodological restrictions, it would be useful to generatea wide discussion on, for instance, how narrative writing is valued for

76 Yong Yi



Downloadedfrom


8/10

the significant contribution it makes to literacy and language acquisitionand how we can develop teaching and assessment to address a dynamic,relevant, and culturally appropriate understanding of the place ofnarrative. This focus on stimulating engagement and conversationbetween academics and practitioners is to fundamentally address thequestion of how we can best prepare our students to interact withconfidence in the diversity of English-speaking environments.

Final revised version received August 2012

ReferencesConnors, R.and A. Lunsford.1988. Frequency offormal errors in current college writing, or Maand Pa Kettle do research. College Composition andCommunication39/4: 395409.Gearhart, M.and S.A. Wolf.1994. Engagingteachers in assessment of their students narrativewriting: the role of subject matter knowledge.

Assessing Writing1/1: 6790.Hamp-Lyons, L.1995. Research on the ratingprocess. Rating non-native writing: the troublewith holistic scoring. TESOL Journal29/4:75962.Hardy, B.1968. Toward a poetics of fiction.NOVEL: A Forum on Fiction2/1: 514.High Plains Regional Technology in EducationConsortium. 2010. 6 trait writing model: narrativewriting rubric - hero story. Available at http://www.fm.coe.uh.edu/resources/language_arts/narrrubric.pdf(accessed on 10 October 2011).Huot, B.1996. Toward a new theory ofwriting assessment. College Composition andCommunication47/4: 54966.Johnson, D.and L. VanBrackle.2012. Linguisticdiscrimination in writing assessment: how ratersreact to African American errors, ESL errors,and standard English errors on a state-mandatedwriting exam. Assessing Writing17/1: 3554.

Khalil, A.1989. A study of cohesion andcoherence in Arab EFL college students writing.System17/3: 35971.Yancey, K.B.1999. Looking back as we lookforward: historicizing writing assessment.College Composition and Communication50/3:483503.Yi, Y.2011. The effects of visualization training

on intermediate Chinese EFL learners narrativewriting. Unpublished PhD thesis, LeedsMetropolitan University, UK.Zamel, V.1982. Writing: the process ofdiscovering meaning. TESOL Quarterly16/2:195209.

The authorYong Yihas been involved in EFL teaching formany years in Qufu Normal University, China.She holds a PhD in ELT from Leeds MetropolitanUniversity, UK. Her research interests includeEFL teaching, second language acquisition, andusing visualization to develop language skills.She is also currently involved in a large-scaleproject comparing translations of the Analects ofConfucius.Email:[email protected]

Appendix ANumber 20s post-test writing

Five years ago I lost the most close friend all my life. My dear sister, alovely, diligent, clever and kind-hearted girl, left us and never got back.Thats disaster of the whole family.

In those days, tears just running down whenever I saw the bed we shared,the books she had read, the dolls we played and the pictures she haddrawn. I couldnt figure out why she left me all of a sudden. I couldntaccept it. So did my mother. She seemed to be older and heart broken.

But I couldnt understand my father. He just kept doing all the thingsin the house and arrange all the things left by the disaster. He talked

little to us. He just kept doing all the time. Didnt he love his littledaughter? No, that cant be true. Im sure. But why he seemed to be soindifferent?




Downloadedfrom
http://www.fm.coe.uh.edu/resources/language_arts/narrrubric.pdfhttp://www.fm.coe.uh.edu/resources/language_arts/narrrubric.pdfhttp://www.fm.coe.uh.edu/resources/language_arts/narrrubric.pdfmailto:[email protected]://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/http://eltj.oxfordjournals.org/mailto:[email protected]://www.fm.coe.uh.edu/resources/language_arts/narrrubric.pdfhttp://www.fm.coe.uh.edu/resources/language_arts/narrrubric.pdfhttp://www.fm.coe.uh.edu/resources/language_arts/narrrubric.pdf


9/10

Then about one month later. It was a dark dusk, mother asked me tofind father and had our supper. I walked to our new house, I thoughthe should be working there. When I was about 50 meters from the doorof our new house, a gloomy sound, which sounded like the roar of atiger, came to me. I was just scared. I didnt know what it was. I hadnever heard of a sound like that before. I found that the sound seemedto be controlled not to let others hear it.

I just stopped my steps, I didnt know what was going on in the house.But my father must be in the house. So I calmed myself down andwalked very gently into door. I saw a man sitting in the corner. Therewas no light I couldnt see him clearly. But I was sure he was my father.He was crying with his head burying into his arms. I couldnt believe it.

He didnt want us to know it. I mustnt let him know what I saw himcrying. So I gently walked out and when I was about 50 meter away.I called, Dad, where are you? Time for supper.

After a little while, he answered, OK, Ill be back right away.

After that answer, tears came out again. I just cant help myself.

When he got back I saw gentle smile on his face. I just smiled back.But I was crying deep in my heart!

His precious tears just taught me what true love is! Dear dad, I love you.

Appendix BNumber 20s pre-test

writing

Twenty-one years has passed since I was born into the colourful world.I have experienced so many happy days. But that day was the happiest

day in my life.It was July 26th. I planned to come back to school by train fromBeijing. Another boy who was expressing his true love to me wanted tosend me back. I had tried to refuse his advice. But he had tried all theways to let me know how much he loved me.

That night, I was in the train ahead of the time. But he was sloweddown by the heavy traffic. He sent me news that he almost cried out.He had done a lot for it, for that day. But I dont know anything. I toldhim not to worry. If he couldnt catch the train, he could go back home.He wouldnt lose anything. Thats better for him.

However, he said that he wouldnt give up. I didnt know how he got onthe train at last, But he did.

What I didnt know at all was that it was July 7th in Chinese old way ofshowing time. He brought me 99 roses with him. There was so manypeople that the way was full of people. You couldnt get out to the W.C.If you want to move, you have to ask a lot of people to move their feet.There was no more space for your feet.

He was not in the same railway carriage. He must get through all the

people in the carriages. I couldnt imagine how he could do it. I was notsure how many people would try to stop him. He didnt have to do that.

He sent me the news with only a few words: JUST WAIT FOR ME.

78 Yong Yi



Downloadedfrom


10/10

Then about one hour later. He brought the roses to me. My eyes wereall wet. I didnt know how to express my feeling. But I was movedcompletely. All the people around smiled at us. That moment I knewI was the happiest people in the world. That day, July 7th was thehappiest day in my life.




Downloadedfrom

Questions Arising From the Assessment of EFL Narrative Writing

Documents

Transcript of Questions Arising From the Assessment of EFL Narrative Writing