90961 Accepted manuscript · demands are greater in dictation production tasks. Specifically, they...

Author(s): Daffern, T.L. ; Mackenzie, N.M. ; Hemmings, B.C.

Title: Testing spelling: How does a dictation method measure up to a proofreading and editing format?

Journal: Australian Journal of Language and Literacy

ISSN: 1038-1562 Year: 2017 Pages: 28 - 45

Volume: 40 Issue: 1

Abstract: In response to increasing data-based decision making in schools comes increased responsibility for educators to consider measures of academic achievement in terms of their reliability, validity and practical utility. The focus of this paper is on the assessment of spelling. Among the methods used to assess spelling competence, tasks that require the production of words from dictation, or the proofreading and editing of spelling errors are common. In this study, spelling achievement data from the ...

1

Testing spelling: How does a dictation method measure up to a proofreading and

editing format?

Tessa Daffern

Science, Technology, Engineering and Mathematics Education Research Centre (SERC),

University of Canberra

Noella Maree Mackenzie

Research Institute for Professional Practice, Learning and Education (RIPPLE), Charles Sturt

University

Brian Hemmings

Research Institute for Professional Practice, Learning and Education (RIPPLE), Charles Sturt

University

2

ABSTRACT

In response to increasing data‐based decision making in schools comes increased

responsibility for educators to consider measures of academic achievement in terms of their

reliability, validity and practical utility. The focus of this paper is on the assessment of

spelling. Among the methods used to assess spelling competence, tasks that require the

production of words from dictation, or the proofreading and editing of spelling errors are

common. In this study, spelling achievement data from the National Assessment Program –

Literacy and Numeracy (NAPLAN) Language Conventions Test (a proofreading and editing

based measure) and the Components of Spelling Test (CoST) (a dictation based measure)

were examined. Results of a series of multiple regression analyses (MRAs) were based on a

sample of low‐achieving and high‐achieving spellers from the Australian Capital Territory

(ACT) in Year 3 (n=145), Year 4 (n=117), Year 5 (n=133) and Year 6 (n=117). Findings

indicated significant relationships between scores in the spelling domain of the NAPLAN

Language Conventions Test and the phonological, orthographic and morphological

subscales scores of the CoST. Further, the orthographic subscale of the CoST was generally

the main predictor of NAPLAN spelling across year level. Analysis also demonstrated that

gender was not an influential factor. Implications for assessment and instruction in spelling

are discussed in this paper, and the CoST is offered as a valid, reliable and informative

measure of spelling performance for use in school contexts or future research projects.

3

Introduction

School teachers are held accountable for the development of students’ language and literacy

skills by the education community, policy administrators, parent bodies and the media

(Snyder, 2009). Moreover, the emphasis on accountability and data‐based decision making

in schools has led to an increased need for reliable and sensitive measures of academic

performance (Al Otaiba & Hosp, 2010). Assessment comes in different forms and can serve

several purposes: a) identify strengths and needs in student academic achievement; b)

measure improvements in performance over time; and c) determine the efficacy of

instructional approaches (Kohnen, Nickels, & Castles, 2009; Westwood, 2005).

This paper is concerned with the assessment of Standard English spelling

achievements. Planning and implementing optimal and targeted instruction in spelling is

largely contingent upon adequate spelling assessment (Kohnen et al., 2009). However, it has

been argued that existing measures of spelling performance are “not sufficiently structured

or standardised to provide the reliable, sensitive data that teachers need to plan instruction”

(Al Otaiba & Hosp, 2010, p. 4). Specifically, the study reported here aimed to establish if

there is a relationship between performance as measured by a proofreading and editing

format, and performance based on a dictation format.

Issues with assessing competency in spelling

In Australia, popular spelling assessment tools have broadly been characterised by dictation

formats (see, for example, Westwood, 2005), as well as proof reading and editing formats

(see, for example, Australian Curriculum, Assessment, & Reporting Authority (ACARA),

2016). However, researchers such as Critten, Pine, and Messer (2013) and Willett and

Gardiner (2009) contend that these approaches measure different aspects of spelling. From

4

their analysis of the proofreading and editing format adopted in the annual Australian

National Assessment Program‐Literacy and Numeracy (NAPLAN) to measure spelling

performance of students in Years 3, 5, 7 and 9 since 2008 (ACARA, 2016), Willett and

Gardiner (2009) assert that the format does not accurately measure student knowledge of the

spelling system. They claim that assessments based on production from dictation have

“fewer confounding variables” because students can “focus all their cognitive resources on

the activity of spelling a single word at a time” (Willett & Gardiner, 2009, p. 15). They also

suggest that “students have higher facility rates” when completing dictation tasks, while

proofreading tasks are challenging because of “readability” issues, including difficulty with

the identification of “misspelling cues” (spelling errors) even before any correction is made

(Willett & Gardiner, 2009, p. 15). In contrast, Critten et al. (2013) argue that the cognitive

demands are greater in dictation production tasks. Specifically, they found that the mean

number of words correctly recognised by the children in their study (n=101 aged 4 to 6

years) was higher than the mean number of words correctly produced (Critten et al., 2013, p.

206). Consequently, they assert that “representations may be more advanced for

recognition” tasks than production tasks “even though the type of knowledge required for

both tasks is arguably the same” (p. 202).

Qualitative methods of spelling assessment have also been adopted. For example,

Sharp et al. (2008, p. 213) gathered responses from students’ retrospective self‐reports, by

asking them questions such as ʺHow did you spell _ ?ʺ and ʺWhat did you do to decide on

those letters?”. Coding and analysis of their data resulted in the identification of behaviours

and strategies used when spelling, providing valuable insights into the cognitive processes

that underpin spelling. In addition, spelling assessment has involved analyses of spelling

5

errors observed in free compositional writing (see, for example, Bahr, Silliman, Berninger, &

Dow, 2012). Although observations of compositional writing can yield important insights

regarding the extent to which spelling knowledge may be applied when composing such

texts, an analysis of spelling errors is limited to the words a student chooses to include. As

Kohnen et al. (2009) point out, some students may avoid words that are problematic for

them to spell, or their word choice may be restricted on the basis of topic, genre and targeted

audience. Moreover, students’ level of receptive and expressive vocabulary knowledge may

influence word selection (Bahr, Silliman, & Berninger, 2009). Kohnen et al. (2009) encourage

the use of diagnostic tests to identify strengths and weaknesses in students’ spelling. In

particular, they caution against the sole use of tools that are based on observations of free

writing samples or purely on real word spelling tests, with the assumption that such

measures may lead to an underestimation of spelling difficulties because some students may

have come to remember how to spell certain words that are tested. Kohnen et al. (2009)

instead promote the application of a variety of spelling assessment tools, and in particular,

those that are based on non‐words (pseudo words, or nonsense words). The use of non‐

word measures allows the assessment of various linguistic skills involved in spelling to be

measured and remove the possibility of whole‐word knowledge.

Theoretical considerations

Spelling assessment regimes have traditionally been based on stage or phase theories

of spelling development or have tended to provide a summary of words that are correct and

those that are not (see, for example, Bear, Invernizzi, Templeton, & Johnston, 2012; Ehri,

2005; Gentry, 2012). Given that the types of information gathered by assessment measures

can influence instructional approaches in spelling, it is imperative that they reflect

6

contemporary perspectives of learning to spell. According to Bahr et al. (2012, p. 1587),“it

takes a long time to develop a robust … lexicon that coordinates phonology, orthography

and morphology and supports word‐specific, conventional spelling”. With this in mind,

methods of assessment should seek to capture such linguistic nuances in children’s spelling.

Triple Word Form Theory (TWFT) offers a non‐linear stance of learning to spell, contending

that students are capable of drawing concurrently on phonological, orthographic and

morphological skills from the early years of learning to write (Berninger, Abbott, Nagy, &

Carlisle, 2010; Richards et al., 2006). TWFT has been validated in a series of brain imaging

studies (see for example, Berninger et al., 2010) and behavioural studies (Garcia, Abbott, &

Berninger, 2010; Nagy, Berninger, & Abbott, 2006), offering a potentially innovative and

well‐grounded framework from which to assess proficiency in spelling. According to

TWFT, spelling is an essential word formation process and product of writing, and requires

the coordination of three linguistic codes. These codes have been succinctly defined by

Bahr, Silliman, Danzak, and Wilkinson (2015):

1) The phonological code … functions as an analyser of phonemes in spoken

words;

2) The orthographic code … serves to analyse letters, letter groups, and larger

letter patterns in written words; [and]

3) The morphological code … analyses root words, prefixes, and inflectional

and derivational suffixes in both spoken and written words. (p. 74).

Words with phonologically regular constituents may require encoding of individual

phonemes into their corresponding grapheme units (Garcia et al., 2010). For example, the

individual letters in the word rob can be written by encoding each phoneme using a

7

corresponding letter (r‐o‐b). While many disyllabic and polysyllabic words may be

morphologically complex, they may still contain constituents that are phonologically

regular. For example, when encoding the phonologically regular medial graphemes in the

word recognition (‐gni‐), accurate blending of those three medial phonemes needs to occur at

a syllable juncture. The polysyllabic nature of this word may pose difficulties with

phonological encoding. Simultaneously, application of the rules that govern how affixes

attach to base words or roots (e.g., the derivational suffix, ion, in recognition) may be

required when spelling polysyllabic words. Indeed, as Apel (2014) points out, the

determination of whether or not affixes are correctly applied (e.g., recognition/recognishun)

when spelling is one plausible method of measuring morphological awareness.

Not all words in the English language contain one‐to‐one phonological

correspondence, but they do follow orthographic conventions. Specifically, ‘positional

constraints’ is a term often used to explain how the positioning of a particular phoneme

within a word determines how the phoneme is likely to be orthographically represented

(Bahr, 2015; Holmes & Ng, 1993; Treiman & Kessler, 2006). For example, it is possible to use

vowel doublets in initial, medial or final positions of words, as in eerie, need and tree.

Consonant doublets can plausibly be used in medial positions of words (as in the word,

bottle) and final positions (as in the word, fall) but they are very rarely used in the initial

position of words (as in the word, llama) (Read & Treiman, 2013). Potential orthographic

confusions can also occur when representing the ou versus ow diphthong (as in out or how).

In this instance, the correct spelling choice for the diphthong is determined by specific

positional constraints. That is, the graphemes ow are required if the diphthong is present in

the final position of a word (as in how) or if followed by the grapheme l (as in growl), or a

8

single n (as in town). For the majority of other words constituting the same diphthong, the

graphemes ou are usually needed instead (in words such as ground, stout and couch). The

region of the brain responsible for processing the orthographic word form is thought to be

sensitive to such letter sequences rather than the visual shapes of individual letters (Richards

et al., 2006). Developing heightened orthographic sensitivity requires knowledge of the

‘legal’ (conventional) letter patterns within words (Conrad, Harris, & Williams, 2013).

Critically, while systematic linguistic error analysis of encoded words has the

potential to identify specific breakdowns in spelling (Silliman, Bahr, & Peters, 2006),

assessment systems should seek to encapsulate the phonological, orthographic and

morphological complexities associated with spelling (Bahr, 2015). Assessment in spelling

based on an error analysis of written words should therefore consider utilising words that

are monosyllabic, disyllabic and polysyllabic but also feature common phonological

regularities, legal letters sequences and morphemic complexities.

The study

The present study forms part of a larger mixed‐methods study which compared the

spelling performance of low‐achieving spellers and high‐achieving spellers in Years 3 to 6.

The primary purpose of the study discussed in this paper was to determine whether there is

a relationship between performance as measured by a proofreading and editing method,

and performance as measured by a dictation format. In doing so, we also build on an earlier

study which sought to develop and test the reliability of a dictation‐based spelling

assessment tool informed by TWFT: the Components of Spelling Test (CoST) (Daffern,

Mackenzie, & Hemmings, 2015). A detailed description of the development and initial

9

testing of the CoST, including the theoretical framework that informed the study, as well as

the process of selecting words and items can be found in Daffern et al. (2015).

The study reported here aimed to examine the relationships between two different

measures of spelling performance. In this paper, results are reported in response to the

following research questions:

1. What are the relationships between students’ spelling results, as measured by

NAPLAN spelling and the linguistic components of spelling, as measured by the

CoST, for low‐achieving spellers and high‐achieving spellers?

2. Are these relationships affected by gender and/or year level?

Sampling

Data reported in this paper derive from a sub‐sample of students in Years 3 to 6 (see

Table 1) who participated in a larger mixed‐methods study (n=1,198) in 2013. For the

present study, eight schools from the Australian Capital Territory (ACT) were randomly

selected to participate. Following ethics approval from the researchers’ university Human

Research Ethics Committee and the ACT Catholic school system, informed written consent

was obtained from the participating school principals, teachers, students and their parents.

The sub‐sample represents those students identified as low‐achieving spellers (Year 3, n=71;

Year 4, n=56; Year 5, n=55; Year 6, n=55) and high‐achieving spellers (Year 3, n=74; Year 4,

n=61; Year 5, n=78; Year 6, n=62). These groups of students were identified from within the

larger study as those who performed in the bottom third and top third of the spelling

measure within the NAPLAN Language Conventions Test, respectively.

10

Table 1. Means, standard deviations and age range (in months), for participating students

(Years 3 to 6)

<INSERT TABLE 1 HERE>

Spelling achievement data

As can be seen in Table 2, students’ NAPLAN Language Conventions Test results

from 2012 and 2013 were collected. These data were gathered from school databases or

directly from the parents of the participating student/s if the school did not hold the records.

In addition, the CoST was administered by the principal researcher (first author) to

participating students in October, 2013, in collaboration with the respective school principals

and teachers.

Table 2. Schedule for collection of NAPLAN and CoST data


A proof reading and editing format (The NAPLAN Language Conventions Test: Spelling

domain)

The NAPLAN Language Conventions Test is part of a series of national standardised

tests administered annually in May to all Australian students in Years 3, 5, 7 and 9 (ACARA,

2016). The 40 minute test assesses aspects of student achievement in spelling, grammar and

punctuation. This test requires students to complete visually oriented tasks,

decontextualised from compositional writing processes. Specifically, students identify and

edit spelling errors in words presented either in isolation or within a short phrase, as well as

identify and label some common grammatical and punctuation conventions such as the

correct use of pronouns, conjunctions and verb forms. Student participants’ results from

11

items identified only within the spelling domain of the Language Conventions Test were

utilised for the present study (see Appendix 1 for the 2012 and 2013 NAPLAN spelling

domain items).

A dictation format: The Components of Spelling Test (CoST)

The CoST is a dictation test designed to measure knowledge of the linguistic

components of the Standard English spelling system. Closely aligning with TWFT, the CoST

provides a means from which to interrogate student knowledge of the spelling system

without confining spelling achievement into a specific stage or phase of development.

Strong internal consistency results for this spelling test have been reported, with Cronbach’s

alphas ranging from .78 to .94 (Daffern et al., 2015).

The CoST requires students to write 70 words presented to them orally, each within

the context of a sentence (see Appendix 2); however, the measure comprises 15 constructs

and 101 individual items across three subscales: i) Phonological Component; ii) Orthographic

Component; and iii) Morphological Component. Appendix 3 contains the scoring templates for

the three subscales of the CoST. Appendix 4 includes norms obtained from the sample of

students who participated in the larger study.

What follows is an explanation of the administration and scoring procedures of the

CoST. This information may be useful for educational researchers and practitioners seeking

to utilise the CoST. However, to ensure that integrity to the validity of the tool is

maintained, potential test administrators are urged to carefully consider and follow the

recommended protocols, outlined below. In addition, it is strongly recommended that any

potential re‐testing does not occur within a timeframe of approximately twelve months.

12

Administration protocols for the CoST

In the study, the CoST was administered (by the first author) to groups of students in

regular classroom settings. Testing time was approximately 30 minutes per group. Students

were not given the opportunity to study the 70 target words in advance of testing. Prior to

commencing the dictation test, the administrator orally provided the following instructions

to the student/s:

“I am going to ask you to write 70 words. Some of the words may be easy to spell; some may

be difficult. If you do not know how to spell a word, spell it the best you can. First, I will say the

word, and then I will use the word in a sentence. If you didn’t hear it, listen very carefully because I

will repeat the word one last time. If you really didn’t hear a word, put up your hand and wait for me

to ask you what the problem is.”

The participating students were encouraged to complete the entire test, even if the

target words appeared to be very difficult for them. Potential test administrators should be

mindful that if a student finds the words too difficult, the student should still be encouraged

to write the words. However, if a student is clearly having significant trouble with the more

difficult words and has not attempted more than five words in a row, the student could be

invited to stop. In addition, if a student does not hear a word, the test administrator needs

to make a judgement whether or not to repeat the word. It is important to bear in mind that

too much repetition could disrupt the testing situation and potentially invalidate the results

if inconsistencies arise in the administration of the CoST. In some instances, the

administrator needs to remind the student/s to listen carefully as words would not be

repeated again.

13

For this study, the participating students were required to write their responses on

lined paper, as illustrated in the sample shown in Figure 1. At the conclusion of testing, the

administrator immediately collected the response papers from all participating students.

<INSERT FIGURE 1 HERE>

Figure 1. Sample of a student’s response in the CoST

The sentence prompts that were used by the test administrator are provided in

Appendix 2 for the reader’s convenience. The administrator called each target word

(indicated in italics) aloud, used it in a sentence, and then repeated the target word at the

end of the sentence. Each word was dictated without deliberate or artificial emphasis of any

particular feature, including phoneme or syllable.

Scoring procedure

After collecting the response papers from the students, the administrator analysed

and scored the spelling of each word for each participating student. This scoring process

involved an analysis of phonological, orthographic and morphological errors that a student

may have made in the list of 70 words that were written. For each student, the scoring was

documented on the Scoring Templates (see Appendix 3). As specified in the CoST, some

words contain only one measureable item (target grapheme/s), while other words contain

two or more measurable items. In the Scoring Templates, each item is located in a cell to the

right of each word. The items in the CoST are designed to measure a students’

representation of specific linguistic features within words (rather than the spelling of whole

words). Each item is assessed dichotomously (that is, as correct or incorrect).

14

Mirror reversals of graphemes (letters) were accepted as correct, unless they lead to

other correct graphemes (e.g., rob as rod; speaker as sqeaker). Both upper and lowercase letters

were accepted as correct. If an item contained more than one grapheme, all of the

graphemes were written in the order specified in the respective cell for that item to be

marked as correct. For example, in the word smudged, the item containing the letters dge

should have been spelled exactly as dge for the item to have been marked as correct. Any

other spelling alternations such as gde, dg, or bge were incorrect.

The scoring process began by placing the Phonological Component Scoring Template

beside a student’s response paper. For each word listed on the far left column of the

Phonological Component Scoring Template (see Appendix 3), the scorer (first author) examined

whether the corresponding word part (item) was correctly spelled by the student. The items

which the student correctly spelled were colour highlighted to indicate that those items were

correctly written by the student. For example, if a student spelled tag as tug, the letter a

listed in the corresponding ‘short vowel grapheme’ cell was not highlighted, while the t and

g were highlighted in the corresponding ‘initial and final consonants’ cells to indicate correct

responses. When all items in the Phonological Component Scoring Template were

dichotomously assessed, the sum of correct responses for each linguistic feature (construct)

was computed. Finally, the total Phonological Component raw score was then calculated.

The process described above was repeated for the Orthographic Component and Morphological

Component, using the respective Scoring Templates (see Appendix 3).

Method of analysis

15

All achievement data sets (that is, from the NAPLAN Language Conventions Test

and the CoST) were entered into SPSS (Version 20). Pearson’s bivariate correlation analyses

(Bryman, 2008) were then conducted to establish the relationships between NAPLAN

spelling and the three linguistic components of spelling, as measured by the CoST, for low‐

achieving spellers and high‐achieving spellers. Multiple Regression Analyses (MRAs) were

also performed to determine whether these relationships were affected by gender and/or

year level.

Specifically, for Years 3 and 4, bivariate correlations were carried out to examine the

relationships between the Year 3 NAPLAN spelling scores and the CoST subscale scores for

low‐achieving and high‐achieving spellers. This procedure was repeated for Years 5 and 6,

however, only the students’ Year 5 NAPLAN Language Conventions Test spelling scores

were used for these correlations. Separate bivariate correlation analyses for males and

females were conducted to examine gender differences. MRAs were then used to test if the

three components of spelling, as measured by the CoST, predicted NAPLAN spelling for

low‐achieving and high‐achieving spellers in Years 3 to 6. These were followed by another

series of MRAs that were carried out to see if gender was predictive of NAPLAN spelling for

low‐achieving and high‐achieving students in Years 3 to 6.

Results

Results for low‐achievers and high‐achievers across the four cohorts indicated

significant positive correlations between the NAPLAN spelling results and the three

subscales (components) of spelling, as measured by the CoST (see Table 3). For low‐

achieving spellers across all year levels, the strongest correlation was between NAPLAN

16

spelling and the orthographic subscale, r = .64, p < .001 (Year 3); r = .69, p <.001 (Year 4); r =

.67, p < .001 (Year 5); and r = .64, p <.001 (Year 6). It is worth noting, however, that in the

Year 6 low‐achieving group an equally strong relationship was observed with the

morphological subscale. For the high‐achieving spellers, the strongest relationship was

between the NAPLAN spelling and the morphological subscale, with the exception of Year

5, in which the relationship with the phonological subscale was marginally stronger, r = .51,

p < .001, than the morphological subscale, r = .50, p <.001.

Table 3. Relationships between the NAPLAN spelling scores and the CoST subscale

scores for low‐achievers and high‐achievers (Years 3 to 6)


Separate bivariate correlation analyses for males and females were also conducted to

examine gender differences (see Table 4). A testing of the significance of the CoST subscale

correlations between males and females, using the low‐ and high‐achieving spelling groups,

revealed no significant findings. This testing was based on a two‐tailed t‐test with a

Bonferroni correction of .05/3. It needs to be noted, however, that there were some high

correlations in spite of the truncated NAPLAN spelling scores in both spelling groups. In

addition, a comparison of gender in the low‐achieving and high‐achieving spelling groups

by year level revealed a dominance of females in the high achieving groups; however, there

was no obvious pattern of gender difference in the low‐achieving groups, as can be seen in

Table 4.

Table 4. Correlation analyses for males and females by spelling group


17

MRAs were then used to test if the three components of spelling, as measured by the

CoST, predicted NAPLAN spelling for low‐achieving and high‐achieving spellers in Years 3

to 6. The results of the analyses indicated that the three CoST subscales, across all year

levels, for low‐achieving spellers and high‐achieving spellers were significantly associated

with NAPLAN spelling (see Table 5). For example, for low‐achievers in Year 3 about 41% of

the variance was explained by the CoST, R²adj=.413, F(3,67)=17.43, p<.001; and approximately

30% for high‐achievers, R²adj =.299, F(3,70)=11.40, p<.001.

As indicated in Table 5, the results of the MRAs for the Year 3 cohort revealed that

the orthographic subscale score was the only significant predictor in this model at p=.002; for

the high‐achievers, the morphological subscale score was the only significant predictor at

p<.001. For low‐achieving spellers in Years 4 and 5, the only significant predictor of the

three CoST subscale scores was the orthographic score. For high achieving spellers in Year

5, the orthographic subscale was the strongest predictor, followed by the phonological

subscale. Although no CoST subscale scores independently predicted NAPLAN spelling in

Year 6, the overall model was a good fit, with about 42 percent of the variance in NAPLAN

spelling jointly explained by the CoST for low‐achievers and 24 percent for the high‐

achievers.

Table 5. CoST subscale scores for low‐ and high‐achieving groups as predictors of

NAPLAN spelling (Years 3 to 6)


While significant, these results do suggest, particularly for high‐achieving spellers,

that NAPLAN spelling involves other competencies not measured by the CoST. Further, it

seems reasonable to assume that high‐achieving spellers are equipped with greater linguistic

18

agency than low‐achieving spellers. It also needs to be kept in mind that the spelling

component of the NAPLAN Language Conventions Test is based on a proofreading and

editing task, whereas the CoST uses a dictation format.

Another series of MRAs were carried out to see if gender was predictive of NAPLAN

spelling for low‐achieving and high‐achieving spellers in Years 3 to 6. When gender was

entered in step one (that is, as a control variable) for the MRAs, none of the R² values

reached a significant level (that is p< .05). As a second step, the three CoST subscale scores

for each year level were entered. This second step not only showed that the model for each

year level was significant, but permitted calculation of an R² change value for each MRA (see

Table 6). The R² change was calculated by subtracting the R²adj value (for gender in step one

of the MRA) from the R²adj value (for gender, phonological subscale, orthographic subscale

and the morphological subscale measures in step two of the MRA). In cases where the R²adj

value, for step one, was less than zero, the R²adj value was treated as zero. The recording of

a negative R²adj value occurs when only one variable is entered and this step leads to a very

low value (Tabachnick & Fidell, 2001).

Table 6. R² change values for low‐ and high‐achieving groups (Years 3 to 6) for a two‐step

entry


As can be seen in Table 6, results indicate that for the low‐achieving groups of

students across the year levels, the R² change values were relatively large, while for the

high‐achieving spelling groups the R² change values were somewhat smaller. Further, the

orthographic subscale was generally the main predictor of NAPLAN spelling across year

levels, with the exception of the Year 6 high‐achieving group and both Year 4 groups, where

19

no individual subscale was a significant predictor. In addition to the orthographic subscale,

the phonological subscale was a significant predictor, but only in the Year 5 high‐achieving

group.

Discussion

In a climate where mandatory national testing of student achievement appears to be

influencing educational practices in literacy (Hardy, 2013), there is a pressing need for

classroom teachers to have access to student achievement data that enable them to make

informed decisions regarding instructional approaches and priorities. When used in

isolation, the NAPLAN results do not provide Australian classroom teachers with specific

feedback about the linguistic skills that need to be addressed in order to support student

learning in spelling. It has also been argued that the proofreading and editing format of the

NAPLAN spelling domain may potentially “confuse the picture of students’ spelling

ability” (Willett & Gardiner, 2009, p. 2). To address these issues, we sought to test whether

there is a relationship between the results from the NAPLAN spelling domain and a

dictation‐based measure of the three linguistic constituents in spelling, namely the

phonological, orthographic and morphological components.

Our findings reveal significant relationships between the two testing formats.

Importantly, the results indicate that even though orthographic accuracy, as measured by

the CoST seems to be a particularly strong predictor of NAPLAN spelling, a combination of

accurate phonological, orthographic and morphological representations does indeed

constitute success with spelling. The results also align with the fundamental principle

underpinning TWFT that learning to spell “depends on developing awareness of

20

phonological, orthographic, and morphological word forms … and coordinating them”

(Richards, Berninger, & Fayol, 2009, p. 332).

The results of this study also provide evidence of criterion‐related validity for the

CoST. Specifically, significant relationships between the NAPLAN spelling results and the

three subscale scores of the CoST, for low‐achieving and high‐achieving spellers, across the

four cohorts were found. This suggests that students who perform poorly on a spelling

measure that employs a proofreading and editing format, for example, as measured by the

spelling domain of the NAPLAN Language Conventions Test, may also perform poorly on

spelling tests that utilise a dictation format, such as the CoST. The same pattern of

correlation can be construed for high‐achieving spellers. This finding seems to contradict

Willett and Gardiner (2009), who have questioned the validity of the NAPLAN format and

Critten et al. (2013) who suggest that production tasks are cognitively more demanding and

proofreading/editing tasks. Indeed, the finding initiates an opportunity for future research

aiming to undertake further validity testing of the CoST, particularly by utilising other

measures of phonological, orthographic and morphological representations in spelling. It

needs to be noted that further validity testing was considered for the present study;

however, it was deemed impossible because adequate measures of phonological,

orthographic and morphological representations of real word spelling did not appear to

exist. Indeed, future research should seek to develop parallel or alternative tests for the

CoST to strengthen its validity, but also to increase its utility. Also worthy of future

examination is an analysis of the relationships between the CoST subscales and non‐word

measures of phonology (see, for example, Wagner, Torgesen, Rashotte, & Pearson, 2013),

21

orthography (see, for example, Conrad et al., 2013) and morphology (see, for example,

Nunes, Bryant, & Olsson, 2003).

Another important finding revealed in this study is that for low‐achieving spellers

across all year levels, the strongest correlation was between NAPLAN spelling and the

orthographic subscale; an equally strong relationship was observed with the morphological

subscale for the Year 6 low‐achieving spellers. This finding builds on the work of Conrad et

al. (2013), who demonstrated that orthographic knowledge, at age seven to nine years,

contributes to spelling “over and above the contributions of phonological skills” (p. 1223). It

also supports research by Rothe et al. (2014) who provided evidence that orthographic

knowledge predicts spelling in German‐speaking Kindergarten students. While these two

studies did not include a morphological measure, the present study did, and the results

showed that for the high‐achieving spellers, the strongest relationship was between the

NAPLAN spelling measure and the morphological subscale, with the exception of Year 5, in

which the relationship with the phonological subscale was marginally stronger than the

morphological subscale. In addition, a strong relationship was observed with the

morphological subscale for the Year 6 low‐achieving spellers.

While gender differences were examined by testing the significance of the CoST

subscale correlations between males and females, using the low‐ and high‐achieving spelling

groups, no significant findings were revealed. Critically, the results suggest that the

students who performed poorly in the NAPLAN spelling retained their status as poor

spellers up to 18 months later, as measured by the CoST, and that gender was not an

influential factor. Similarly, high‐achieving spellers (as measured by NAPLAN spelling)

remained as high‐achieving spellers (as measured by the CoST) during the same time

22

period. Resonating with this finding is the stability of spelling performance reported in

Abbott et al.’s (2010) longitudinal research in spelling acquisition. This highlights the need

to systematically utilise assessment outcomes in order to guide instruction, and in particular

to provide early and effective intervention in spelling for those students experiencing

difficulties with spelling. Specifically, teachers could use the CoST, in combination with

other forms of assessment, to help them ascertain the linguistic skills which do require

instructional attention in order to improve spelling achievements.

While differences were found between low‐achieving spellers and high‐achieving

spellers, the correlations between the two test results do not provide causal evidence.

However, the findings support the view that heightened sensitivity to sub‐lexical

orthographic regularities plays an important role in learning to proofread and edit spelling

errors, but also in learning to spell more broadly (Richards et al., 2009; Rothe et al., 2014). It

is not surprising that the orthographic subscale of the CoST appears to be such a strong

predictor of NAPLAN spelling, considering that the latter is essentially a visual‐spatial

exercise that involves scanning word/s, and identifying and correcting orthographic

anomalies. Nevertheless, future research is needed to conclusively determine why there

may be differences in error types across spelling abilities.

Concluding remarks

The generalisability of the findings in the study is limited to schools across the ACT.

It also needs to be acknowledged that the ACT broadly represents one of the highest

performing jurisdictions in Australia in terms of academic achievement in school, as

measured by NAPLAN. Further, the sample size for this study poses some limitations to the

findings. Specifically, hierarchical regression analyses were limited by the truncated scores,

23

particularly in the Year 6 cohort. As such, replication of this study involving other

jurisdictions is welcomed. It may also be intriguing to explore whether or not similar

correlations demonstrated in the present study are to be found, with the advent of ‘online’

NAPLAN testing, as opposed to paper and pen modes of delivery.

One single assessment tool cannot provide the complete answer to a child’s

knowledge about the spelling system and it should be accompanied by other methods of

assessment. Indeed, being able to understand the extent to which children can explain and

justify their spelling is insightful. Although the findings reported in this paper offer

evidence that the CoST is an informative and valid assessment tool, we assert that multiple

forms of assessment should still be integrated into an instructional program. These may

include linguistic analyses of dictated non‐words, but also spelling errors recorded in

students’ written compositions. In addition, conducting open‐ended interviews with

students, or including questionnaires and/or surveys as part of assessment regimes in

classroom contexts are also very valuable.

Appendices

<INSERT APPENDIX 1 HERE>



24

References

Abbott, R., Berninger, V., & Fayol, M. (2010). Longitudinal relationships to levels of

language in writing and between writing and reading in grades 1 to 7. Journal of

Educational Psychology, 102(2), 281‐291. doi:10.1037/a0019318

Al Otaiba, S., & Hosp, J. (2010). Spell it out: The need for detailed spelling assessment to

inform instruction. Assessment for Effective Intervention, 36(1), 3‐6.

Apel, K. (2014). A comprehensive definition of morphological awareness: Implications for

assessment. Topics in Language Disorders, 34(3), 197‐209.

doi:10.1097/TLD.0000000000000019

Australian Curriculum, Assessment, & Reporting Authority (ACARA). (2016). NAP national

assessment program. Retrieved from http://www.nap.edu.au/about/about.html

Bahr, R. (2015). Spelling strategies and word formation processes. In R. Bahr & E. Silliman

(Eds.), Routledge handbook of communication disorders (pp. 193‐203). London:

Routledge.

Bahr, R., Silliman, E., & Berninger, V. (2009). What spelling errors have to tell about

vocabulary learning. In C. Woods & V. Connelly (Eds.), Contemporary perspectives on

reading and writing (pp. 109‐129). New York: Routledge.

Bahr, R., Silliman, E., Berninger, V., & Dow, M. (2012). Linguistic pattern analysis of

misspellings of typically developing writers in grades 1‐9. Journal of Speech, Language,

and Hearing Research, 55, 1587‐1599.

Bahr, R., Silliman, E., Danzak, R., & Wilkinson, L. (2015). Bilingual spelling patterns in

middle school: It is more than transfer. International Journal of Bilingual Education and

Bilingualism, 18(1), 73‐91. doi:10.1080/13670050.2013.878304

Bear, D. R., Invernizzi, M., Templeton, S., & Johnston, F. (2012). Words their way: Word study

for phonics, vocabulary, and spelling instruction (5th ed.). Upper Saddle River, NJ:

Pearson Education, Inc.

Berninger, V., Abbott, R., Nagy, W., & Carlisle, J. (2010). Growth in phonological,

orthographic, and morphological awareness in grades 1 to 6. Journal of

Psycholinguistic Research, 39(2), 141‐163. doi:10.1007/s10936‐009‐9130‐6

Bryman, A. (2008). Social research methods (3rd ed.). New York: Oxford University Press, Inc.

Conrad, N., Harris, N., & Williams, J. (2013). Individual differences in children’s literacy

development: The contribution of orthographic knowledge. Reading and Writing,

26(8), 1223‐1239. doi:10.1007/s11145‐012‐9415‐2

Critten, S., Pine, K., & Messer, D. (2013). Revealing childrenʹs implicit spelling

representations. British Journal of Developmental Psychology, 31(2), 198‐211.

doi:10.1111/bjdp.12000

Daffern, T., Mackenzie, N. M., & Hemmings, B. (2015). The development of a spelling

assessment tool informed by Triple Word Form Theory. Australian Journal of Language

& Literacy, 38(2), 72‐82.

Ehri, L. C. (2005). Learning to read words: Theory, findings, and issues. Scientific Studies of

Reading, 9(2), 167‐188. doi:10.1207/s1532799xssr0902_4

Garcia, N., Abbott, R., & Berninger, V. (2010). Predicting poor, average, and superior spellers

in grades 1 to 6 from phonological, orthographic, and morphological, spelling, or

reading composites. Written Language & Literacy, 13(1), 61‐98.

25

Gentry, J. R. (2012). An analysis of developmental spelling in GNYS AT WRK. In D. Wyse

(Ed.), Literacy teaching and education (Vol. 3, pp. 347‐357). London: SAGE Publications.

Hardy, I. (2013). Testing that counts: Contesting national literacy assessment policy in

complex schooling settings. Australian Journal of Language & Literacy, 36(2), 67‐77.

Holmes, V., & Ng, E. (1993). Word‐specific knowledge, word‐recognition strategies, and

spelling ability. Journal of Research in Reading, 31, 136‐156.

Kohnen, S., Nickels, L., & Castles, A. (2009). Assessing spelling skills and strategies: A

critique of available resources. Australian Journal of Learning Difficulties, 14(1), 113‐150.

Nagy, W., Berninger, V., & Abbott, R. (2006). Contributions of morphology beyond

phonology to literacy outcomes of upper elementary and middle‐school students.

Journal of Educational Psychology, 98(1), 134‐147.

Nunes, T., Bryant, P., & Olsson, J. (2003). Learning morphological and phonological spelling

rules: An intervention study. Scientific Studies of Reading, 7(3), 289‐307.

doi:10.1207/s1532799xssr0703_6

Read, C., & Treiman, R. (2013). Childrenʹs invented spelling: What we have learned in forty

years. In Piattelli‐Palmarini & R. C. Berwick (Eds.), Rich languages from poor inputs

(pp. 197‐211). New York: Oxford University Press.

Richards, T., Aylward, E., Field, K., Grimme, A., Raskind, W., Richards, A., . . . Berninger, V.

(2006). Converging evidence for triple word form theory in children with dyslexia.

Developmental Neuropsychology, 30(1), 547‐589. doi:10.1207/s15326942dn3001_3

Richards, T., Berninger, V., & Fayol, M. (2009). fMRI activation differences between 11‐year‐

old good and poor spellers’ access in working memory to temporary and long‐term

orthographic representations. Journal of Neurolinguistics, 22(4), 327‐353.

Rothe, J., Schulte‐Körne, G., & Ise, E. (2014). Does sensitivity to orthographic regularities

influence reading and spelling acquisition? A 1‐year prospective study. Reading and

Writing, 27(7), 1141‐1161. doi:10.1007/s11145‐013‐9479‐7

Sharp, A. C., Sinatra, G. M., & Reynolds, R. E. (2008). The development of childrenʹs

orthographic knowledge: A microgenetic perspective. Reading Research Quarterly,

43(3), 206‐226.

Silliman, E., Bahr, R., & Peters, M. (2006). Spelling patterns in preadolescents with atypical

language skills: Phonological, morphological, and orthographic factors.

Developmental Neuropsychology, 29(1), 93‐123. doi:10.1207/s15326942dn2901_6

Snyder, I. (2009). The stories that divide us: Media (mis)representations of literacy

education. English in Australia, 44(1), 13‐23.

Tabachnick, B. G., & Fidell, L. S. (2001). Using multivariate statistics (4th ed.). Boston, MA:

Allyn and Bacon.

Treiman, R., & Kessler, B. (2006). Spelling as statistical learning: Using consonantal context

to spell vowels. Journal of Educational Psychology, 98(3), 642‐652.

Wagner, R. K., Torgesen, J. K., Rashotte, C. A., & Pearson, N. A. (2013). Comprehensive Test of

Phonological Processing (2nd ed.). Austin, TX: Pro Ed.

Westwood, P. (2005). Spelling: Approaches to teaching and assessment. Camberwell, Vic: ACER

Press.

Willett, L., & Gardiner, A. (2009). Testing spelling: Exploring NAPLAN. Paper presented at the

Australian Literacy Educatorsʹ Association Conference.

27

Tessa Daffern is an Assistant Professor in Teacher Education at the University of Canberra.

She has also been a member of the Research Institute for Professional Practice, Learning and

Education, and a Subject Coordinator in the Master of Education at Charles Sturt University.

She is an accredited provider of professional learning with the Teacher Quality Institute, in

the Australian Capital Territory, and regularly engages in professional work with school

teachers.

Email: [email protected]

Noella Maree Mackenzie is a Senior Lecturer in literacy studies at Charles Sturt University.

Noella’s research has focused on the teaching and learning of writing and teacher PL. Her

research informs, and is informed by, her ongoing professional work with teachers in

schools and her university teaching. Noella has been recognised for teaching excellence. Her

work has been published in professional (e.g. Practical Literacy) and research journals (e.g.

Australian Educational Researcher).


Brian Hemmings is currently the Sub‐Dean (Graduate Studies) and Deputy Director,

Research Institute for Professional Practice, Learning and Education (RIPPLE) at Charles

Sturt University. He has published widely and his most recent publications appear in

Professional Development in Education and the Australian Journal of Teacher Education.


90961 Accepted manuscript · demands are greater in dictation production tasks. Specifically, they...

Documents

Transcript of 90961 Accepted manuscript · demands are greater in dictation production tasks. Specifically, they...