Interpreting NAPLAN results and setting targets€¦ · 3. EMSAD Draft at Nov 2011 USING SMART DATA...

1. EMSAD Draft at Nov 2011

Interpreting NAPLAN results and setting targets

Vers

ion

: 7 N

ov

em

be

r 2

011


INTRODUCTION The purpose of this document is to assist the principal and school self evaluation committee to interpret their NAPLAN results using SMART data analysis tools. It also offers suggestions on how to write achievable targets for literacy and numeracy based on NAPLAN data.


USING SMART DATA ANALYSIS Three SMART data analysis tools:

o Percentage in Bands,

o Student Growth, and

o Means & Standard Deviations,

will assist schools in developing targets based around improving the growth data for students. When setting targets it is important to remember that each year group is not retested until two years later. Each year group will be unique but the school improvement should be focused on improvement for individual student results.

Growth data is used by different groups within the school:

Executive/Teacher Principal • How do our results compare with the state?

• How do our results compare with the region? • What are our strengths in teaching? • What areas do we need to focus on? • Can we see the impact of changed teaching practice?

Principal / School Self Evaluation team • Where do we need to focus our teaching to improve student learning outcomes? • What needs to be done to equal or better state growth? • What professional learning and resources need to be allocated to address our targets? • Are our standards and expectations high enough to address students from all backgrounds? • Are we setting SMART targets around improving our growth?

Percentage in Bands

Trend Data

Student Growth

Means and standard

deviation


Using Percentage in Bands for target setting

The percentage in bands analysis provides detailed information on student groups and their performance relative to the state, region, and SEG in specific performance bands. There are three years worth of data displayed at once, which allows the school to see if there have been changes in the number of students achieving in the various performance bands. The information in the percentage in bands analysis can be of importance for schools that are performing below the state mean. The graphs may show improved school performance through the movement of students from lower bands into higher bands. As an example, the graph below shows increased percentages in the top two bands over three years (51% in 2008; 62% in 2009; 67% in 2010. Targets can be based around the percentage of student performance in bands. This is applicable in schools larger than ten students in the testing group. Schools can set targets around increasing student performance into higher bands. Where this is used, the increases should be a band or more as a target for Year 5 to Year 7 and Year 7 to Year 9 and two bands or more for Year 3 to Year 5. The following target was written after the school below considered the graph and the target setting tool:

Increase the proportion of Year 3 students achieving in the proficiency bands for reading from 62% in 2010 to 65% in 2011.

The target setting tool helped the school decide on a feasible target for the coming year. Note: Very small schools needs to be careful as a single student may be worth a high percentage. E.g. With four students each student is worth 25% (See section on small schools.)


Using Trend Data / Means and Standard Deviations for target setting Note: Both these analysis tools display information about the school mean. Because the mean is a measure of the middle, half of all schools in the state will be below the mean. In NSW there is approximately 1600 government primary and 400 secondary schools. If your school is below the mean it does not indicate it is a failure. Where schools are of concern is when they are below the state mean and when the average growth for students is not at state or better. It is important to note that schools can have improved growth above the state average growth for a year level. While this is an improvement for individual students as it indicates the school has taken them from lower band to a higher one, it could be that students are still operating well below what they are capable.

When analysing mean, consider the size of the group. In small groups the mean can be affected by a number of either high or low achieving students. If your school has a small group of students, other data sources should be analysed to develop a better understanding of actual student and school achievement. The use of growth is more appropriate for small schools. A large standard deviation indicates a wide range of scores and a small standard deviation indicates a cluster around the mean score. When comparing means, what is considered an appreciable difference?


To answer this, look at how far the schools mean sits from the state mean, in terms of the proportion of the state standard deviation. As a rule, if the school’s mean lies within 0.2 standard deviations of the state mean, then there is no appreciable difference. Beyond 0.2, the effect may be large enough to warrant an examination of contributing factors. In this numeracy example: The difference between the school and state mean is: 412.4 – 394.7 = 17.7 When 17.7 is divided by the state mean 88.7 the result is 0.19. Since this result is within 0.2 of the state standard deviation there is not a difference worthy of further consideration. This information should be considered in the school context and other school – based assessment data. To easily and quickly analyse the means and standard deviations data, use the NAPLAN Analysis Spreadsheet which can be accessed from the EMSAD website.

https://portalsrvs.det.nsw.edu.au/f5-w-68747470733a2f2f6465747777772e6465742e6e73772e6564752e6175$$/directorates/schoimpro/EMD/npa/pubs/SSE/NAPLAN%20ANALYSIS%20TOOL.xlsx The Analysis spreadsheet will calculate and show your results using a range of five colours – dark green / green (well above and above state average), yellow (within state average) and red / dark red (problem and severe problem). At a glance you will be able to determine which aspect needs further investigation.

Using Student Growth for target setting Data from this section is particularly important to all schools. Schools that are not achieving the state mean may still be providing the environment for students to achieve the expected growth in reading and numeracy. Conversely a school that may be at or above the state mean may not have appropriate growth occurring for their students. Growth on the NAPLAN scales varies depending on prior scores and because of this, a measure based on the percentage of students achieving expected growth is more useful for diagnostic and school planning purposes than a measure based on average growth alone. In

https://portalsrvs.det.nsw.edu.au/f5-w-68747470733a2f2f6465747777772e6465742e6e73772e6564752e6175$$/directorates/schoimpro/EMD/npa/pubs/SSE/NAPLAN%20ANALYSIS%20TOOL.xlsx




general, there is a tendency for expected growth to be higher when you start with a low prior score than from a higher prior score. In using the growth data in SMART, it is important to recognise that:

most students have a growth figure in a relatively small range around the state

average

the 2010 expected growth value thresholds should be considered interim until

sufficient data are available over time to provide confidence in the measure .

If the box with the displayed unmatched students is ticked, students who were not previously at the school for the last testing period will be indicated on the graph. Growth Charts There are corresponding growth charts that display student growth for the different aspects. These graphs provide a wealth of information on student performance and school performance. Further information is provided in a tabular form below the graph. Average Scaled Score Growth: There is a table at the bottom left that displays the average scaled score growth achieved by the students who were tested. It is separated into state, region and school growth and is also indicated for the whole group, and various selected groups. Schools need to be aware of their group size as small group results may not be valid, particularly if under 10 students.


The SSG is where a group of 5 or more selected schools have been used to compare results. Percentile Ranges: The expected minimum growth in scores that is expected for the testing interval (approximately two years apart) is the percentage of school students who did not reach the minimum state growth. • Less than 25th indicates the percentage of school matched students in the bottom 25% of the state for growth • 25th – 75th indicates the percentage of school students in the middle 50% of the state, this is also known as the inter-quartile range where half of all students score. • 75th and above indicates the percentage of the school students in the top 25% of the state. The example below indicates that too many students are in the lowest percentile, less in the middle and less in the top. Targets could be set to address this imbalance and to ensure the school performance matches the expected, or is better with a decrease in the number for the lowest percentile and higher in the middle and the top.

Percentile Ranges School Result

Less than 25th 42.9 (Should be 25 or less)

25th – 75th 35.7(Should be 50 or less)

75th and above 21.4 (Should be 25 or higher) In the example below there are no students in the lowest percentile and more in the middle and the top indicating the students have been progressed.

Percentile Ranges School Result

< 25th 0

25th – 75th 70

75th and above 30 Expected Growth: The amount next to the blue wording is the percentage of school students who did not reach the expected state growth. The amount next to the green wording is the percentage of school students who reached or exceeded the state growth. Schools with negative growth Schools will often mention that students who are often operating at the top end in performance on the testing program can easily decrease on testing two years later if they make a few mistakes. This was often the case in the Basic Skills Test where a student at the top in Year 3 could easily show negative growth by making a few mistakes in Year 5. While this is true, it is no excuse for student underperformance for those students not at the top end where the majority of students are located. Schools need to look deeply into teaching performance, the learning environments and individual students where the data shows negative growth or there is an increase of students into the lower performance bands.


Small School Targets Small schools (less than ten students in the testing Year level) have to be very cautious in using the statistical data provided from the NAPLAN. Means and standard deviations are not necessarily relevant and can be greatly influenced by individual student performance. Growth is often more applicable for schools when setting improvement targets as it is more individual specific. Using percentages in the student performance in bands is problematic as an individual student is worth a large percentage. In a school with five students being tested, each student is worth 20%. Thus setting targets of increasing student performance by bands would have to be in 20% groups at least. For example having an increase in the top band by 5% is meaningless as it would have to be at least 20% to represent the student numbers for a school of five students. If one student leaves the school the target percentages would be no longer applicable. Targets that are more appropriate focus on students will be achieving the appropriate minimum growth for the Year group being tested. Thus targets such as; students will achieve growth equivalent or better to state level, are more appropriate to schools with less than 10 students. Because school plans are public domain documents and small schools have few students it may be possible for people to identify the individual performance of students. In these cases it is better not to specify the baseline in a document such as the school plan. Baseline data should be kept separately for discussions with the School Education Director.

QUANTIFYING AND MONITORING TARGETS

Extract from the School planning guidelines

3.4.6 Monitoring of the school plan focuses on progress towards the intended

outcomes of the three year plan through the achievement of annual targets.

3.4.7 School education directors can assist principals and school communities in

identifying and setting targets. Principals also have access to a statistical tool

that provides school staff with past student performance data and can support

the development of literacy and numeracy targets based on this data.

When deciding on “how much” to set as the goal for improvement it is useful to consider your baseline data. The following indicate graphical interpretations of achievable target trajectories from SMART data. It is reasonable to assume that state and regional trends will be relatively stable, thus in the examples below we have drawn state and regional trajectories on that basis.


Data reported in SMART

Graphical example

Means

Improving the overall mean

Percentage in Bands

Improving the percentage in the top band

Growth

Improving the expected growth

The target setting tool uses the Percentage in Bands methodology above to provide advice about the NSW state targets for reading and numeracy. It also can be used in assisting schools to set targets for improved performance on National Assessment Program – Literacy and Numeracy (NAPLAN). It should be used once the school has set its school identified priority areas and outcomes. A significant benefit of the target setting tool is that it allows target setting in term of movement of whole students across the achievement bands. As well, there is a means to allow schools to adjust for local knowledge of the student group’s history and capacity to achieve the levels of performance that other student groups have achieved over time. The tool can be used to monitor achievement of targets over time.

0%

20%

40%

60%

80%

100%

2008 2009 2010 2011 2012 2013

% less than expected growth

% greater than or equal to expectedgrowth

Assumed state and regional trajectory

Achievable trajectory for school

Targets for three years

Assumed state and regional trajectory

Targets for three years

Achievable trajectory for school


The target setting tools for your school, preloaded with your schools NAPLAN data, are downloadable from the ASR Access tool


EXAMPLE 1 A large primary school in regional NSW

(Demonstration school 13)

Year 2010 Year 5

Aspect Numeracy Percentages in bands

Band School State Region Did not meet minimum standard (Bottom band for Year Group)

6 4 -

Above minimum standard (Middle three bands)

78 64 -

Proficient (Top two bands for Year Group)

16 32 -

Focus Questions Results How is school performance in the bottom band? How is school performance in the middle bands? How is school performance in the top 2 bands?

How big is the tested group?

A higher percentage of all students are below the national minimum standard (Band 3 - school 6%, state 4%)

A higher percentage of students are in the middle bands (Bands 4, 5, 6 - school 78%, state 64%)

A lower percentage of students are in the proficient bands (Band 7 & 8 -school 16%, state 32%)


9% of boys are performing below the

national minimum standard in Band 3 compared with state - 4%

3% of girls are performing below the

national minimum standard in Band 3 compared to state at 4%

73% of boys are performing in the middle bands - 4, 5, 6 compared with state at 59%

84% of girls are performing in the middle bands 4, 5, 6 compared with state at 68%

19% of boys are performing in the proficient bands compared with state at 35%

13% of girls are performing in the proficient bands compared with state at 28%

Mean and Standard Deviation Data

Aspect Numeracy Year 5

State Mean 499.5 Standard Deviation

79.7

Region Mean 467.4 Standard Deviation

71.8

School Mean 466.3 Standard Deviation

65.8


Focus Questions Results Is data within state parameters? Is data of concern? Is data of serious concern? How big is the tested group?

[Use calculation spreadsheet to determine]

From this mean data the school performance in this aspect is worthy of investigation. (The NAPLAN Analysis Tool indicates that there is a problem.)

Data from Student Growth chart


Average Scaled Score Growth

Category State Region School

All 89.11 79.4 79.0

Boys 90.58 78.5 74.2

Girls 87.6 80.5 85.2

Focus Questions Results Is school data above state average growth? Is any particular group doing better?


The school has a lower average growth than the state but comparable to region. Boys’ growth is well below the state and region average. Girl’s growth is just under state average and above region growth.

Percentile Ranges

Percentile Ranges School Results

<25th 31.25%

25th – 75th 53.13%

75th and above 15.63%

Focus Questions Result Is school data 25, 50, 25? Is school data more towards 25-75 and 75+?


More students fall in the <25th percentile range.

Less in the 75th or above range.

39.02% of boys are in the less than 25th percentile

31.25% of girls are in the less than 25th percentile

48.78% of boys are in the 25th – 75th percentile

53.13% of girls are in the 25th – 75th percentile

12.2% of boys are in the 75th and above

15.63% of girls are in the 75th and above percentile

Expected growth

Less than expected growth 46.9%

Greater than or equal to expected growth

53.1%


Focus Questions Result Is school data higher in >=?

How big is the tested group? Slightly more than half of all students are achieving equal to or expected growth. Almost half of all students have not achieved expected growth. Boys with less than expected growth – 53.7% Girls with less than expected growth – 46.9% Boys with greater than or equal to expected growth – 46.3% Girls with greater than or equal to expected growth – 53.1%

Target Setting

Target based on Advice on writing target Percentages in student bands Target is to reduce students in the below minimum band and increase in the proficient band

Target can be specified in number of students reduced in the below minimum band or increased in the proficient band. This can be against the region or state

School Mean and Standard Deviation From the calculation spreadsheet based on data being with 0.2 or 0.5 Standard Deviations, indicating if data is within state parameters, of concern or of serious concern Targets should set increase to move school from: • serious concern to of concern • of concern to within state parameters • within state to equal or better than state

Target should show an increase, specified in marks, necessary to change from the various levels [use calculation spreadsheet to alter school mean] Eg: does the school need 8 marks to change from serious concern to of concern?

School Average Growth Target should be equal to and higher than state average All groups should be performing at state average or better

Can be specified as a number but is better written as equal to or better state average growth

Percentile Ranges Target should be better than the 25, 50, 25. Targets should favour the middle or higher percentiles

Target can be specified in reduction of students in the lower percentiles and increase in the higher percentiles

Minimum Growth Target is based on 100% achieving minimum growth Target should focus on more students at or above state minimum growth

Target can be specified in more of the group at the minimum growth level


Targets

From this data there are a number of possible targets:

Decrease the percentage of students who do not meet national minimum standards in numeracy from 6% in 2010 to y% in 2011.

Increase the percentage of students achieving in the proficiency bands for numeracy from 16% in 2010 to y% in 2011.

Raise the schools’ average growth in numeracy (particularly boys’) to be equal to or above state average growth by 2011.

By 2011, increase the percentage of students achieving in the ‘75th or above percentile range’ by x%

By 2011, reduce the percentage of students achieving in the ‘less than the 25th percentile’ by y%

The target setting tool should be used to determine achievable data for 2011.


EXAMPLE 2 A large, metropolitan high school (Demonstration school 26)

Year 2010 Year 7

Aspect Writing

Percentages of students in bands


16 10 N/A


71 65 N/A


15 24 N/A



A higher percentage of students are performing in the bottom band compared to state (Band 4, school 16%, state 10%)

A higher percentage of students are performing in the middle bands compared to state (Bands 5, 6, 7 School 71%, state 65%)

A lower percentage of students are performing in the top two bands compared to state (Bands 8, 9 school 15%, state 24%)

20% of boys are performing in bottom band


10% of girls are performing in bottom band

71% of boys are performing in middle bands

69% of girls are performing in middle bands

10% of boys are performing in top bands

26% of girls are performing in top bands


Aspect Writing Year 7


77.4


86.0

School Mean 496.4 Standard Deviation

77.3




From this mean data the school performance in this aspect is worthy of investigation. (The NAPLAN Analysis Tool indicates that there is a problem.)

Data from Growth chart



All 37.13 35.8 29.7

Boys 35.41 32.8 26.0

Girls 38.89 39.2 37.8



The schools’ average growth is below state and region. Boys’ average growth is well below state and region. Girls’ average growth is close to state and region.


Percentile Ranges


<25th 27.9%

25th – 75th 54.68%

75th and above 18.23%



Fewer students are performing in the top percentile range. More boys in the bottom percentile range (30.71%) Fewer boys in the top percentile range (13.57%) Fewer girls are in the bottom percentile range (19.05%) More girls are in the top percentile range (28.57%)

Expected growth

Less than expected growth 53.2%


46.8%


How big is the tested group? 46.8% of students achieved expected growth. 43.6% of boys achieved expected growth. 54.0% of girls achieved expected growth.

Target Setting

Target based on Advice on writing target Percentages in student bands Target is to reduce students in the below minimum band and increase in the proficient band











Targets


Decrease the percentage of students performing in the bottom band for

writing from 16% in 2010 to x% in 2011.

Increase the percentage of students performing in the proficiency bands for writing from 15% to y% by 2011.

Raise the schools’ average growth in numeracy (particularly boys’) to be equal to or above state average growth by 2011.

By 2011, increase the percentage of students achieving in the ‘75th or above percentile range’ from 18.23% to x%.

By 2011, reduce the percentage of students achieving in the ‘less than the 25th percentile’ from 27.9% to y%.

Increase the percentage of students achieving above expected growth from 46.8% to y by 2011.


EXAMPLE 3 A small school in a semi-rural community on the metropolitan fringe (Demonstration school 7)

Year 2010 Year 5

Aspect Reading



17 7 N/A



51 59 N/A


34 33 N/A



A higher percentage of students are below the national minimum standard (Band 3 - school 17%, state 7%)

A lower percentage of students are in the middle bands (Bands 4, 5, 6 – school 51%, state 59%)

A similar percentage of students are in the proficient bands (Band 7 & 8 - school 34%, state 33%)

However, the cohort consists of only 6 students. One student, below the national minimum standard resulted in the 17% result.


Aspect Reading Year 5


82.9


81.4

School Mean 474.3 Standard 84.8


Deviation



From this mean data the school performance in this aspect is worthy of investigation. (The NAPLAN Analysis Spreadsheet indicates that there is a problem.) However, it is important to note that the mean is unreliable unless there are at least ten students.

Data from Growth chart




All 83.77 83.3 87.9

Boys 83.78 83.9 95.5

Girls 83.76 82.8 57.6



The school has a higher average growth than the state. Boys’ growth is well above the state average.

Percentile Ranges


<25th 20%

25th – 75th 80%

75th and above 0



The school has most students in the 25th – 75th percentile. There are no students in the 75th and above percentile.

Expected growth

Less than expected growth 20%


80%


How big is the tested group? All boys show greater than or equal to expected growth. One girl has made less than expected growth. One girl was unmatched.

Target Setting

Target based on Advice on writing target


Percentages in student bands Target is to reduce students in the below minimum band and increase in the proficient band










Targets


Increase the proportion of students achieving in the proficiency bands for reading by 2011

School trend data for reading shows positive increases from 2010

Reduce the proportion of students making less than expected growth by 2011


The following two examples the EMSAD model for writing small school targets

EXAMPLE 4 A small school in a rural, regional centre

Year 2010 Year 3, 5

Aspect Various aspects


2008 – 3/10 students in top three bands 2009 – 11/17 students in top three bands 2010 – 6/7 students in top three bands In 2011 there will be 10 students sitting NAPLAN (information from ASR).

Target setting

2008 2009 2010 2011 2012

Numbers in top three bands

3/10 11/17` 6/7

% aggregated

20/34=59%

Reporting 2009 2010 2011


11/17 6/7 7/10

% aggregated

24/34=71%

Target: Increase the three year aggregated percentage of Year 3 students achieving in the top three bands to 71% (from the 2008-10 average of 59%)

Year 5 Writing


2008 – 4/8 students in top three bands 2009 – 4/12 students in top three bands 2010 – 1/7students in top three bands In 2011 there will be 14 students sitting NAPLAN.

Target setting

2008 2009 2010 2011 2012


4/8 4/12 1/7

% aggregated

9/27=33%

Reporting 2009 2010 2011


4/12 1/7 12/14

% aggregated

17/33=52%

1n 2009 17 students sat the Year 3 writing. 14 achieved in the top three bands. A prediction of 11/14 is made for 2011 based on school based data, and knowledge of student background. Target: Increase the three year aggregated percentage of Year 5 students achieving in the top three bands to 52% (from the 2008-10 average of 33%)


Year 5 Writing

2008 –No data 2009 – No data 2010 – 100% less than expected growth Target: Increase the percentage of students achieving greater than or equal to expected growth from 0% in 2010 to 33% in 2011. (The figure 33% was taken from the school’s ASR target.)


Year 5 Numeracy

In 2009, 17 Year 3 students sat the numeracy test. 11 students performed in the top 3 bands. In 2011, 14 Year 5 students are expected to sit the NAPLAN numeracy test.


Target: Increase the percentage of the 2009 Year 3 cohort in the top 3 NAPLAN bands from 65% to 79% in Year 5 2011 (Aim for 11/14 students in top three bands)

EXAMPLE 5 A small, rural central school

Year 2010 Years 3, 5, 7

Aspect Various aspects

Year 3 Reading

2008 – 5/9 students in top three bands 2009 – 5/9 students in top three bands 2010 – 3/8 students in top three bands In 2011 there will be approximately 10 students sitting NAPLAN (information from 2008 ASR).

Target setting

2008 2009 2010 2011 2012


5/9 5/9 3/8

% aggregated

13/26 = 50%

Reporting 2009 2010 2011


5/9 3/8 7/10

% aggregated

15/27=56%


Target: Increase the three year aggregated percentage of Year 3 students achieving in the top three bands to 56% (from the 2008-10 average of 50%)

Year 3 Writing

2008 – 6/9 students in top three bands 2009 – 7/8 students in top three bands 2010 – 4/8 students in top three bands Anticipated number for 2011 is 10 students.

Target setting

2008 2009 2010 2011 2012


6/9 7/8 4/8

% aggregated

17/25=68%

Reporting 2009 2010 2011


7/8 4/8 7/10

% aggregated

18/26=69%

1n 2010 8 students sat the Year 3 writing. 4 achieved in the top three bands. A


prediction of 7/10 is made for 2011 based on school based data, and knowledge of student background. Target: Increase the three year aggregated percentage of Year 3 students achieving in the top three bands to 69% (from the 2008-10 average of 68%)

Year 3 Spelling

2008 –3/9 students in top three bands 2009 – 6/8 students in top three bands 2010 – 4/8 students in top three bands

Target setting

2008 2009 2010 2011 2012


3/9 6/8 4/8

% aggregated

13/25=52%

Reporting 2009 2010 2011


6/8 4/8 7/10


% aggregated

17/26=65%

Target: Increase the three year aggregated percentage of Year 3 students achieving in the top three bands to 56% (from the 2008-10 average of 50%) Year 3 Grammar and Punctuation


Target setting

2008 2009 2010 2011 2012


5/9 5/8 5/8

% aggregated

15/25=60%


Reporting 2009 2010 2011


5/8 5/8 7/10

% aggregated

17/26=65%

Target: Increase the three year aggregated percentage of Year 3 students achieving in the top three bands to 65% (from the 2008-10 average of 60%) Year 3 Numeracy


Target setting

2008 2009 2010 2011 2012


7/9 5/9 6/8


% aggregated

18/26=69%

Reporting 2009 2010 2011


5/9 6/8 8/10

% aggregated

19/27=70%

Target: Increase the three year aggregated percentage of Year 3 students achieving in the top three bands to 70% (from the 2008-10 average of 69%) Year 5 Reading


Target setting

2008 2009 2010 2011 2012


5/10 2/4 3/8

% aggregated

10/22=45%


Reporting 2009 2010 2011


2/4 3/8 5/8

% aggregated

10/20=50%

In 2009, 9 Year 3 students sat the reading test. 5 students performed in the top 3 bands. In 2011, 8 Year 5 students are expected to sit the NAPLAN reading test. Target: Increase the percentage of the 2009 Year 3 cohort in the top 3 NAPLAN bands from 56%% to 63% in Year 5 2011 (Aim for 5/8 students in top three bands) Increase the percentage of students achieving greater than or equal to expected growth from 42.9% in 2010 to above 50% in 2011. Year 5 Writing


Target setting

2008 2009 2010 2011 2012



5/10 2/4 7/8

% aggregated

14/22=64%

Reporting 2009 2010 2011


2/4 7/8 7/8

% aggregated

16/20=80%

In 2009, 8 Year 3 students sat the reading test. 7 students performed in the top 3 bands. In 2011, 8 Year 5 students are expected to sit the NAPLAN writing test. Target: Increase the percentage of the 2009 Year 3 cohort in the top 3 NAPLAN bands from 88% to 86% in Year 5 2011 (Aim for 7/8 students in top three bands) Increase the percentage of students achieving greater than or equal to expected growth from 42.9% in 2010 to above 50% in 2011. Year 5 Spelling 2008 –5/10 students in top three bands


2009 – 2/4 students in top three bands 2010 – 3/8 students in top three bands

Target setting

2008 2009 2010 2011 2012


5/10 2/4 3/8

% aggregated

10/22=45%

Reporting 2009 2010 2011


2/4 3/8 5/8

% aggregated

10/20=50%

In 2009, 4 Year 3 students sat the reading test. 2 students performed in the top 3 bands. In 2011, 8 Year 5 students are expected to sit the NAPLAN spelling test. Target: Increase the percentage of the 2009 Year 3 cohort in the top 3 NAPLAN bands from 50% to 63%% in Year 5 2011 (Aim for 5/8 students in top three bands) Increase the percentage of students achieving greater than or equal to expected growth from 42.9% in 2010 to above 50% in 2011. Year 5 Grammar & Punctuation



Target setting

2008 2009 2010 2011 2012


5/10 2/4 2/8

% aggregated

9/22=41%

Reporting 2009 2010 2011


2/4 2/8 5/8

% aggregated

9/20=45%

In 2009, 4 Year 3 students sat the reading test. 2 students performed in the top 3 bands. In 2011, 8 Year 5 students are expected to sit the NAPLAN grammar % punctuation test. Target: Increase the percentage of the 2009 Year 3 cohort in the top 3 NAPLAN bands from 50% to 63% in Year 5 2011 (Aim for 5/8 students in top three bands) Increase the percentage of students achieving greater than or equal to expected growth from 28.6% in 2010 to X% in 2011.


Year 5 Numeracy


Target setting

2008 2009 2010 2011 2012


5/10 2/4 3/8

% aggregated

10/22=45%

Reporting 2009 2010 2011


2/4 3/8 5/8

% aggregated

10/20=50%

In 2009, 4 Year 3 students sat the reading test. 2 students performed in the top 3 bands. In 2011, 8 Year 5 students are expected to sit the NAPLAN numeracy test. Target: Increase the percentage of the 2009 Year 3 cohort in the top 3 NAPLAN bands from 50% to 63% in Year 5 2011 (Aim for 5/8 students in top three bands) Increase the percentage of students achieving greater than or equal to expected growth from 57.1% in 2010 to X% in 2011.


Year 7 Reading


Target setting

2008 2009 2010 2011 2012


3/3 3/9 5/7

% aggregated

11/19=58%

Reporting 2009 2010 2011


3/9 5/7 5/7

% aggregated

13/23=57%

In 2009, 3 Year 3 students sat the reading test. 2 students performed in the top 3 bands. In 2011, 8 Year 5 students are expected to sit the NAPLAN reading test. Target: Increase the percentage of the 2009 Year 3 cohort in the top 3 NAPLAN bands from 50% to 63% in Year 5 2011 (Aim for 5/8 students in top three bands)


Increase the percentage of students achieving greater than or equal to expected growth from 57.1% in 2010 to X% in 2011.

INTERPRETING RESULTS AND SETTING TARGETS

TARGET SETTING SCAFFOLD

Year Cohort

Aspect








Aspect Year

State Mean Standard Deviation

School Mean Standard Deviation



Data from Growth chart Average Scaled Score Growth


All

Boys


Girls



Percentile Ranges


<25th

25th – 75th

75th and above



Expected growth

Less than expected growth




Target Setting

Target based on Advice on writing target School Mean and Standard Deviation From the calculation spreadsheet based on data being with 0.2 or 0.5 Standard Deviations, indicating if data is within state parameters, of concern or of serious concern Targets should set increase to move school from: • serious concern to of concern • of concern to within state parameters • within state to equal or better than state






Minimum Growth Target can be specified in more of the group


Target is based on 100% achieving minimum growth Target should focus on more students at or above state minimum growth

at the minimum growth level

Percentages in student bands Target is to reduce students in the below minimum band and increase in the proficient band


Target Setting


Using Item Analysis to undertake a review of student performance Once your school has reviewed the means and standard deviations, percentages in bands, and school growth information for each year group, you will have a clear idea of how the group has performed compared to the state, region, SEG etc. Your school will have discussed where they need to focus their resources to bring about improvement in student performance. An integral part of this review, and particularly important to classroom teachers, is the information found in the Item Analysis.

Teachers can use this information to closely examine each aspect. It will help them to determine what their students know, what needs to be taught or retaught, and access strategies to help teach skills / concepts.


Note: The information in the Item Analysis is important to all teachers, not just the teachers of the cohort tested. The following table shows where a stage could start their analysis –

Stage 1 - Year 3 data

Stage 2 – Year 5 data



The NAPLAN tests are held early in the school year with students who have just exited a stage of learning. The performance of students can be a reflection of the teaching that took place in the previous stage. It is therefore important for teachers to analyse the types of questions that the students are proving difficult for their students. It may also be useful to check past years and see if there are any trends evident. To assist you with this process a template has been included that may help your team to analyse the questions causing difficulty in your school. Record the outcome, refer to the

syllabus and spend some time examining the knowledge and skills required to competently achieve this outcome.


EXAMPLE NAPLAN Item analysis – LITERACY QUESTION

Record the skill band.

Insert data from the Item Analysis tool for your school, state, region & SEG for this question.

A good idea would be to insert a copy of the NAPLAN question and resource material – these can be accessed from Item Analysis – Question details.

This section is very important! Have a discussion with colleagues about the responses your students gave. Use your syllabus to clarify the requirements of the outcome. Examine the questions. Can you determine why a considerable number of students may have given an incorrect answer? See below

Once you have discussed the demands of the question and identified possible student misconceptions, discuss the strategies that could be implemented to improve student understanding.

This section offers a checklist of areas to consider when analysing student performance.


Question Number: 19

Outcome: RS 2.6: Uses efficiently an integrated range of skills and strategies when reading and interpreting written texts. Infers a character's motivation in a fable.

Skill Band: RS2.6

Regional % Correct: 74

State % Correct

School % Correct

Difference

Year 3

77

49

-28

School State Difference

1 He wanted to punish his son. 12% 4% 8%

2 He was fighting with his son. 10% 4% 6%

3 He was too old to go himself. 27% 13% 14%

4 He wanted his son to learn.* 49% 77% -28%

Question: Why did the father send his son into the forest?


Demands: Possible reasons for large difference:

Students responded to story

emotionally – a dad sending a child out

on his own, foreign concept to children,

parents don’t leave children alone –

this is mean.

Students don’t understand that a fable

is a story with a moral. Children didn’t

ask the question – what is this story

trying to teach me?

Lack of understanding of the word

‘necessity”.

Perhaps read other stories where

fathers are old and seek help from their

younger, more-able children.

Reasoning - why would a father send a

son to do something he was able to do

himself?

Inability to read the text and guessed

the answer.

Statements of Learning: When students interpret texts they have the opportunity to draw inferences from directly-stated descriptions and actions. (p. 17) Teaching Strategies: Developing Characters in Narratives

Check: Literacy demand (e.g. metalanguage;

aspects of grammar) Word/sentence/text level Context; background knowledge Literal Vs inferential comprehension Visual literacy Distractors

Teaching strategies, as accessed from the Question Details screen, are an important resource that may be a starting point in investigating strategies to support teaching and learning in the classroom. The teaching strategies have been specifically developed to address the skills and knowledge assessed and are linked to state syllabuses and the NSW Quality Teaching framework. They can be incorporated into school resources. A teaching strategy has been linked to every test item. Clicking on the Teaching Strategies button accesses a bank of literacy and numeracy strategies – literacy strategies categorised by skill and numeracy strategies categorised by strand and sub-strand. There are strategies available for the different stages of learning. The bank of teaching strategies also include Supporting Aboriginal Students, Supporting ESL Students and Supporting Students with Learning Difficulties.

Interpreting NAPLAN results and setting targets€¦ · 3. EMSAD Draft at Nov 2011 USING SMART DATA...

Documents

Transcript of Interpreting NAPLAN results and setting targets€¦ · 3. EMSAD Draft at Nov 2011 USING SMART DATA...