Analysing ANA results

Nicky RobertsWits School of EducationKelello Consultingnickyroberts@icon.co.za

Context

Focus on Primary Maths project Errors in marking, adding and recording

results (2012) Using ANAs as a diagnostic tool for teaching

and development (2013)

Research in two schools in Cape Town• Focus on Primary Maths 2012-2014

– School based maths teams– Professional development for FP teachers in

maths• Mini seminars• Co teaching interventions• Mathematical thinking course

– Responding to ANA attainment– Sharing lessons

Responding to ANA attainment

Tracking ANA results year on year Question-by-question capture and analysis Planning for fourth term based on ANA

analysis Rasch analysis comparing the two schools

Background to the two schools• 2 schools are equal partners in development project, chose

to be named at school level• Capricorn Primary School (CPS)

– 700 learners, 2011 below provincial mean (ANA and systemics)– Township, backyard dwellers, poverty, gangsterism and taxi

violence, drugs (Tic), alcohol (FAS). Mostly households in poverty.

– Minority not English main lang, LOLT is English• The Grove Primary school

– 700 learners, history of excellent attainment (ANA and systemics)

– Suburban, model C, mostly affluent households– Majority English main lang, LOLT is English

No anchor questions Predictable test Not year-on-year

comparison Assume system is

stable, then performance relative to national mean is indicator of overall performance

Indicator of proportion of learners meeting grade level requirements (relative to national pass rate, on that ANA)

WCED systemic (standardized questions and marking, so year-on-year comparison)

Question by question data capture

Real power for instructional design

Q1 Q2 Q3 Q4...

Learner 1Learner 2Learner 3Learner 4Learner 5….

CorrectIncorrectFacility

ANA analysis template

Class capture sheet Summary sheet Question summary sheet Spreadsheet template suburban school Spreadsheet template township school

What we did in 2013

Before ANA marking Import the learner names from EMIS into classes. One class

per sheet. Check formulas and that the template works as you want it

to

On the marking day Laptops/ computers with template in marking venue Capture marks for each script Queries – go back to teacher who was marking

What we did in 2013

After marking daysCreate grade summariesCompare to previous yearsLook for questions of common difficulty

Very poor: colour code in green Poor: colour code code in pink Revise maths teaching and development plan for

fourth term

Functions used in Excel template

SUM AVERAGE MIN MAX STDEV

Some notes on using Excel

IF-THEN COUNT IF

What we did in 2014

Repeated the descriptive statisticsUsed this to create grade level ‘mini lesson studies’

cycles, eg: Teaching bonds in Grade 1 Teaching views and positional language in Grade 2 Teaching analogue time in Grade 3

Plan – teach- reflect cycleTeacher presentations at mini conference, AMESA

regional and AMESA national

Mini conference (Day 1 internal)

Mini conference (Day 2 district)

Mini conference

AMESA regional

What we did in 2015

Repeated the descriptive statistics Used the data to conduct Rasch analysis:

Misfitting itemsBiases between schools (based on DIF values)

Used this to target problematic questions and review these qualitatively

Qualitative analysis (from initial qunat data)

Total marks 1Mean 0.02Standard deviation 0.14Mean percentage 2%Number writing 80Number getting 0 79Percentage getting 0 99%

Identified as a major area of weakness

Errors on Q 25.1

Some completely off

Most common error was to break up number rather than converting

Conclusion

ANA is a major investment in SA education It gives annual standardised data across schools If anchor questions were included, it would also give year-on

year comparisons It gives schools rich data for error analysis (if item level analysis

is conducted) which can lead to lesson study cycles/ interventions

Teachers need support in marking ANAs (again directs attention to the mathematics) and reporting common errors to district/ province

nickyroberts@icon.co.za