Jta conference2012

A visual analysis of relationships between

GSAT Grade 7 & 9 Diagnostic and CSEC

performance in secondary schools in Jamaica 2008-2011JTA Education Conference 2012

Robert Philips, Education Specialist

e-Learning Jamaica CompanyApril 2012

www.e-ljam.net

Background

• The e-Learning Jamaica Project is a collaboration between the Ministry of Science Technology Energy and Mining and the Ministry of Education to systematically strengthen inputs to instruction in high schools.

• Information and communications technologies and instructor capabilities are being enhanced, with a view to increasing success of students at the CSEC level

Glossary

• GSAT-- the national terminal examination for the primary curriculum serving as the placement tool for secondary school

• PSLE—the equivalent exam in Singapore

• Communication task—the two part writing component of the GSAT examination

• CXC—Caribbean Examinations Council—regional authority for school examinations

• CSEC—CXC exams typically done at the end of 11th grade providing a basis for college matriculation

• GNDT—Grade 9 diagnostic test—an examination designed by CXC to provide planning and intervention information to schools and individual students, adopted by the MOE for national implementation

• All-Age Schools—schools with some secondary grades in addition to full primary

• Primary and Junior High—another type of school with secondary grades in addition to full primary

Project Components

• Infrastructure• Materials• Assessment• Remediation• Training

Organisation

• Context• Vision• Practice• Results • Analysis• Lessons learned• Sustainability strategies

Context

• GOOGLE search on “GSAT ANXIETY” has 79,400 hits

• Search on “PSLE ANXIETY” has 46,600 hits

Vision

If you want to improve a system over time you will get surprises if you don’t have a tracking process

Practice

• Ministry of Education agreed to a National 9th grade Diagnostic implemented by e-Ljam

• Caribbean Examinations Council contracted to organise and deliver– End-of-year multiple choice test with– Continuous internal assessment– Voluntary 7th grade assessment

The roots of school performancer=0.99 !!!!

Math 2009 vs Math 2008 GSAT

0

20

40

60

80

100

120

0 20 40 60 80 100 120

Math GSAT 2008

Mat

h G

SA

T 20

09

School NameMath 2008

Math 2009

Aabuthnott Gallimore High 58.88 53.13

Albert Town High 46.73 44.73

Annotto Bay 53.96 50.62

Ardenne High School 94.39 93.63

Ascot High 48.52 45.29

A school perception index ?

• Intake GSAT Math means for two consecutive years for high schools

• Explanation invited

• Submission that school perception as indexed by selection and/or placement is the main determinant of future performance

• Visual anomalies are technical schools which became normal clients of GSAT in 2009

• Source :MOE data


0

20

40

60

80

100

120

0 20 40 60 80 100 120

Math GSAT 2008

Math

GSAT

2009

A school perception index ?

• Intake GSAT scores are technically independent of the internals of schools

• Schools with lowest intake GSAT scores have lowest levels of academic success

• A new rationale for placement may be required


0

20

40

60

80

100

120

0 20 40 60 80 100 120

Math GSAT 2008

Math

GSAT

2009

Results of Assessment Component

Growth in participation2009 7000 pilot grade 9 30 schools2010 30000 national grade 9 150 schools2011 64000 grade 9 and grade 7 173 schools * 2012 82000 projected 180 schools

*10 Benchmark All-Age and Primary Jnr High included in 2011

Scanning Technology

• Examview• Use scanners supplied by

project• Local, decentralised solution• Automatic Document feed• Powerful reporting suite• Replaced CXC registration

strategy and MOE centralized scanning

• 7 day turnaround

Performance Trends 2010

• 10% of schools don’t participate• 11% of students in participating schools

miss exams• 81% of participants score 45% or greater

on multiple choice test for English• 38% of participants score 45% or greater

on Maths test

2010---Math—level 1• Fairly good performance is seen on statistics

items, with an average of 70%.• Many students missed items on Number &

Computation such as: order of operations (70% of students), place value (5-digits (60%), decimal – hundredths (70%)) (easily Grades 6-7 objectives).

• Students answered correctly, on average, only one-third of the items on Measurement & Geometry and Algebra.

• Source Faulkner,D. CXC Exam Consultant•

2010—math level 1

• Females students performed better than male students in all sub-topics except in Measurement and Geometry.

• The average difference is 2%. • The largest differences are in

Statistics (7.6%) and Consumer Arithmetic (4.1%)

• The difference in favour of males in Measurement and Geometry is 0.5%.

• 36.4% of female students and 31.6 % of male students scored 45% or more.

• Source Faulkner,D. CXC Exam Consultant

Math 2010—level 2• All sub-topics, except Algebra have

mean percent scores above 40%.• Statistics, Consumer Arithmetic and

Relations & Graphs have the highest mean percent scores.

• Algebra has the lowest scores on this paper.

• Students need more work in the following areas: Number and Computation, Measurement and Geometry and Algebra.

• Source Faulkner,D. CXC Exam Consultant

Select schools 2010

• School A has been closed

Select schools 2010

• School B labelled as failing

Select schools 2010

School X a classic outlier GSAT scores >95 or +3 standard deviations

Select schools 2010

• Ecological validity of the assessment system tested using six Catholic high schools

• English > Math

Select schools 2010


• “posh” vs “poor” * x,c vs a,b

* after Miller

Select schools 2010


• girls vs boys

y,c vs z

Summary resultsGrade 9

2011• Available cohort 41946• Patricipation 37362• Scoring less than 45% English 8816• National Means

– Math 37.06– English 54.99– Integrated Science 44.65– Social Studies 51.53

Summary resultsGrade 7--2011

• Available cohort est. 45000• Patricipation 27000• Scoring less than 45% English 6800• Scoring less than 45% Math 11900

• National Means– Math 45.7– English 55.8– Integrated Science 50.6– Social Studies 50.5

Grade 9 Math vs English 2011

• 140+ of 173 schools

• Gap in raw scores lowest for outliers

how schools math and language performance covary at grade 9

0

10

20

30

40

50

60

70

80

90

1 8 15 22 29 36 43 50 57 64 71 78 85 92 99 106 113 120 127 134

school no.

tes

t m

ea

n f

or

sc

ho

ol

math_9_2011

eng_9_2011

GSAT scores 2008 vs Grade 9 2011

• Same cohort after 3 years

• GSAT means of 80 give ca 100% probability of ninth grade success for English

• Math performance would be interesting to track

English GSAT vs Math GSAT 2010

• Between school means of English and Math abnormally linear

• Explanation invited• Is reading the

common factor?• Is there a variable

called test taking?

Chart Title y = 0.9262x + 5.2538

R2 = 0.9935

0

10

20

30

40

50

60

70

80

90

100

0 20 40 60 80 100 120

math gsat 2010en

glis

h gs

at 2

010

egs2010

Linear (egs2010)

GSAT means 2010 vs Gseven 2011

gsat 2010 vs gseven 2011y = 0.7231x + 10.42

R2 = 0.9327

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

90.0

0.0 20.0 40.0 60.0 80.0 100.0 120.0

gsat 2010

gs

ev

en

20

11

• Low contribution of instruction to the variance after a year in high school

• R (g6 vs g7) > .95

• Sources: MOE 2010 GSAT results and e-Ljam grade 7 exams 2011

GSAT communication task

• Currently the only production test in GSAT

• Divides schools into two quadrants• Communication task of 7 and

below associated with lower overall performance

• Planned for replacement by a book report


GSAT communication taskone year later

• Urgent review of the future of the communication task is recommended

• No evidence that book reports are going to represent students work given the behavior with CSEC SBAs


Indices of CSEC performance

• Bill Johnson—Ralph Thompson –Gleaner 2020 index

• Adopted by e-Ljam ,with some reservations as a meaningful index

• Simply – school score is 100 if every student in cohort scores a Grade 1

• Grade 1 is valued at 4



Tracking variables for six select schools

tracking variables for selected schools

01020304050607080

o1 o2 b1 h1 s1 t1

school

perce

nt or

inde

x for

gsat

or gn

ine or

csec

mgs2008

egs2008

math_9_2011

eng_9_2011

e10

m10

avg_gsat

av_g9

• GSAT• GNINE• CSEC english• CSEC math

Visual analysis—grade 7--2011

schools by comm task score

0

20

40

60

80

100

120

1 8 15 22 29 36 43 50 57 64 71 78 85 92 99

school

gs

at

gs

ev

en

cx

c in

de

x

mgs2010

egs2010

scigs2010

socgs2010

math_7_2011

eng_7_2011

sci_7_2011

soc_7_2011

eng-jt_2007

math-jt_2007

e09

m09

e10

m10

Visual analysis Grade 7--2011

• CSEC indices m 10—math 2010 e 10---- english 2010• GSAT mgs 2010 egs 2010• Grade 7 –math_7_2011 eng_7_2011

Visual analysis

• Partially cross-sectional 2010 GSAT vs CSEC

• partially longitudinal 2010—2011 GSAT vs G7

• Intake GSAT scores trend similarly to CSEC indices

• Floor CSEC results for more than half the schools

• GSAT placement reinforces a culture of failure in the majority

Conclusions

• GSAT intake means are a measure of the perception of schools

• GSAT means forecast Grade 7 Grade 9 and CSEC performance with the robustness of physical science measures

• Maths and English gap is evident from grade 7

• Geometry and algebra weaknesses are evident

Contrary to the view that GSAT is too hard, there are not enough moderate to high performance students to share among the 170+ schools

• Students scoring less than 80 could be placed on geography and some of those above could receive advanced placement and seek to complete high school in four years

• The GSAT communication task should be reviewed for retention as independent writing is encouraged

• The communication task discriminates and is associated with varying levels of terminal secondary performance

• Data is available to inform policy and strategy choices

• An education system that does not acknowledge the foundations of literacy will have low levels of success

Lessons learned

• Jamaican teachers are eager to incorporate technology into their professional practice, including assessment

• The consistent placement of weaker students in the same schools reinforces the culture of failure

• Assessment at ninth grade indicates that even at current levels of primary school output quality, higher levels of success in CSEC Math and English are achievable

Sustainability

• The Ministry of Education has contracted a supplier for the strategic overhaul of the lower secondary curriculum and related assessment processes

• E-Learning Jamaica has been requested to liaise with the supplier

Question Time

Robert Philips, Education Specialist

www.e-ljam.net

Jta conference2012

Education

Transcript of Jta conference2012