UKFIET International Conference on Education and Development
15-17 September, 2009
Oxford, England
The Politics of Formal Examinations:
language policy and progress
ConvenorsPauline Rea-Dickins
Guoxing Yu
2 The symposium
• Rationale: all researching in contexts where:– English as an L2/L3 is used in high stakes
formal examining contexts: SSA, Malaysia, USA & UK
– Politics & policy impacts on issues of fair & ethical assessment
– Policy is intended to lead to progress & equitable distribution of resources & opportunities for ALL learners
– Progress?
The symposium
Dr Ong Saw Lan (Universiti Sains Malaysia )
Narrowing the science achievement gap with the
dual-language test-booklet
Professor Jamal Abedi (University of California,
Davis)
Impact of language factors on the assessment
outcomes of English language learners
4UKFIET International Conference on Education and Development
15-17 September, 2009
Oxford, England
The Politics of National Examining Processes:
the case of languages in Zanzibar
Abdulla H. Mohamed, Pauline Rea-Dickins, Guoxing Yu
Student Performance in National Examinations: the dynamics of language in school achievement (SPINE)
www.bristol.ac.uk/spine (ESRC/DfID RES-167-25-0263)
5 Introduction
• Empirical data & documentary evidence reflecting some of the
realities of national examining processes:– Student performance on exam items– Statistical data on student exam performance– Implementation of classroom assessment
• The policy & the politics: historical perspectives: focus on CWA
• Conclusions : policy & politics & issues of justice in examining– progress?
5
6 Student performance on exam items
English RC Qn: How whales resemble man
45 students took this item:
• 35.6% = no answer
• 26.7% = wrong answer
• 28.9% = partially correct answer
• 8.8% = correct answer
7Interview: D1 who didn’t answer Q3 explains
D1: “because I did not understand by this this … resemble” (lines 115-117)
Int: “If I tell you that resemble means ‘to look like’ … can you do the question now?
D1: “Yes”Int: OK so what’s the answer?D1: “Man … is warm blooded … and
whales also … whales have lungs and man also have lungs …” (122-133)
8 Biology: responses to original item
• No answer = 67.4%
• Wrong answer = 21.7%
• Partially correct answer = 6.5%
• Correct answer = 4.4%
9 Biology: on locusts
10
Biology: on locusts
11 Biology: on locusts
Questions:
a) In which picture do you think the locust will/may
die?
b) Why do you think it will/may die?
12 Original item modified:
• Greater contextualisation
• Simplification of instruction
• 2 structured parts: A & B
• Visual clues to support information retrieval
• Rephrasing of the item
• Altering item layout
13 Results
• Original item– Only 32.6% of students wrote an answer – Just under 11% gave a partially correct or
correct answer
• Modified item– 100% responded to this item– 42% gave a partially correct answer to Part A– 53% gave a partially correct answer to Part B
14Changes in student response
OR
Sc
Written responses on Modified Item
MOD
Score
Comment
(Neil Ingram: biologist)
H3 0 In picture A the locust may/will die
I think it will/may dies because the locust get its breathing by using its body
2 + 2 Understands that locusts breathe using the body
C2 1 I think it will/may die because the boy is dipping the locust in the water to all the bodies with its trachea that used to respiration as a respiratory surface of a locust
2 + 2 Getting very close to a very complex answer “I think a very able pupil indeed”. The right answer but with great difficulty in expressing this in English
Means of Form II Exam Results (MoEVT) 15
KISW Islamic
studies
BIO CHEM MATH PHYS
2004 49 37.1 18.4 30.8 21.1 25.8
2005 46 44.3 22.6 34.2 15.3 25.4
2006 36 47.9 21.5 34.3 15.5 25.1
2007 35.5 44.5 21.5 32.0 14.3 23.9
2008 50.6 47.5 24.5 34.8 16.4 29.1
16 ENG and MATHTotal: 64.717, school=29.18%,
pupil=70.82%
ENGLISH explains (90.4-
64.717)/90.4=28.41% of the
maths total variance
Total=90.4, School=30.75%,
Pupil=69.25%
17 ENG and BIO17
Total=65.646, school=18.17%,
pupil=81.83%
ENGLISH alone explains
(114.968-65.646)/114.968=42.90%
of the total variance in BIOLOGY
Total=114.968, school=18.79%,
pupil=81.21%
18 English and CHEMTotal=158.281, school=23.06%,
pupil=76.94%
ENGLISH explains (275.993-
158.281)/275.993=42.65% of the
total CHEM variance
Total=275.993, school=22.05%,
pupil=77.95%
19 How about KISWAHILI & ARABIC?
• Although other two languages (Kiswahili and Arabic) are also significant
predictors of the students’ performance in maths, biology and chemistry,
it is noted that they are less capable of explaining the variances than
ENGLISH.
• KISWAHILI explains (275.993-88.954)/275.993=31.54% of CHEM total
variance, (114.968-76.585)/ 114.968=33.39% of BIO total variance,
(90.4-75.678)/90.4=16.29% of MATH total variance.
• ARABIC explains (275.993-203.452)/275.993=26.28% of the CHEM total
variance, (114.968-88.569)/114.968=22.96% of the BIO total variance,
(90.4-71.885)/90.4=20.48% of the maths total variance.
20 Summary of the multilevel models (a)
• It is very clear that ENGLISH is a significant and substantial predictor of the students’ performance in MATH, BIO & CHEM.
• The school-level variances explained in the cons models as well as in the models including ENGLISH as the single explanatory variable demonstrated a substantial proportion of the variance is attributable to school factors
21 Summary of the multilevel models (b)
• Not much improvement in terms of fitness of the models (measured
by the change of % of school-level variance in the total variance).
• Therefore, essential to collect further school- and pupil-level data to
examine what factors (e.g. English language learning opportunities at
home and at school, academic English proficiency) and how much
they account for the variances (in the tradition of school effectiveness
studies) – our plan for nationwide data collection using pupil and
headteacher questionnaires and vocabulary knowledge test
Coursework assessment as part of national examining: implementation issues
In Form II, Teacher Assessment (TA) was implemented
based on the following assessment procedures:
• Class work
• Oral questions
• Homework
• Weekly tests
• Terminal tests
•
22
Continuous Assessment Format & Guidelines
• Term: --------- Class: --------- subject---------
Pupils’
name
Month
1
10%
Month
2
10%
Month
3
10%
Month
4
10%
Total
C/work
40%
Exam
60%
Grand
Total
100%
X 5 6 3 8 22 30 52
Coursework assessment as part of national examining: implementation issues
• The first three activities make up 40% and the fourth
makes 60% which when combined make 100%
• The final score (out of 100% ) obtained is sent to the
Department of Curriculum and Examination for
further
processing (CWA is 25% of overall exam score)
Coursework assessment as part of national examining: implementation issues
However: significant variability among teachers:
• the types of specific activities that constitute teacher
assessment
• the number of activities from which they take marks for
each month
• decisions on how to handle student absenteeism
• how to get 10% of marks for each month; some picked
the one that students did better on, some picked
randomly, and some put the average of all marks
Coursework assessment as part of national examining: implementation issues
• Understanding of what assessment is to be done
• Awareness of guidelines
• Percentage of marks that go to the final examination
• Sharing what teachers know and practice about
assessment guidelines among teachers within and
across schools
• Level of monitoring of Teacher Assessment by head
teachers.
Continuous Assessment Format & Guidelines
• Note: Coursework assessment will be obtained from
class works, home works and weekly tests or as it will
be directed otherwise by the Department or the
Ministry. All the marks scored by the pupil from the
mentioned activities within a particular month, will be
combined and transformed into percentage of ten (10%)
before filled in the relevant space in this form.
• Source: Ministry of Education Working Guidelines: Guideline
No. 9 (1995)
Continuous Assessment format & Guidelines
Additional notes from another version
• If the term is less or more than four (4) months, four activities
chosen by the teacher him or herself should be picked at equal
intervals. The teacher is advised to give as many tests as he can
in a term but should choose only four for recording.
• In getting pupil’s marks for terminal exam for each subject, the
exam itself should contribute 60% and classroom assessments
(coursework assessments) should contribute 40%
Coursework assessment as part of national examining: implementation issues
• The language to be used is as indicated in the lesson plan.
• These notes appear to contradict with the first version which
might be leading to confusion. This version requires the
teacher to find average of marks in a month while the
second requires the teacher to choose one activity from
which the marks can be recorded.
• Source: Ministry of Education Working Guidelines: Guideline
No. 9 (1995)
The Policy & the Politics: historical perpsectives
• The need to make Assessment procedures more friendly,
less competitive and practically focused
• The prescription that student assessment must:– Include works that can enable students to function
effectively in their environments– Cover both student performance in theoretical works and
in practical productive works– Be done throughout the year
The Policy & the Politics: historical perpsectives
• The National Examination Council of Tanzania (NECTA),
formalised the new examination system by including
projects, character assessment, exercises and tests in the
assessment system
Teachers were then given responsibility to:
• measure students’ ability to use knowledge and skills
acquired, in their local environments and,
• keep ongoing records of their progress
The Policy & the Politics: historical perpsectives
Tanzania and Zanzibar Education Policies• Both The Tanzania Education and Training Policy (1990:79) & The
Zanzibar Education Policy (2006:28) emphasised that continuous assessment combined with the final exam paper would be the basis for awarding of certificates at Secondary Education levels
• For selection purposes and quality control, consistency between teacher assessment and that of the National examinations was emphasized in The Zanzibar Education Master Plan of 1996-2006
Language Policy and Assessment
• Students assessment is affected by Language policy
because students have to demonstrate knowledge and ability
through a language accepted by the system.
• The Zanzibar Education Policy of 1995, Kiswahili version
Section 5.72 (28), The Zanzibar Education Master Plan of
1996 (43), the Education Policy of 2006, all emphasized the
use of English Language as a language of instruction for all
subjects in secondary education except for Kiswahili and
Islamic studies.
Language Policy and Assessment
So, English, as a language of instruction in
teaching science, maths and so forth at secondary
level must be a language of assessment
Does this have to be the case?
Policy & Politics: progress for learners & learning?
Given that English is politically& policy wise the language of the examinations:
• “No test taker shall be harmed by the test (Hamp-Lyons 1989:13)
• No learner should be disadvantaged, in the interests of justice for ALL learners
• Ethical issues re. fairness: need to give individual a fair chance
35
36 Policy & Politics: What progress towards social justice?
Impact/Potential Disadvantage (examples)
Consequences/Injustice: (examples)
•Learners do not engage or respond poorly in examinations•Subject area (e.g. Biology, maths) construct can only be assessed where a linguistic construct has been successfully negotiated
•Loss of self-esteem & motivation for learning•Learners fail to reach their potential (glass ceiling effect) or fail altogether•Leave school as unsuccessful (e.g. at end of Basic Education)•Unequal access to available resources, educational experiences & work opportunities •Unskilled & unable to join the workforce in turn contributing to social & economic deprivation
37Policy & Politics: What progress towards social justice?
Impact/Potential Disadvantage
Consequences/Injustice
•Teachers do not use full range of LTA procedures & processes•Use of inappropriately constructed assessment frameworks•Inaccurate CWA of learners•CWA implemented as a series of tests
•Learners not fully supported in their language & content knowledge development & fail to reach potential •Test performance valued over learning •Leave school with poor educational outcomes•Inadequate d-base for decision making about student & learning progression
Top Related