Centre for Learning Resources
PARBHANI Baseline Study - English Improvement Program
Contents 1. Executive Summary ......................................................................................................................................... 3
2. Background ...................................................................................................................................................... 4
2.1 We Learn English Program, Parbhani District ............................................................................................ 4
2.2 Overview of the Baseline Study ................................................................................................................. 5
3. Methods .......................................................................................................................................................... 5
3.1 Research Question ..................................................................................................................................... 5
3.2 Study Procedures ....................................................................................................................................... 5
3.2.1 Sampling Strategy ............................................................................................................................... 5
3.2.2 Sample ................................................................................................................................................ 5
3.3 Instruments and Procedure ....................................................................................................................... 7
3.4 Data Collection Procedure ......................................................................................................................... 8
4.Data Analysis (WLE – Listening and speaking skills) ......................................................................................... 8
4.1 WLE Level 1 Test ........................................................................................................................................ 8
4.2 WLE Level 2 Test ........................................................................................................................................ 9
4.3 WLE Level 3 Test ...................................................................................................................................... 10
5. Findings .......................................................................................................................................................... 12
5.1 Grade 4 results on WLE Test Level 1 ....................................................................................................... 12
5.2 Grade 5 results on WLE Test Level 2 ....................................................................................................... 13
5.3 Grade 6 results on WLE Test Level 3 ....................................................................................................... 14
6. Conclusion ..................................................................................................................................................... 15
Table of Figures
Table 1: Overview of WLE sample size ................................................................................................................ 6
Table 2: Overview of WLE Level 1 testing tool .................................................................................................... 7
Table 3: Overview of WLE Level 2 testing tool .................................................................................................... 7
Table 4: Overview of WLE Level 3 testing tool .................................................................................................... 7
Table 5: Overview of data collection procedure ................................................................................................. 8
Graph 1: WLE Level 1 Test - Grade 4 ................................................................................................................. 12
Graph 2: WLE Level 2 Test - Grade 5 ................................................................................................................. 13
Graph 3: WLE Level 3 Test - Grade 6 ................................................................................................................. 14
1. Executive Summary
This document describes the results of a baseline study conducted in July 2019. The presented study was
undertaken at the beginning of implementation of the We Learn English Program in Zilla Parishad
Schools in Parbhani District, Maharashtra. This programme is a joint initiative of Zilla Parishad Parbhani
District as implementation agency and Centre for Learning Resources (CLR) as technical and capacity
building resource partner. The WLE program aims to improve English listening and speaking skills in
primary school students. WLE is an audio program consisting of serialized stories revolving around day-
to-day conversations between an audio teacher and children. The lessons enable vocabulary building and
speaking with correct syntax. The WLE lessons has been integrated into the regular time table of
students of the Zilla Parishad Schools in Parbhani District.
This baseline study was guided by the questions- What is the current level of English language
comprehension and speaking skills in Grade 4, Grade 5 and Grade 6 children of Zilla Parishad Schools?
To answer this question, the WLE Testing tool consisting of Level 1, Level 2 and Level 3 questionnaire
was used to measure the listening and speaking skills in students through one on one interviews.
The study was conducted in a total of 42 schools from the 9 talukas Gangakhed, Jintur, Manvat, Palam,
Parbhani, Pathri, Purna, Selu and Sonpeth from Parbhani District. 4 schools from Gangekhed, 7 schools
from Jintur, 3 schools from Manvat, 5 schools from Palam, 7 schools from Parbhani, 3 schools from
Pathri, 6 schools from Purna, 3 schools from Selu and 4 schools from Sonpeth taluka participated in the
study. WLE tests was administered on 4-5 students each belonging to Grade 4, Grade 5 and Grade 6
from these schools.
2. Background
2.1 We Learn English Program, Parbhani District WLE program in Parbhani District, Maharashtra was initiated in 2019 by Zilla Parishad Parbhani in
partnership with CLR. The Program was officially launched on 15th August by Palak Mantri Shri
.Gulabrao Patil along with Honorable C.E.O B.P. Prithviraj and former District Collector P.
Shivshankar. WLE aims to improve English listening and speaking skills of students in the Zilla Parishad
School in Parbhani, Maharashtra. The project aims to benefit students from Grades 4-6 in Zilla Parishad
Schools. The program has been rolled out in a total of 1128 Zilla Parishad Schools in the 9 Talukas of
Parbhani District. Of these 600 schools are upper primary schools and the rest are primary schools.
To enable improved English skills in students, English content and support in the form of WLE (a
listening and speaking program) has been integrated into the regular time table of students. WLE is an
audio program consisting of interesting serialized stories revolving around the day-to-day conversations
between an audio teacher and children. The lessons enable vocabulary building and speaking with the
correct syntax. These lessons are transmitted in the classrooms at fixed timings using the services of
Aakashwani Parbhani and FM 98.8 Parbhani. In addition, the lessons are also live telecast on the FM
Parbhani mobile application at the same timings as the radio broadcast. The mobile application is a
measure to overcome the issues of poor mobile network and power outage in classrooms.
The program implementation plan was drawn taking into account student English proficiency, their
current Grade level and programmatic technicalities. This year, all three Grades 4-6 are to receive WLE
Level 1 lessons. WLE lessons have been scheduled Monday to Friday across all Grades.
In July 2019, a 50 member team of the Cluster Resource Group of Parbhani District received one round
of training on the technical aspects of the WLE program. Upon completion of their WLE program
orientation, the Cluster Resource Group members conducted similar orientation sessions for the Zilla
Parishad teachers in their Talukas. This enabled the teachers to have sufficient training in anchoring the
WLE program in their classrooms. After the completion of the intended orientation workshops the
program implementation began in schools from August 2019 onwards.
Prior to initiation of the programmatic components a baseline study of the program was conducted in
July 2019 to assess the current level of English proficiency of students.
2.2 Overview of the Baseline Study The baseline study of the program was conducted in July 2019 to assess the current level of English
proficiency of students. The study was conducted in a total of 42 schools from the 9 talukas Gangakhed,
Jintur, Manvat, Palam, Parbhani, Pathri, Purna, Selu and Sonpeth from the Parbhani District. 4 schools
from Gangekhed, 7 schools from Jintur, 3 schools from Manvat, 5 schools from Palam, 7 schools from
Parbhani, 3 schools from Pathri, 6 schools from Purna, 3 schools from Selu and 4 schools from Sonpeth
talukas participated in the study. WLE tests were administered on 4-5 students each belonging to Grade
4, Grade 5 and Grade 6 from these schools.
40 trained investigators from the Cluster Resource Group collected data from these 42 schools. WLE
tests were administered in a one on one fashion. The WLE Level 1 test was taken by 199 students from
Grade 4, WLE Level 2 test was administered to 205 Grade 5 students, and 200 students from Grade 6
took the WLE Level 3 test.
3. Methods
3.1 Research Question The study was aimed at examining the research question, “What is the current level of English language
comprehension and speaking skills in Grade 4, Grade 5 and Grade 6 children of Zilla Parishad Schools?”
3.2 Study Procedures
3.2.1 Sampling Strategy The study was conducted amongst students from schools where the WLE program was to be launched.
A sample of 42 schools were included in the baseline study from a total of 600 Zilla Parishad schools.
The WLE tests were administered on Grade 4, Grade 5 and Grade 6 students. Grade 4 was administered
the WLE test Level 1, Grade 5 was administered test Level 2 and finally Grade 6 students took Level 3
test. About 4-5 students each belonging to Grade 4, Grade 5 and Grade 6 were chosen through alternate
roll number call for participating in the baseline study.
3.2.2 Sample A total of 42 schools were included in the sample. About 4-5 students each belonging to Grade 4, Grade
5 and Grade 6 were chosen through alternate roll number call for participating in the WLE tests. The
details of the sample size for the study are provided in the table given below.
Table 1: Overview of WLE sample size
No of schools
No of school per taluka No of students Grade 4
No of students Grade 5
No of students Grade 6
Level 1 40 Gangakhed-4
Jintur-7
Manvat-3
Palam-3
Parbhani-7
Pathri-3
Purna-6
Selu-3
Sonpeth-4
199 - -
Level 2 40 Gangakhed-4
Jintur-6
Manvat-3
Palam-5
Parbhani-7
Pathri-3
Purna-6
Selu-3
Sonpeth-3
- 205 -
Level 3 40 Gangakhed-4
Jintur-7
Manvat-3
Palam-5
Parbhani-6
Pathri-3
Purna-6
Selu-3
Sonpeth-3
- - 200
3.3 Instruments and Procedure The research question was aimed at understanding the current level of English Language comprehension
and speaking ability of the students. This was accomplished using the WLE Level 1, Level 2 and Level 3
testing tool, which was administered one on one for each selected child. The table below provides an
overview of the questions and the learning outcomes covered in the 3 levels of WLE level tool.
Table 2: Overview of WLE Level 1 testing tool
Question No Questions
Learning outcome tested
1 Answering 10 simple questions in English
Listening comprehension and Speaking skills about oneself
2 Picture description Speaking skills- Speaking using picture cue, present tense and present continuous tense
3
Speaking about a friend Speaking skills – description without cues
Table 3: Overview of WLE Level 2 testing tool
Question No Questions Learning outcome tested
1 Answering 5 questions in English Listening comprehension and Speaking skills about oneself
2 Speaking using picture cue Speaking skills –Use of prepositions in picture description
3
Translating English sentences to Marathi
Listening Comprehension-Past Tense and Future tense
4 Speaking about yourself Speaking skills- description without cuees
Table 4: Overview of WLE Level 3 testing tool
Question No Questions Learning outcome tested
1 Answering 15 questions in English Listening comprehension and Speaking skills about oneself
2 Translating English sentences to Marathi
Listening Comprehension- Compound & complex sentences in past, future and Present
3
Translating Marathi sentences to English
Listening comprehension & Speaking skills-Compound & complex sentences in past , future and Present
4 Picture Description Speaking using picture cue- Action words, Present continuous tense, Preposition
5 Speaking about a teacher Speaking skills – description without cues
3.4 Data Collection Procedure A team of 40 investigators, comprising of members of Cluster Resource Group were involved in data
collection. Prior to data collection, CLR‟s master trainers conducted a one-day orientation workshop for
the investigators. During the orientation workshop the investigators were introduced to the tools,
interview techniques, data collection and entry process. The table below represents the data collection
procedures and tools used to examine the research questions. A discussion of the procedures and tools
in detail follows.
Table 5: Overview of data collection procedure
4.Data Analysis (WLE – Listening and speaking skills) While administering the WLE tests the investigators recorded the exact verbatim of student responses in
the response sheet. These duly filled in response sheets were then scored by the team at Centre for
Learning Resources based on a predetermined coding scheme. The scores were then entered in Excel
and averaged. Below we provide a glimpse into the tools and the data analysis procedure for the WLE
Level 1, Level 2 and Level 3 Tests.
4.1 WLE Level 1 Test The WLE Level 1 test had three questions, as seen in table no. 2. The first question, tested children on
listening comprehension and speaking skills. It had 10 sub-questions. Children received scores out of 10
for the number of questions they could comprehend. These scores were then averaged for all Grade 4
students. To score their speaking skills, their response to each question was categorized as „appropriate
answer in full grammatically correct sentence‟, „appropriate answer in broken sentence‟ and „appropriate
answer in one word‟. Subsequently, for each child speaking scores included „number of appropriate
answers in full grammatically correct sentence‟, „number of appropriate answers in broken sentence‟ and
„number of appropriate answers in one word‟. Each of these scores were then averaged for all Grade 4
students.
Participants in the study WLE (Listening Comprehension and
Speaking ability)
Method
Grade 4 WLE Level 1 One on one interview
Grade 5 WLE Level 2 One on one interview
Grade 6 WLE Level 3 One on one interview
In the second question children were expected to speak as many sentences as possible in English when
describing a picture. This included use of vocabulary such as colors, pronouns and action words in
present continuous tense. The children‟s responses were categorized as „appropriate answer in full
grammatically correct sentence‟, „appropriate answer in broken sentence‟ and „appropriate answer in one
word‟. Subsequently, for each child speaking scores included „number of appropriate answers in full
grammatically correct sentence‟, „number of appropriate answers in broken sentence‟ and „number of
appropriate answers in one word‟. Each of these scores were then averaged for all Grade 4 students.
In the third question children were expected to speak as many sentences as possible telling the
investigator about their friend – his/ her name, where s/he lives, what does s/he do and like doing etc.
Again, the children‟s responses were categorized as „appropriate answer in full grammatically correct
sentence‟ and „appropriate answer in broken sentence‟. Subsequently, for each child speaking scores
included „number of appropriate answers in full grammatically correct sentence‟ and „number of
appropriate answers in broken sentence‟. Each of these scores were then averaged for all Grade 4
students.
4.2 WLE Level 2 Test The WLE Level 2 test had three questions, as seen in table no. 3. The first question, tested children on
listening comprehension and speaking skills. It had 5 sub-questions. Children received scores out of 5
for the number of questions they could comprehend. These scores were then averaged for all Grade 5
students. To score their speaking skills, their response to each question was categorized as „appropriate
answer in full grammatically correct sentence‟, „appropriate answer in broken sentence‟ and „appropriate
answer in one word‟. Subsequently, for each child speaking scores included „number of appropriate
answers in full grammatically correct sentence‟, „number of appropriate answers in broken sentence‟ and
„number of appropriate answers in one word‟. Each of these scores were then averaged for all Grade 5
students.
In the second question, children had to answer 5 questions based on a picture shown to them. The
questions were about the actions of the characters. They were expected to respond to the question using
action words and prepositions. The children‟s responses were categorized as „appropriate answer with
accurate sentence structure‟ and „appropriate answer with inaccurate sentence structure‟. Subsequently,
for each child speaking scores included number of „correct answer with accurate sentence structure‟ and
number of „appropriate answer with inaccurate sentence structure. Each of these scores were then
averaged for all Grade 5 students.
For the third question, children had to comprehend 5 sentences spoken in English by the investigator
and translate them into Marathi using past and future tense. The children‟s responses were categorized as
„appropriate answer with accurate sentence structure‟ and „appropriate answer with inaccurate sentence
structure‟. Subsequently, for each child speaking scores included number of „appropriate answer with
accurate sentence structure‟ and number of „appropriate answer with inaccurate sentence structure. Each
of these scores were then averaged for all Grade 5 students.
In the final question children were expected to speak as many sentences as possible telling the
investigator about their daily routines using past tense. The children‟s responses were categorized as
„appropriate answer in full grammatically correct sentence‟, „appropriate answer in broken sentence‟ and
„appropriate answer in one word‟. Subsequently, for each child speaking scores included „number of
appropriate answers in full grammatically correct sentence‟, „number of appropriate answers in broken
sentence‟ and „number of appropriate answers in one word‟. Each of these scores were then averaged for
all Grade 5 students.
4.3 WLE Level 3 Test The WLE Level 3 test had five questions, as seen in table no. 4. The first question, tested children on
listening comprehension and speaking skills. It had 15 sub-questions. Children received scores out of 15
for the number of questions they could comprehend. These scores were then averaged for all Grade 6
students. To score their speaking skills, their response to each question was categorized as „appropriate
answer in full grammatically correct sentence‟, „appropriate answer in broken sentence‟ and „appropriate
answer in one word‟. Subsequently, for each child speaking scores included „number of appropriate
answers in full grammatically correct sentence‟, „number of appropriate answers in broken sentence‟ and
„number of appropriate answers in one word‟. Each of these scores were then averaged for all Grade 6
students.
For the second question, children had to comprehend 5 sentences in spoken in English by the
investigator and translate them into Marathi using compound and complex sentences in past, future and
present tense. The children‟s responses were categorized as „appropriate answer with accurate sentence
structure‟ and „appropriate answer with inaccurate sentence structure‟. Subsequently, for each child
speaking scores included number of „appropriate answer with accurate sentence structure‟ and number
of „appropriate answer with inaccurate sentence structure. Each of these scores were then averaged for
all Grade 6 students.
In the third question, children had to comprehend 5 sentences in spoken in Marathi by the investigator
and translate them into English using compound and complex sentences in past, future and present
tense. The children‟s responses were categorized as „appropriate answer with accurate sentence structure‟
and „appropriate answer with inaccurate sentence structure‟. Subsequently, for each child speaking scores
included number of „appropriate answer with accurate sentence structure‟ and number of „appropriate
answer with inaccurate sentence structure. Each of these scores were then averaged for all Grade 6
students
For the fourth question children were expected to speak as many sentences as possible in English when
describing a picture. This included use of vocabulary such as colors, pronouns, prepositions and action
words in present continuous tense. The children‟s responses were categorized as „appropriate answer in
full grammatically correct sentence‟, „appropriate answer in broken sentence‟ and „appropriate answer in
one word‟. Subsequently, for each child speaking scores included „number of appropriate answers in full
grammatically correct sentence‟, „number of appropriate answers in broken sentence‟ and „number of
appropriate answers in one word‟. Each of these scores were then averaged for all Grade 6 students.
In the final question children were expected to speak as many sentences as possible telling the
investigator about their teacher. The children‟s responses were categorized as „appropriate answer in full
grammatically correct sentence‟, „appropriate answer in broken sentence‟ and „appropriate answer in one
word‟. Subsequently, for each child speaking scores included „number of appropriate answers in full
grammatically correct sentence‟, „number of appropriate answers in broken sentence‟ and „number of
appropriate answers in one word‟. Each of these scores were then averaged for all Grade 6 students.
5. Findings
5.1 Grade 4 results on WLE Test Level 1
Graph 1: WLE Level 1 Test - Grade 4
In the first question as part of the listening comprehension and speaking test, Grade 4 children were
asked 10 questions about themselves and their family in English. On an average they were able to
comprehend around 6 questions. While answering these questions, they were able to provide almost four
correct answers in one word. Their performance on the rest of the Level 1 questions indicates extremely
poor speaking skills.
5.2 Grade 5 results on WLE Test Level 2
Graph 2: WLE Level 2 Test - Grade 5
In the first question as part of the listening comprehension and speaking test, when Grade 5 children
were asked 5 questions about themselves and their family in English, on an average they were able to
comprehend 2-3 questions. Further, they were able to provide correct answers in one word for about 2
of these questions. Level 2 questions under the first section of listen, comprehend and speak are
comparatively more difficult than Level 1 questions under the same section, which explains the much
lower scores of Grade 5 students. Their attempts at the rest of the Level 2 questions is poor.
5.3 Grade 6 results on WLE Test Level 3
Graph 3: WLE Level 3 Test - Grade 6
As part of the basic listening comprehension and speaking on the Level 3 test, when Grade 6 children
were asked 15 questions about themselves and their family in English, on an average they were able to
comprehend close to eight questions. Interestingly, they also answered 6-7 of these questions in one
word. However, similar to the trend seen in Grade 4 and Grade 5 students, Grade 6 students were also
not very successful in answering the subsequent questions which checked their authentic speaking skills
and comprehension.
6. Conclusion
Based on WLE test results it can be concluded that students possess very rudimentary English listening
comprehension and speaking skills. The ability to comprehend familiar expressions and situations and
the more complex skills, of speaking in whole sentences and accurately were found to be lacking. It is
clear that students have very little exposure to English reading and writing in the normal school course
work.
In order to reach the desired levels of proficiency these students will require a structured English
Learning, and Speaking course and a sustained exposure over a substantial period of time. The WLE
Program, which builds familiarity with the language and helps develop, listening comprehension and
speaking skills is suitable for this target group.
The program implementation plan needs to be designed taking into account student English proficiency,
their current Grade level and programmatic technicalities. Based on the results, it is recommended that
this year, all three Grades 4-6 receive Level 1 lessons.
Top Related