St. Anthony’s School 16 Jan 2013 1. Background information 2.
-
Upload
mikaela-newborn -
Category
Documents
-
view
220 -
download
2
Transcript of St. Anthony’s School 16 Jan 2013 1. Background information 2.
St. Anthony’s School
16 Jan 2013 1
Background information
2
Enhance the effectiveness of learning and teaching
3
Evaluation after Assessment
Why did students
choose A, not B, C or D?
What are the follow-up actions?
Are the questions
too difficult or too easy?
Have the students learned? Have they
studied hard?
4
5
Student MCIItem MCICronbach AlphaGraph shape
Data – Good to start
6
2 Modified Caution IndexesStudent Modified Caution Index
Item Modified Caution Index
Students’ Performance
Question items’
Performance
7
Student MCIxP Chart
2012-13 P.4 1st
Reading Exam
8
Student MCIxP Chart
2012-13 P.4 1st
Reading Exam
9
More focused strategiesType A Type B
The teaching and learning are suitable for them. Encourage them to keep on.
Know the difficult questions Miss the simple questionsCareless mistakesDon’t rely on luck
Type D Type C
Adjust the teaching contentRaise their learning motivation Need more consolidation
Answering techniquesCareless mistakesFocus on the foundationDon’t rely on luck
10
Item MCI xP Chart
2012-13 P.4 1st
Reading Exam
11
Item MCI xP Chart
Need improveme
nt
Difficult question
s
Problem question
s
12
MCI item P-value chartItem Type
1112 P.3 Final Exam
1213 P.4 First Exam
A 20 17B 1 2C 1 0D 3 7Total no. of questions
25 26According the Item Modified Caution Index, most of the items set are normal.
13
Cronbach Alpha (Reliability)
Cronbach Alpha
Reliability
0 – 0.3 low
0.7-0.9 acceptable
> 0.9 high
14
2011-2012 P.3 First Exam
Slightly Easy
15
2012-2013 P.4 First Exam
Appropriate
16
Which question
items should be improved?
17
Which question items are difficult to
our students?
18
SP Xpress analysis on ‘2011-2012 Primary 3 Third Exam Paper’SP Xpress analysis results:
Type A: 20 itemsType B: 1 itemType C: 1 itemType D: 3 items
Let’s examine one item from each category (Type B, C, D)
19
Exam item example: Type B
On which floor of the centre, can Grandpa find presents for Jack?A. 1/F B. 2/F C. 3/F D.4/F
20
Possible reasons for item being Type BType B: Item is not difficult but requires
amendments because many students couldn’t answer it.
Possible reasons:Item design flawsNot close to what teacher taught in classStudents lack specific training in reading
comic strips
21
Teacher examined the Type B itemStudents found it difficult because …
Need to process and connect many ideas all at the same timeQuestion Comic strip
Which floor? 1/F,2/F,3/F,4/Ffloor directory
… of the Centre … Star Shopping Centre
… can Grandpa find presents …
We can buy all the gifts …
presents? presentsrobotstoys(4/F)
… can Grandpa … ‘ We’ refers to ‘grandparents’
22
Recommended solutionsIncrease students’ chances in reading
English comic strips in classroom teaching or leisure reading
Teacher needs to illustrate more clearly in connecting ideas
Teacher needs to teach more explicitly which means more visual demonstrations
23
Exam item example: Type CIn which step does Jack’s mum add ‘soy sauce’ and ‘wasabi’ to the sushi?A. Step 1 B. Step 3 C. Step 6 D. Step 10
24
Possible reasons for item being Type CType C: Item is difficult and requires
amendments because many students couldn’t answer it.
Possible reasons:Confusing descriptionQuestion misleading studentsQuestion containing error(s)
25
Teacher examined the Type C itemStudents did not have the relevant personal
experience in making and/or eating sushiThe recipe did not mention dipping sushi into ‘soy
sauce’ or ‘wasabi’ after the finishWithout experience, students could not interpret the
correct answer which is the final stepThis question contains the error which requires
outside knowledge uncommon among the students
26
Exam item example: Type DAfter cutting the vegetables and crab meat, Jack’s mum ____.C.takes some rice wine vinegar and mixes it with the sushi rice
This is the answer but the option rephrased the instructions.
27
Features of Type D itemType D: Item is difficult and doesn’t require
amendments. It is suitable for screening the high ability students. Low and mid- ability students can’t answer it.
Type D items should be kept at a low proportion in an assessment.
The rephrasing is difficult for students to interpret the correct answer.
28
Analysis of TSA P.3 Reading PaperPredict P.3 students’ results in the upcoming TSA
reading assessmentCompare the overall students’ ability from different
yearsExamine the neglected elements in our teaching Assumption: TSA assessment covers the basic
competency that all students are supposed to have acquired by the end of a learning stage
29
SP Xpress analysis on ‘2011 TSA P.3 Reading Paper (3ER1)’
2011-2012 P.3
2012-2013 P.3
Mean 19.28 18.95
SD 4.78 4.76
Type A 18 20
Type B 6 3
Type C 0 1
Type D 2 230
An Interesting Fact: A common Type B item across 2 yearsPart 1B Question #1
How many chapters are there in this book?‘Chapter / Table of Contents’ can be an obstacleStudents never encounter ‘Chapter’ in storybook
teaching (replacing textbook teaching)
31
An Interesting Fact: A common Type B item across 2 years Sam buys two sandwiches with the Rainbow Fast Food
coupon. How many sandwiches can he get?C. three sandwiches (= Buy 2 + 1 Free)
Students are not familiar with ‘Buy 2 Get 1 Free’ concept even though they can understand every single word
32
An Interesting Fact: A common Type D item across 2 years “ … too. In the end, John wins the race. All the
people cheer loudly …” Read lines 11-12. What do the people do when John
wins the race?C. shout happily
Students are not familiar with synonyms
cheer loudly
shout happily33
Personal reflection1. Teachers become aware of the quality of the
assessments Accumulate professional experience in setting
better assessments
2. Teachers prepared for the unexpected and unlearned
Teachers become cautious and adjust their teaching
34
Things to do Suggest Ts reflect on their exam paper
using SP Xpress Facilitate Ts process the data with
technical supportsSet up a school based question bankNurture a culture on giving a more
constructive feedback to both Ts and Ss after assessment
35
Thank you so much for your time!