1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do...
-
Upload
polly-daniels -
Category
Documents
-
view
213 -
download
0
Transcript of 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do...
![Page 1: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/1.jpg)
1
LANGUAE TEST RELIABILITY
![Page 2: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/2.jpg)
2
What Is Reliability?
Refer to a quality of test scores, and has to do with the consistency of measures across different time, test form, raters, and other characteristics of the measurement context.
![Page 3: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/3.jpg)
3
Con. Test reliability is related to high variance
of the true score distribution. (person separability)
reliability is a measure of accuracy, consistency, dependability or fairness of scores resulting from administration of the particular examination.
![Page 4: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/4.jpg)
4
The Measurement Model Observed score= True score + Error score X = T + E Observed score: a score that a test taker actually
received on a test. (Raw or Obtained score).True Score: as there is always some error in any
measurement an individual true score on a test would be his observed score minus some error.
T = X - E
![Page 5: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/5.jpg)
5
Standard Error of Measurement The standard error of measurement (SEM)
is an estimate of error to use in interpreting an individual’s test score.
SEM = s 1 – r)
S = the standard deviation for the test
r = the reliability coefficient for the test
![Page 6: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/6.jpg)
6
Standard Error of Measurement
For example, A test has a split-half reliability
coefficient of .96 and a standard deviation of 15 calculate the SEM for this test.
![Page 7: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/7.jpg)
7
Standard Error of Measurement
SEM = s ( 1 – r ) =
15 ( 1-.96) = 15 .04
= 15 x .2 = 3
![Page 8: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/8.jpg)
8
Threat to test Reliability
Sources of Error What are some of the factors that introduce error into
measurement? 1) Student Factors 2) Construction of the Items 3) Test administration- 4) Scoring 5) Length, difficulty and boundary effect of the Test 6) Regulatory Fluctuation 7)Discriminability, Speediness, and Homogeneity 8)Fluctuation in Response
![Page 9: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/9.jpg)
9
Sources of Error
(1) Student Factors--Student fatigue, illness, or anxiety can induce error and lower reliability because they affect performance and keep a test from being a measure of their true ability or achievement.
2) Construction of the Items -- A major threat to reliable measurement is poorly worded or ambiguous questions or tricky questions.
![Page 10: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/10.jpg)
10
Sources of Error 3) Test administration--Environmental
factors such as heat, light, noise, confusing directions, and different testing time allowed to different students can affect students' scores.
![Page 11: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/11.jpg)
11
Sources of Error 4) Scoring – An objective test is more reliable because the test
scores reflect true differences in achievement among students and not the judgment and opinions of the scorer.
subjectivity in score or mechanical errors in scoring process may introduce inconsistency in score and produce unreliable measurement, that usually occur with in or between the rater themselves.
![Page 12: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/12.jpg)
12
Scoring A. Intra- Rater Reliability(mark/er-mark
reliability) (Bachman, 1990) when an individual subjectively judges or rates
the adequacy of a given sample of language performance for at least two times and gives consistent results, we say that this rating have intra- rater reliability.
B. Inter-rater reliability Which refers to consistency of rating given by
different raters to a sample of language performance.
![Page 13: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/13.jpg)
13
Sources of Error (5) length, difficulty and boundary effect of the
Test A- reliability is affected by number of item in the
test. More items in the test make a grater range of
score and grater reliability. B- A test that is either too easy or too difficult for
the class taking it will typically have low reliability. This occurs because the scores will be clustered together at either the high end or the low end of the scale, with small differences among students( boundary effect).
![Page 14: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/14.jpg)
14
Sources of Error (6) Regulatory Fluctuation –Differences in
the clarity of instructions, the time of test administration, test administrator interaction with examinees, prevention of cheating behavior, and reporting of time remaining are all potential source of measurement error.
![Page 15: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/15.jpg)
15
Sources of Error (7)Discriminability, Speediness, and
Homogeneity A- Discriminability: the degree to which a test or an item of the
test distinguishes among stronger and weaker test taker.
Great discriminate = Great reliability
![Page 16: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/16.jpg)
16
Sources of Error B- Speediness: Speed test: A test in which the items are easy but
the time limits are so short that a few or non of the test takers can complete all the items. such a test aims to determining the speed of the testees to do certain task
Power test :A test in which item difficulty generally increase gradually but ample time is given to all candidate. The aim is determine how much an individual is able to do, not how rapidly.
![Page 17: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/17.jpg)
17
Sources of Error In power test failure to allow examinees a
reasonable amount of time to complete the test will reduce the reliability.
If the test becomes more difficult as a result of the element of speedness , reliability will diminish.
C- Homogeneity We can increase reliability and reduce error by
including items of similar format and content.(e.g split half method)
![Page 18: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/18.jpg)
18
Sources of Error (8)Fluctuation in Response
A- Response arbitrariness B- Wiseness and familiarity Response
![Page 19: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/19.jpg)
19
Methods of Reliability Computation
The choice of the method of computation of reliability will depend on such factor as
Nature of threats to reliability present Ease of computation Nature of the test Testing situation
![Page 20: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/20.jpg)
20
Methods of Reliability Computation
Test-Retest Method Parallel Form Method Inter – Rater Reliability Split Half Reliability KR-20 KR-21
![Page 21: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/21.jpg)
21
Methods of Reliability Computation
1-Test-Retest Method Refer to correlation of two sets of score for
the same persons. An approach to estimating reliability in
which we administer the test twice to the group of individuals and then compute the correlation between two sets of scores .
R= r1,2
![Page 22: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/22.jpg)
22
Methods of Reliability Computation
Test-Retest Method disadvantage: 1-time consuming. it is difficult to arrange
two testing session an preparing similar condition for the same group of examinee.
2- test effect .students may learn or memorize some question
![Page 23: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/23.jpg)
23
Methods of Reliability Computation
2-Parallel Form Method Two tests of the same ability, and with
equal length and difficulty that are administrated to the same sample of persons.
disadvantage: constructing two parallel forms of a test
is not an easy task.
![Page 24: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/24.jpg)
24
Methods of Reliability Parallel Form Method
Equated test: Any two sets of scores from
different test (assuming that the same trait is being tested) that have been reduced to a common scale to facilitate comparison.
![Page 25: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/25.jpg)
25
Methods of Reliability Parallel Form Method
Random parallel tests: It has been used to described tests that have
been composed of items drawn randomly from the same population of items
ru = rA,B ru = the reliability coefficient rA,B = the correlation of form A with the
form B of the test
![Page 26: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/26.jpg)
26
Methods of Reliability Computation
3-Inter – Rater ReliabilityEstimation based on the correlation of scores
between/among two or more raters who rate the same item, scale, or instrument .
the actual level of reliability will depend on number of raters or judges.
the more rater present in the determination of the mark, the more reliable will be the mark.
![Page 27: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/27.jpg)
27
Intra- Rater Reliability(mark/er-mark reliability) (Bachman, 1990)
when an individual subjectively judges or rates the adequacy of a given sample of language performance for at least two times and gives consistent results, we say that this rating have intra- rater reliability.
![Page 28: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/28.jpg)
28
Methods of Reliability Inter – Rater Reliability
there are two steps in the estimation of inter-rater reliability:
1-an average of all correlation coefficients 2-Spearman Brown Prophecy Formula
![Page 29: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/29.jpg)
29
Methods of Reliability Computation
4-Split Half Reliability Obtained from a single administration by
dividing the tests into two comparable halves and comparing the resulting scores for each individual (split into odds and evens).
an approach to estimating the internal consistency of a test.
![Page 30: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/30.jpg)
30
Methods of Reliability Split Half Reliability
disadvantage: a- reliability can be change according to the
manner in which the test is divided. (split into odds and evens)
b- homogeneous item. Because assuming the equality between the two halves is not always the safe assumption.( different subsection , in a test e.g. grammar , vocab, reading,…will change test homogeneity and thus reduce the test score reliability )
![Page 31: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/31.jpg)
31
Methods of Reliability Split Half Reliability
advantage: it is more practical than other. Because: 1-no need to administer the same test
twice. 2-not necessary to develop two parallel
forms of the same test. 3-single administration will be enough .
![Page 32: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/32.jpg)
32
Methods of Reliability Split Half Reliability
Spearman Brown Prophecy Formula.
e.g.
if the reliability coefficient of half of the test is computed to be 0.80 .what would be the reliability of the total test?
![Page 33: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/33.jpg)
33
Methods of Reliability Split Half Reliability
it should be logically clear that the reliability of the total test will always be higher than the reliability of half of the test.
![Page 34: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/34.jpg)
34
Methods of Reliability Computation
5-KR-20 Kuder-Richardson Formula 20 Permit
us to arrive at the same final estimate of reliability without having to compute reliability estimates for every possible split half combination.
![Page 35: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/35.jpg)
35
Kuder-Richardson Formula 20It is based on number of item on the test = n or K difficulty of the individual items variance of the total test score = V
![Page 36: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/36.jpg)
36
Methods of Reliability Computation
6-KR-21 Kuder-Richardson Formula 21 is a formula
that is easier to use but less accurate than KR 20.
This formula is based on the assumption that all item in the test are designs to measure a single trait.
![Page 37: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/37.jpg)
37
Kuder-Richardson Formula 21 This formula, known as KR-21
![Page 38: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/38.jpg)
38
Kuder-Richardson Formula 21 e.g. Suppose we gave a 50-item test and
the mean score was 43 and the variance was 25 Putting these values into KR-21.
K= 50 X= 43 V= 25
![Page 39: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/39.jpg)
39
Kuder-Richardson Formula 21
Solving for r obtains: r= (1.02) (0.76) = 0.78 the reliability coefficient is greater than 0.70, so we can
use this test with some degree of confidence.
![Page 40: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/40.jpg)
40
Correction for Attenuation Henning,1987 A way of holding reliability constant when
making comparison among correlation coefficient . It is made by dividing the correlation coefficient by the square root of the cross-product of reliability.
![Page 41: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/41.jpg)
41
Correction for Attenuation
E.g If a test of composition writing correlated 0.55 with the test
of grammar usage , disattenuate this correlation, assuming that KR20 reliabilities of the tests were 0.70 for composition writing and 0.80 for grammar usage .
![Page 42: 1 LANGUAE TEST RELIABILITY. 2 What Is Reliability? Refer to a quality of test scores, and has to do with the consistency of measures across different.](https://reader036.fdocuments.in/reader036/viewer/2022062807/5697c00c1a28abf838cc916c/html5/thumbnails/42.jpg)
42
Correction for Attenuation