Basic Testing Terminology

39
Lecturer: Yee Bee Choo IPGKTHO

description

Topic 3 Language Assessment (TSL3123)

Transcript of Basic Testing Terminology

Page 1: Basic Testing Terminology

Lecturer: Yee Bee Choo

IPGKTHO

Page 2: Basic Testing Terminology

Types

of Test

Norm-

Referenced

and

Criterion-

Referenced

Objective

and

Subjective

Formative

and

Summative

YBC

Page 3: Basic Testing Terminology

According to Brown (2010), in NRTs an individual test-taker’s score is interpreted in relation to a mean (average score), median (middle score), standard deviation (extent of variance in scores), and/or percentile rank.

The purpose of such tests is to place test-takers along a mathematical continuum in rank order.

In a test, scores are commonly reported back to the test-taker in the form of a numerical score for example, 250 out of 300 and a percentile rank for instance 78 percent, which denotes that the test-taker’s score was higher than 78 percent of the total number of test-takers but lower than 22 percent in the administration.

YBC

Page 4: Basic Testing Terminology

YBC

Page 5: Basic Testing Terminology

NRT is administered to compare an individual performance with his peers’ and/or compare a group with other groups.

In the School-Based Evaluation, NRT is used for the summative evaluation, such as in the end of the year examination for the streaming and selection of students.

YBC

Page 6: Basic Testing Terminology

Gottlieb (2006) refers Criterion-referenced tests as the collection of information about student progress or achievement in relation to a specified criterion.

In a standards-based assessment model, the standards serve as the criteria or yardstick for measurement.

YBC

Page 7: Basic Testing Terminology

Following Glaser (1973), the word ‘criterion’ means the use of score values that can be accepted as the index of attainment to a test-taker.

Thus, CRTs are designed to provide feedback to test-takers, mostly in the form of grades, on specific course or lesson objectives.

YBC

Page 8: Basic Testing Terminology

YBC

Page 9: Basic Testing Terminology

Curriculum Development Centre (2001) defines CRT as an approach that provides information on student’s mastery based on the criteria determined by the teacher.

These criteria are based on learning outcomes or objectives as specified in the syllabus.

YBC

Page 10: Basic Testing Terminology

The main advantage of CRTs is that they provide the testers to make inferences about how much language proficiency, in the case of language proficiency tests, or knowledge and skills, in the aspect of academic achievement tests, that test takers/students originally have and their successive gains over time.

As opposed to NRTs, CRTs focus on student’s mastery of a subject matter (represented in the standards) along a continuum instead of ranking student on a bell curve.

YBC

Page 11: Basic Testing Terminology

Norm-Referenced Test Criterion-Referenced Test

Definition A test that measures student’s achievement as compared to other students in the group

An approach that provides information on student’s mastery based on a criterion specified by the teacher

Purpose •Determine performance difference amongindividual and groups•Foster learning

•Determine learning mastery based on specified criterion and standard•Classify/ Group students

Test Item From easy to difficult level and able to discriminateexaminee’s ability

Guided by minimum achievement in the related objectives

YBC

Page 12: Basic Testing Terminology

Norm-Referenced Test Criterion-Referenced Test

Group size Large group Relatively small group

Range of tested ability

Wide range of abilities Relatively homogenous

Text length Large number of questions Few questions

Time allocation

Long (2-4 hours) administration

Relatively short time

Cost Test booklet, delivery fee, invigilation and examiner’s fee

Teacher-made, duplication

YBC

Page 13: Basic Testing Terminology

Norm-Referenced Test Criterion-Referenced Test

Expectation Do not know content Know content to expect

Levels of Generality

Overall/ Global Classroom, specific

Types of Test

Aptitude, Proficiency, Placement

Diagnostic, progress, achievement

Appropriateness

Summative evaluation Formative evaluation

Example Public exams: UPSR, SPM and STPM

Mastery test: monthly test, coursework, project,exercises

Score report strategies

Only scores go to students Scores and answers go to students

YBC

Page 14: Basic Testing Terminology

Formative test is a kind of feedback teachers give students while the course is progressing.

Formative assessment can be seen as assessment for learning. It is part of the instructional process.

We can think of formative assessment as “practice.”

With continual feedback the teachers may assist students to improve their performance.

The teachers point out on what the students have done wrong and help them to get it right.

YBC

Page 15: Basic Testing Terminology

Based on the results of formative test, the teachers can suggest changes to the focus of curriculum or emphasis on some specific lesson elements.

Students may also need to change and improve.

Due to the demanding nature of this formative test, numerous teachers prefer not to adopt this test although giving back any assessed homework or achievement test present both teachers and students healthy and ultimate learning opportunities.

YBC

Page 16: Basic Testing Terminology

Summative test refers to the kind of measurement that summarise what the student has learnt or give a one-off measurement.

Summative assessment is assessment of student learning.

Students are more likely to experience assessment carried out individually where they are expected to reproduce discrete language items from memory.

The results then are used to yield a school report and to determine what students know and do not know.

YBC

Page 17: Basic Testing Terminology

It does not necessarily provide a clear picture of an individual’s overall progress or even his/her full potential, especially if he/she is hindered by the fear factor of physically sitting for a test, but may provide straightforward and invaluable results for teachers to analyse. It is given at a point in time to measure student achievement in relation to a clearly defined set of standards, but it does not necessarily show the way to future progress.It is given after learning is supposed to occur. End of the year tests in a course and other general proficiency exams are some of the examples of summative tests.

YBC

Page 18: Basic Testing Terminology

Summative Test Formative Test

It measures students’ competency

It improves individualIt provides feedback to students

It assess students’ overall performance

It assess students’ mastery and understanding of skills

End of instruction Before or during instruction

End of year On-going throughout the year

Aligned to content area state standards

Linked to learning experience

Final exams Entrance exams

Anecdotal recordsQuizzes and essaysDiagnostic tests

YBC

Page 19: Basic Testing Terminology

According to BBC Teaching English, an objective test is a test that consists of right or wrong answers and thus it can be marked objectively.

Objective tests are popular because they are easy to prepare and take, quick to mark, and provide a quantifiable and concrete result.

They tend to focus more on specific facts than on general ideas and concepts.

YBC

Page 20: Basic Testing Terminology

The types of objective tests include the following:

i. Multiple-choice items/questions

ii. True-false items/questions:

iii. Matching items/questions; and

iv. Fill-in the blanks items/questions.

YBC

Page 21: Basic Testing Terminology

Multiple-choice items are all receptive, or selective, response items in that the test-takers chooses from a set of responses rather than creating a response.

Multiple-choice question is select type where students are expected to select or choose answer from a list of options

Fill-in the blank question is supply type where the students are expected to supply the answer by giving the correct word or phrases because no optional responses are provided.

YBC

Page 22: Basic Testing Terminology

Stimulus Item Options Key DistractorsMCQ

item

Terminology in Multiple-Choice Question (MCQ):

YBC

Page 23: Basic Testing Terminology

Terminology in MCQ:

1. Stimulus

Stimulus can be used with a multiple choice item, something like a bar graph, a table, a map, a short text, etc.

YBC

Page 24: Basic Testing Terminology

Terminology in MCQ :

2. Stem

Every multiple-choice item consists of a stem (the ‘body’ of the item that presents a stimulus).

Stem is the question or assignment in an item.

It is in a complete or open, positive or negative sentence form.

Stem must be short or simple, compact and clear. However, it must not easily give away the right answer.

YBC

Page 25: Basic Testing Terminology

Terminology in MCQ :

3. Options or alternatives

They are known as a list of possible responses to a test item.

There are usually between three and five options/alternatives to choose from.

YBC

Page 26: Basic Testing Terminology

Terminology in MCQ :

4. Key

This is the correct response.

The response can either be correct or the best one.

Usually for a good item, the correct answer is not obvious as compared to the distractors.

YBC

Page 27: Basic Testing Terminology

Terminology in MCQ :

5. Distractors

This is known as a ‘disturber’ that is included to distract students from selecting the correct answer.

An excellent distractor is almost the same as the correct answer but it is not.

YBC

Page 28: Basic Testing Terminology

1. Which city would you go to if you want to experience snowfall?

STIMULUS

STEM

KEY

DISTRACTORS

COMPONENTS OF MCQ ITEM

World Weather : Min[C] Max [C]Amsterdam 12 17 partly cloudyBeijing 14 32 clearGeneva 11 16 rainToronto -01 08 partly cloudyJohannesburg 09 17 clearLos Angeles 10 21 cloudyMexico City 13 20 partly cloudy

A. Geneva B. Johannesburg C. Toronto D. Beijing

OPTIONS

Objective Test

YBC

Page 29: Basic Testing Terminology

CRITERIA OF A GOOD ITEM BUILDER

Understands:

Syllabus Specifications

Current circulars

Exam format

Textbook

Table of Specifications (JSU)

Page 30: Basic Testing Terminology

Guidelines when constructing MCQ:

1. Design an item to measure an important learning outcome.

2. Present a single, clearly formulated problem in the stem.

3. Stem – simple, clear language

4. Put as much of the wording as possible in the stem.

YBC

Page 31: Basic Testing Terminology

Guidelines when constructing MCQ:5. State the item in the positive form

wherever possible. 6. Emphasise negative wording whenever it is

used in the stem. Use: caps, underlining, placing near the end of the stem etc.

7. Intended answer – correct, the best8. Make all the alternatives grammatically

consistent.

YBC

Page 32: Basic Testing Terminology

Guidelines when constructing MCQ :

9. Avoid verbal clues which might enable students to select the correct answer or to eliminate an incorrect alternative.

a. Similarity of wording in both the stem and the key/correct answer.

b. Stating the correct answer in textbook language or sterotype phraseology.

c. stating the correct answer in greater detail.

d. Including absolute terms in the distractors –use of ‘never’, ‘all’, none’, ‘always’ etc.

YBC

Page 33: Basic Testing Terminology

Guidelines when constructing MCQ :

10. Make the distractors plausible and attractive to the

informed.

a. Use common misconceptions and/or errors as distractors.

b. State the alternatives in the language of the students.

c. Use good sounding words e.g. ‘accurate’, ‘important’ etc. in the distractors as well as the correct answer.

d. Make the distractors similar.

e. Do not use extraness/irrelevant unrelated clues in the distractors.

f. Make the alternatives homogenous.

YBC

Page 34: Basic Testing Terminology

Guidelines when constructing MCQ :

11. Vary the length of the correct answer to eliminate length as a clue.

12. Use the alternatives, ‘all of the above’ and ‘none of the above’ with extreme caution.

13. Vary the position of the correct answer in a random manner.

14. Control the difficulty of the items either by varying the problem in the stem or by changing the alternatives.

15. Make certain each item is independent.

16. Use an efficient item format for ease of scoring.

YBC

Page 35: Basic Testing Terminology

Advantages in using multiple-choice items:

versatility in measuring all levels of cognitive ability.highly reliable test scores.scoring efficiency and accuracy.objective measurement of student achievement or ability.a wide sampling of content or objectives.a reduced guessing factor when compared to true-false items. different response alternatives which can provide diagnostic feedback.

YBC

Page 36: Basic Testing Terminology

Weaknesses of multiple-choice items (Brown, 2004):This technique tests only recognition knowledgeGuessing may have a considerable effect on test scoresThis technique severely restricts what can be testedIt is very difficult to write successful itemsWashback may be harmfulCheating may be facilitated

Difficult and time-consuming

YBC

Page 37: Basic Testing Terminology

Contrary to an objective test, a subjective test is evaluated by giving an opinion, usually based on agreed criteria.

Subjective tests include essay, short-answer, vocabulary, and take-home tests.

Some students become very anxious of these tests because they feel their writing skills are not up to par.

YBC

Page 38: Basic Testing Terminology

In reality, a subjective test provides more opportunity to test-takers to show/demonstrate their understanding and/or in-depth knowledge and skills in the subject matter.

In this case, test takers might provide some acceptable, alternative responses that the tester, teacher or test developer did not predict.

Generally, subjective tests will test the higher skills of analysis, synthesis, and evaluation.

In short, subjective test will enable students to be more creative and critical.

YBC

Page 39: Basic Testing Terminology

Objective Test Subjective Test

Require students to select a correct answers from several options or supply a word or short phrase to answer a question

Require students to write and present an original answer

Only one correct answer More than one correct answer

Facts and data Opinions and interpretations

MCQ, True/False, Fill-in the blanks, Matching

Short-answer essay, essay, problem solving, extended-response questions

Fair and free of bias Open to interpretation

Short time is needed Longer time is needed

Difficult to construct the question

Easy to construct the questions

YBC