Testing and Assessing Final presentation.pdf

8/14/2019 Testing and Assessing Final presentation.pdf

1/46


2/46

Objectives

1. To define the basic concepts of languagetesting

2. To specify the reasons for testing3. To identify the basic principles of languagetesting

4. To discuss the different types and forms ofassessment5. To identify the test development procedure

Fundamentals of Language Assessment 11/11/2013


3/46

Outline:

Fundamentals of Language Assessment

1. Concept defining :a. Testingb. Assessmentc. Evaluation

d. Measurement2. Purposes for Assessment3. Cornerstones of Assessment: Criteria of a good

Test4. Types of Assessment5. Test Development Procedure

11/11/2013


4/46

Evaluation

Assessment

Testing

Figure 1: Relationships among Evaluation, Assessment and Tests



5/46

Concept Defining

Evaluation ?

Evaluation is a systematic

analysis of the differentaspects ofan educationalprogram with the aim ofimproving its quality.

Evaluation is theprocess of making

judgments based oncriteria and evidenceand taking decisions .

Evaluation can be defined as thesystematic gathering of information forthe purpose of making decisions (Weiss1972).

Evaluation: The processof determining the worthof something in relation toestablished benchmarksusing assessmentinformation



6/46

Concept Defining

Assessment is the process ofgathering and discussinginformation from multiple anddiverse sources in order todevelop a deep understandingof what students know, understand,and can do with their knowledge asa result of their educationalexperiences; the process culminates

when assessment results are used toimprove subsequent learning .

Huba, M. E. & Freed, J. E. (2000). (p. 8)

gathering, recording,

interpreting, using andreporting information about achild s progress and achievementin developing knowledge, skills

and attitudes

Assessment in the Primary School Curriculum:Guidelines for Schools, p. 7, NCCA, 2007

Assessment?



7/46

Concept Defining

Assessment?The process of gatheringinformation to monitorprogress and make educationaldecisions .It includes testing but it mayalso include other methodssuch as observation,

interviews, peer assessment,project work , portfolios

Systematic collection andanalysis of information toimprove educationalpractice

Method forunderstanding studentlearning



8/46

Concept Defining

TestingA method of measuring apersons ability , knowledgeor performance in a givendomain

(Brown, 2004)

A test is a sample ofbehavior, products,answers, or performancesfrom a particular domain(Carrington, 1994)

it's a systematic method ofeliciting performance which isintended to be the basis forsome sort of decision making "(Hughes, 1989).

Testing is generally

concerned with turningperformance intonumbers.

(Baxter, 1997)Fundamentals of Language Assessment 11/11/2013


9/46

Figure 2 : Relationships among measurement, tests, andevaluation. Bachman (1990)

Concept Defining



10/46

Concept Defining


Measurement andtest involve thequantification ofobservations.Tests are a type ofmeasurement designedto elicit a specificsample of behavior.

( Bachman 1990)

MeasurementMeasurement : assignmentof numbers (quantity), usesvariety of instrument: test,rating scale. It is theprocess of obtainingnumerical description ofthe degree of individualpossesses. Quantifying ofhow much does a learnerlearned.

11/11/2013


11/46

Summary

Evaluation: The placing of value or interpretation of ameasurement. Evaluation encompasses the inferences thatare drawn from measurementsAssessment: A general term used to denote the systematiccollection and interpretation of data that is to be used in

the making of educational decisions, including enhancinginstruction.Test: An instrument or formal process that presents tasksthat yields a specific type of measurement.

Measurement: The explicit quantification of the resultsobtained through testing that is designed to determine thedegree of presence or absence of a particularcharacteristic, quality, or feature.



12/46



13/46

Purposes for AssessmentAssigning grades or ranks: Assessing students provides the teacherwith at least one type of evidence for grading, graduation, admission.Provide feedback on student learningDetermining one's own instructional effectiveness.Diagnose students strengths and needs

Provide a basis for instructional placementInform and guide instructionCommunicate learning expectationsMotivate and focus students attention and effort

Provide practice applying knowledge and skillsMonitoring students' progressInfluencing public perceptions of educational effectiveness. Students'scores on standardized achievement tests are often used to informthe public about the effectiveness of schools.



14/46

Principles of Language Assessment

UsefulnessValidityReliabilityPracticalityWashback EffectAuthenticityTransparency

Security


15/46


11/11/2013Fundamentals of Language Assessment

UsefulnessBachman and Palmer s (1996) model of test usefulnessrequires that any language test must be developed with aspecific purpose, a particular group of test takers and a

specific language use in mind.


16/46


ValidityThe term validity refers to the extent to which a testmeasures what it says it measures A test is valid if itactually assess the objectives and what has been taught.

Content validity: the test assesses the course content andoutcomes using formats familiar to the studentsConstruct validity: refers to the fit between theunderlying theories and methodology of language learning andthe type of assessmentFace validity: the test looks as though it measures what it issupposed to measure.


17/46


ReliabilityIt refers to the consistency of test scores. A reliabletest is consistent and dependable. If you give the sametest to the same students in two different occasions, the

test should yield similar results.Similarly, if we develop two forms of a test that are

intended to be used interchangeably, it should not makeany difference to the test .The student should obtain

about the same score on either form or version of thetest.


18/46



Three important factors affect test reliability:Test factors : such as the formats , content of thequestions , the length of the exam and the graphicpresentation .

Administrative factors : These include the classroomsetting (lighting, seating arrangements, acoustics, lack ofintrusive noise etc.)Test administration factors : These include affectivefactors like anxiety, motivation, encouragement, etc.


19/46


Practicality

An effective test is practical when it:Is not excessively expensiveStays within appropriate time and resourcesconstraintsIs relatively easy to administer

Has a scoring/evaluation/ interpretation procedurethat is specific and time-efficient


20/46


Washback

Washback refers to the effects the tests have on teachingand learning.Washback can be positive or negativeCram courses and teaching to the test are examplesof negative washback.

Positive washback can benefit teachers, students andadministrators. In some cases the student may learn whenworking on a test or assessment, teachers reconsidertheir teaching strategies.


21/46


AuthenticityAn effective test is authentic when :The language in the test is as natural as possibleItems are contextualized rather than isolatedTopics are relevant and meaningful for learnersSome thematic organization to items is providedTasks represent, or closely approximate, real-world tasks


22/46



TransparencyTransparency refers to the availability of clear, accurateinformation to students about testing. Such informationshould include outcomes to be evaluated, formats used,

weighting of items and sections, time allowed to completethe test, and grading criteria.


23/46



SecurityMost teachers feel that security is an issue only in large-scale, high-stakes testing. However, security is part of bothreliability and validity. If a teacher invests time and energy

in developing good tests that accurately reflect the courseoutcomes, then it is desirable to be able to recycle thetests or similar materials.


24/46



Cornerstones Checklist

When developing, administering and grading exams, ask yourself the following questions:Does your exam test the curriculum content?

Does your exam contain formats familiar to the students?

Does your test reflect your philosophy of teaching?

Would this test yield the same results if you gave it again?

Will the administration of your test be the same for all classes?

Have you helped students reduce test anxiety through test-taking strategies?

Do you have enough time to write, grade and analyze your test?

Do you have all the resources (equipment, paper, storage) you need?

Will this test have a positive effect on teaching and learning?

Are the exam tasks authentic and meaningful?

Do students have accurate information about this test?

Have you taken measures to ensure test security?

Is your test a good learning experience for all involved? Coombe/Hubley


25/46

Types of Assessment


Depending on purpose Depending on characteristics

Screening/Selection/Admission Direct Tests/ Indirect Tests

Placement tests Discrete point/ Integrative tests

Proficiency tests Criterion-referenced/ Norm-referenced tests

Aptitude tests Objective tests/ Subjective tests

Diagnostic tests Speed test/Power testsAchievement tests Knowledge tests/ Skill tests

Progress tests High- stakes /Low-stakes Tests

Formal / Informal tests

Standardized tests / Teacher-made tests

Formative / Summative tests

Assessment for learning / Assessment of learning

Continuous / Terminal Assessment

Alternative Assessment


26/46

Types of Assessment


Depending on purpose :

Screening/Selection/Admission : To know if a personhas the required behavior to be successful in a specificprogram (not based on objectives), e.g. IPCs admission test.

Placemen t: To determine the level in which a personshould be placed inside a language program. Typically, theyassign students to classes at different levels.

Proficiency : To measure a person s overall ability orproficiency in a language. e.g The TOEFL test.


27/46

Types of Assessment


Aptitude : To know the talents of a person to do somethingspecific and measure the suitability of a candidate for a specificprogram of instruction. It predicts a person s success prior toexposure to a language course.

Diagnostic : To identify learners' strengths and weaknesses. Ithelps teachers to make decisions on what needs to be taught

Achievement : To know if a determined objective has beencovered successfully. It measures what learners have learnt ona language course - usually given at the end of the course

Progress : To check improvement achieved according to areferential point in a program. It measures a learners' progressduring a language course


28/46

Types of Assessment


Depending on characteristics:

Direct Tests : Test the skill itself. Students performexactly what we want to test.

Indirect Tests : Test abilities related to the skills we areinterested in, e.g. assess grammar and spelling through awritten exercise.

Discrete point : Focus on one linguistic element at atime. e.g. cloze exercise on verb tenses .

Integrative tests : Measure overall language proficiency,e.g. oral interviews = fluency, pronunciation, content, grammar,comprehension, etc.


29/46

Types of Assessment


Norm-Reference Criterion-ReferenceDefinition Norm-Referenced tests measure the

performance of one group of test takersagainst another group of test takers.

Criterion-Reference tests measure theperformance of test takers against thecriteria covered in the curriculum.

Purpose To measure how much a test taker knowscompared to another student.(to classify)

To measure how much the test taker knownbefore and after the instruction is finished.

Content Norm-Referenced tests measure broadskill areas taken from a variety oftextbooks and syllabi.

Criterion-Reference tests measure the skillsthe test taker has acquired on finishing acurriculum.

Administration Norm-Referenced tests must beadministered in a standardized format.

Criterion-Reference tests need not beadministered in a standardized format.

Scorereporting

Norm-Referenced test scores arereported in a percentile rank.

Criterion-Reference test scores are reportedin categories or percentage.

Scoreinterpretation

In Norm-Referenced tests, if a test takerranks 95%, it implies that he/she hasperformed better than 95% of the othertest takers.

In Criterion-Reference, the score determineshow much of the curriculum is understoodby the test taker.


30/46

Types of Assessment


Objective tests : No judgment involved. Answers are eitherright or wrong. (e.g. Yes/no question items)

Subjective tests : Judgment and opinion involved. No right orwrong answer. (e.g. opinion/discussion items)


31/46

Types of Assessment


Speed test : Easy items in a very short time. Assessspeed of performance and strategy, e.g. scanning exercises.

Power test : Difficult items in enough time. Assess

knowledge.

Knowledge tests : Assess the language components,

e.g. grammar quizzes. Skill tests :Assess the skills, e.g. listening quizzes.


32/46

Types of Assessment


High-stakes tests are those where the results are likely to have amajor impact on the lives of large numbers of individuals, or onlarge programs. E.g. Bac, TOEFLLow-stakes tests are those where the results have a relativelyminor impact on the lives of the individual or on small programs.(in-class progress tests or quizzes).

Informal assessment : It is the observation of everyday performance.It is a way of collecting information about students performance innormal classroom conditions without establishing test conditions. It cantake a number of forms like unplanned comments, verbal feedback,observing students perform a task or work in small groups, etc.Formal assessment : uses formal tests or structured continuousassessment to evaluate a learner's level of language. It is systematicallyadministered to gauge the proficiency level of the person taking the test.It takes a number of forms( formative, summative, achievement, etc.)


33/46

Types of Assessment


Teacher -made tests: oral and written tests made up byan individual teacher for his/her class. They are notcommercially standardized and produced. These tests canbe quite difficult and time consuming.

Standardized tests: contain standardized andpredetermined procedures for its administration, scoringand for the interpretation of results. They are designed tobe used with large populations of students, compare

students performance and are reported in quantitativeterms.


34/46

Types of Assessment


Formative Assessment: is the process of gathering ongoinginformation during instruction to determine what studentsknow and can do, and to provide descriptive feedback toimprove learning and inform teaching. It aims to evaluate theeffectiveness of learning at a time during the course , inorder to provide feedback to make future learning moreeffective. It is carried out frequently and is planned at thesame time as teaching.

Summative Assessment: refers to forms of assessmentused to evaluate student learning at the end of an instructional

unit by comparing it against some standard or benchmark. It isthe evaluation, or judgment reached at the end of a topic,theme, unit, semester, term, or school year based onperformances/ products ,midterm exams, final projects, etc.


35/46

Types of Assessment


Assessment of learning Assessment for LearningWhen ? happens after the learning takes

placeAn intergral part of the learningprocess

Who? Information is gathered by theteacher

Information is shared with thelearner

What? Information is transformed intomarks or grades

Information is available on thequality of learning

Why? Check status- Comprison withthe performance of others

Improve learning- Comparisonwith aims and objectives isimportant

Purpose Looks back on past learningpurpose

Looks forward to the next stage oflearning


36/46

Types of Assessment


Terminal assessment : occurs at the end of a course ormajor unit (examination, practical or situational assessment,dissertation, thesis or portfolio). It is carried out forsummative purposes (progress to the next stage of the course,accreditation after having satisfactorily completed a module,

unit, etc., or to contribute to their final mark, grade or degreeclassification).Continuous assessment . It is carried out on an on-goingbasis while students are actually working through a course. It

can take a wide range of forms , including periodic tests, quizzesessays and other types of assignment, on-going assessment ofpractical work. It can take the form of a mixture of formativeand summative assessment.


37/46

Types of Assessment


Alternative assessmentAlternative assessment is any assessmentmethod that is an alternative to traditionalpaper-and-pencil tests . Alternativeassessment requires students todemonstrate the skills and knowledge thatcannot be assessed using a timed shortanswer test. It focuses on authenti c andinteractiv e tasks. AA seeks to revealstudents' critical-thinking and evaluationskills by asking students to complete open-ended tasks that often take more than oneclass period to complete.

Source:http://teachingtoday.glencoe.com/howtoarticles/alternativeassessment-primer

AuthenticPerformance-basedInteractiveFormativeFocus on high-order skillsIntegrativeExpansive


38/46

ReportsRole plays Brochures DiscussionsInterviewsAudio and VideoRecordings

Presentations

Artifacts

Anecdotal

Records

PostersBanners

Checklists, RatingScales, and

Rubrics

Portfolios Designing mindmaps JournalsDiariesDisplays

Conductingsurveys

Designingcartoons and

commentsConferences

Writing andacting outscenarios


Some Alternative Assessment activities

11/11/2013


39/46

Test Development Procedure


Planning

Test Content and Development

Before the Test

Test Administration

After the Test

Reflection on the testing process


40/46



PlanningEstablish purpose of testo place students in programo achievement of course outcomeso diagnosis of strengths and areas for improvement

Identify objectivesMake an inventory of course content and materials

o consider appropriate formatso establish overall weighting

Write test specifications


41/46



Test Content and DevelopmentMap the examo decide on sections, formats, weighting

Construct items according to test specificationsEstablish grading criteriaPilot the exam


42/46



Before the TestProvide information to studentso coverage, weighting, formats, logistics

Prepare studentso student test-taking strategieso practice exam activities


43/46



Test AdministrationDecide on test conditions and proceduresOrganize equipment neededEstablish test-taking policy

Inform students about availability of results


44/46



After the Test

Grade testsCompute basic statisticsGet results to students and provide feedbackConduct exam analysisReport on exam results


45/46



Reflect on the testing process

Learn from each exam Did it serve its purpose? Was it a valid and reliable test? Was it part of the students learning experience?

What future changes would you make?


46/46

ReferencesAlastair Irons.(2008).Enhancing learning through Formative Assessment and Feedback. Routledge. Londonand New York

Anderson. L. W. (2003). Classroom Assessment. Lawrence Erlbaum Associates, Inc.

Bachman, L. F. (1990). Fundamental Considerations in Language Testing. Oxford applied linguistics. Oxford:Oxford University Press.

Brown, H. D. (2004). Language Assessment: Principles and Classroom Practices. New York: Pearson/Longman.

Brown, J. D. (1996). Testing in Language Programs. Prentice Hall Regents.

Gipps. C. V. (1994). Beyond Testing. The Falmer Press.Glenn Fulcher and Fred Davidson. (2007). Language testing and Assessment. Routledge. New York.

Heaton, J. B. (1988). Writing English Language Tests. New Edition. London: Longman,.

Hughs, Arthur. (1989).Testing for Language Teachers. New York: Cambridge University Press,

Kopriva. R. J. (2008). Improving Testing for English Language Learners. Routledge. New York.

Madsen. H. S. (1983). Techniques in Testing. Oxford University Press. New York.

Paul Black et al. (2003). Assessment for Learning: putting it into practice. Open University Press. England.

Peggy. L. Maki. (2004). Assessing for Learning. American Association for higher Education. Virginia.

Phil Race et al. (2005). 500 Tips on assessment. RoutledgeFalmer. London and New York.

Popham. W. J. (2003). Test Better, Teach Better: The Instructional Role of Assessment. Association forSupervision and Curriculum Development. USA.

Testing and Assessing Final presentation.pdf

Documents

Transcript of Testing and Assessing Final presentation.pdf