Assessment by Reginald Smith

APT 501

October 15, 2016

Background:

An “assessment” is not just a test, but a range of items used to gauge a learner’s abilities and progress as well as quality of instruction in addition to evaluating the instructional mediums used. Criterion (or the plural version of criteria) is defined as: Criterion. A standard on which a judgment or decision may be based learners will a characterizing mark or trait answer 85% of quiz Criterion-referenced assessments, also known as objective questions reference assessments, focus on measuring performance items correctly to associated with learner performance and instructional integrity.

OBJECTIVES:

Describe the purpose for criterion-referenced tests.

Describe how entry behaviors tests,pretests,and posttests are used by instructional designers.

Name four categories of criteria for developing criterion-referenced tests and list several considerations within each criterion category

             

 

OBJECTIVE  II:      

n Given  a  variety  of  objectives,write  criterion-­‐referenced,objective-­‐style  test  items  that  meet  quality  criterion  in  all  four  categories.  

n Develop  instructions  for  product  development,live  performance,and  attitude  assessments,and  develop  a  rubric  for  evaluating  learners’work.  

n Evaluate  instructional  goals,subordinate  skills,learner  and  context  analyses  performance  objectives,and  criterion-­‐referenced  test  items  for  congruence.  

                       

MAIN  CONTENTS:        

n What  is  the  purpose  for    criterion-­‐referenced  test?  

n How  entry  behaviors  tests,pretests,  and  posttests  are  used  by  instructional  designers?  

n How  does  one  go  about  designing  and  developing  a  criterion-­‐referenced  test?  

n What  are  criteria  for  developing  criterion-­‐referenced  tests  and  list  several  considerations  within  each  criterion  category?  

n How  to  determine  mastery  levels?  n How  to  set  mastery  criteria?  

           

Background:      

n What  is  criterion-­‐referenced  test?  n The  definitions  of  learner-­‐centered  assessment  are  congruent  with  traditional  definitions  of  criterion-­‐referenced  testing.  

n Why  test  development  appears  at  this  point  in  the  instructional  design  process?  (p146)  

Why  use  the  term  assessment  not  testing.                      

Criterion-­‐Referenced  Assessment:    n A  criterion-­‐referenced  assessment  is  composed  of  items  or  performance  tasks  that  directly  measure  skills  described  in  one  or  more  behavioral  objectives.  

n Learner-­‐centered  assessments  are  to  be  criterion-­‐referenced.  

This  type  of  testing  is  important  for  evaluating  both  learners’  process  and  instructional  quality                        

CONCEPT:    

Test type Designer’s decision Objectives typically tested

Behaviors test

*Are target learners ready to enter

instruction?*Do learners possess the required

prerequisite skills?

*Prerequisite skills or those skills below the dotted line in the instructional

analysis

Pretest *Have learners previously mastered the

enabling skills?*Which particular skills have they previously

mastered? *How can I most efficiently develop this

instruction?

*Terminal objectives

*Main steps from the goal

analysis

         

Designing  a  test:    n Verbal  information:objective-­‐style  test  items(short-­‐answer,alternative  response,matching,and  multiple-­‐choice  items)  

n Intellectual  skills:objective-­‐style  test  items,the  creation  of  a  product,a  live  performance.  

n Attitudinal  :the  learners  state  their  preference,the  instructor  observes  the  learners’behaviors.  

n Psychomotor  :perform  a  sequence  of  steps,a  checklist  or  rating  scale,subordinate  skills.  

     

Determining  mastery  levels:    n As  a  general  principle,mastery  level  for  any  performance  should  be  considered  with  respect  to  both  evaluating  the  performance  at  that  point  in  time  and  enhancing  the  learning  of  subsequent,related  skills  in  the  unit  or  in  the  rest  of  the  course.  

n The  best  definition  of  mastery  is  the  level  required  in  order  to  be  successful  on  the  job.  

       If  the  level  prove  to  be  unrealistic,they  can  be  adjusted  in  the  future.                  

Writing  test  items:    Four  categories  of  test  item  qualities  that  should  be  considered  :    n Goal–centered  criteria    

n Learner-­‐centered  criteria    

n Context-­‐centered  criteria    

n Assessment-­‐centered  criteria                      

Setting  mastery  criteria:    n What  is  the  proper  number  of  items  needed  to  determine  mastery  of  an  objective?  

 n How  many  items  must  learners  answer  correctly  to  be  judged  successful  on  a  particular  objective?  

       Intellectual  skills>3      Verbal  information=1              

Choice  the  appropriate  type:      To  select  the  best  type  of  item  from  among  those  that  are  adequate,consider  such  factors  as  the  response  time  required  by  learners,the  scoring  time  required  to  analyze  and  judge  answers,the  testing  environment,and  the  probability  of  guessing  the  correct  answer.                

Writing  directions:    n The  test  title  suggests  the  content  to  be  covered.  

n A  brief  statement  explains  the  objectives  or  performance  to  be  demonstrated,and  the  amount  of  credit  that  will  be  given  for  a  partially  correct  answer.  

n Learners  are  told  whether  they  should  guess  if  they  are  unsure  of  the  answer.  

               

Evaluating  tests  and  test  items:    n The  designer  should  ensure  the  following:  

n  1. Test  directions    2. Each  test  item      3. Conditions      4. The  response  methods    5. Appropriate  space,time,and  equipment  

   

Evaluating  tests  and  test  items:  n After  writing  the  test,the  designer  should  administer  it  to  a  student  or  individual  who  will  read  and  explain  aloud  what  is  meant  by  both  the  directions  and  questions,and  respond  to  each  question  in  the  intended  response  format.  

n The  designer  should  keep  in  mind  that  tests  measure  the  adequacy  of  (1)  the  test  itself,(2)  

       the  response  form,(3)  the  instructional  materials,(4)  the  instructional  environment  and  situation,and  (5)  the  achievement  of  learners,      

Developing  instruments  to  measure  performances,products,and  attitudes:    n Writing  directions    

n Developing  the  instrument    

n Identify,paraphrase,and  sequence  elements    

n Developing  the  response  format  :    checklist,  rating  scale,  frequency  count    

n Scoring  procedure                

Using  portfolio  assessments:    n Portfolio  assessment  is  defined  as  the  process  of  meta-­‐evaluating  the  collection  of  work  samples  for  observable  change  of  development.  

 n Features  of  quality  portfolio  assessment.  (P162)  

   Portfolio  assessment  is  not  appropriate  for  all  instruction  since  it  is  very  time  consuming  and  expensive                  

Evaluating  congruence  in  the  design  process:    n Evaluate  congruence  in  skill,    performance  objective,  test  items.    

n You  must  create  an  assessment  for  the  terminal  objective.  

   

n The  sequence  of  subskills  presented  on  your  chart  is  important.