Tests and MeasurementsTests and Measurements

Educational Research

Educational Educational MeasurementMeasurement

technique used to measure data

ClassificationsClassifications

Cognitive versus Noncognitive Commercial versus Teacher-made Self-reporting versus Observation

Cognitive Vs NoncognitiveCognitive Vs Noncognitive

Noncognitive examines

personality, attitudes, values

Cognitive measures what a

person knows or is able to do mentally

Commercial or StandardizedCommercial or Standardized

Disadvantages costs of using/ scoring may not match

objectives or needs norms may not reflect

local pop

Advantages comparative norms already put together results compared to

others in norms known reliabilities

and validities

Teacher-madeTeacher-made

disadvantages time involvement unknown reliabilities

and validities

advantages use when

inappropriate to use standardized test

can more closely relate to objectives

inexpensive costs

Self-reporting Versus Self-reporting Versus ObservationObservation

Self-reporting best way to obtain lots

of information quickly NOT always honest do NOT always know

Observation

Important TermsImportant Terms

Raw score # of (in)correct responses amount of time Standard score raw scores transformed to normalized

score (z scores, t scores, GREs)

Grade equivalents norm-referenced grade level performance not interpreted well; limited meaning Percentile ranks describes how well an individual did on a test

compared to a particular group not interpreted well, esp. at extremes

Reliability degree of consistency/dependability of a

measuring instrument increase # items on test range from 0 to +1.0 Validity credibility of measuring instrument does instrument measure what it claims to

measure?

Referenced-based Referenced-based Measurement InterpretationMeasurement Interpretation

Norm-referenced compare one to the many performance reported in %iles, standard

scores, etc. items produce wide range of scores norms refer to typical/avg performance of a

group of people

Criterion-referenced what the student CAN do without

reference to others in reference to a prespecified standard of

performance concerned with mastery items selected to match instructional

objectives

Types of TestsTypes of Tests

Standard questions of cognitive knowledge or skills

Achievement Tests Intelligence Tests Aptitude Tests Performance Assessment

Achievement TestsAchievement Tests

measures mastery in different areas of knowledge

most common form of measurement in educational research

Intelligence TestsIntelligence Tests

look at performance in a specific area NOT measures of innate or pure intelligence dependent on background and schooling (more

measures of scholastic aptitude used to predict school/academic success; IVs

Important persons and tests: Army Alpha and Army Beta

first group IQ tests Stanford-Binet

first individual IQ test David Wechsler

Wechsler intelligence scales most popular in use today

Aptitude TestsAptitude Tests

aimed at measuring a person’s future performance in a specific skill or area of achievement

very close to intelligence

Performance Performance AssessmentAssessment

authentic assessment/alternative assessment

approach to evaluating students by directly examining performance on tasks with intrinsic value

Measures of PersonalityMeasures of Personality

Assess individual differences in aspects of personality such as traits, needs, psychological disorders, values, and attitudes.

Often self-reporting.

TypesTypes

Inventories Projective Techniques association completion role playing creative/construction Attitude Scales/Opinionnaires Likert scales Thurstone Scale Semantic Differential Technique

InventoriesInventories

attempt to yield a measure of the types of activities an individual likes or has a tendency to choose.

Projective TechniquesProjective Techniques

persons project thoughts, feelings, attitudes, needs onto vague stimulus

Rorschach Ink Blot Thematic Apperception Test Association Completion Role Playing

Creative/Construction

Attitude Scales/OpinionnairesAttitude Scales/Opinionnaires

measures degree to which person possesses characteristic of interest

do NOT reflect success/failure or strengths/weaknesses

Likert Scale Thurstone Scale Semantic Differential Technique

Likert ScaleLikert Scale

R. Likert collection of statements about a given topic half are positively stated and half are

negatively stated arranged in random order measured on a scale of 1 - 5

(SA A U D SD)

Semantic Differential ScaleSemantic Differential Scale

Circle one of the #s between each pair of adjectives to best indicate how closely one of the adjectives describes your attitude toward essay questions.

Good 1 2 3 4 5 6 7 Bad Pleasant 1 2 3 4 5 6 7 Unpleasant Fair 1 2 3 4 5 6 7 Unfair Positive 1 2 3 4 5 6 7 Negative

Rating ScalesRating Scales

widely used measuring instrument involves assessment by one person of

another’s behavior/performance Four types

Graphic ScaleGraphic Scale

uses a continuum place a checkmark at the area to describe

the performance/behavior numerical values may be assigned to

descriptive points

Numerical ScaleNumerical Scale

involves assignment of numbers to points on graphic scales

Categorical ScaleCategorical Scale

categories arranged in ordered series rater selects category of person being rated

Comparative Rating ScaleComparative Rating Scale

make judgments compared to others sometimes school systems use this

method for references

Rating ErrorsRating Errors

Halo Effect generalized impression of person that

slants the perception (good manners, bad boy)

Generosity Error giving subjects benefit of the doubt

Rating ErrorsRating Errors

Error of Severity tendency to rate all persons too low on

all characteristics Error of Central Tendency

avoiding the extremes of the scale, rate persons in middle

How to Reduce ErrorsHow to Reduce Errors

Thorough training of raters BEFORE

making ratings

Sociometric TechniquesSociometric Techniques

SOCIOGRAM used to study social relationships present

among various group members study of choices made by each person in a

group

ProcedureProcedure

each member of group indicates other members with whom s/he would MOST like to engage in activity such as recess, lunch, or work time

choices will vary, depending upon the activity

choices are plotted

Sociogram LegendSociogram Legend

boys=triangles girls=circles arrows show direction of selection

Sociogram Sociogram InterpretationInterpretation

star(s) most frequently chosen member(s) clique(s) small subgroups of persons (3 or more)

who mutually select each other isolate member(s) who receives no choices

ExamplesExamples

Students asked to indicate two other children with whom they would MOST like to play: Maria--Juan, Pat Juan--Maria, Pat Pat--Maria, Juan Sue--Tony, Marco Tony--Sue, Marco Laura--Sue, Marco Jackson--Tony, Marco Marco-Laura,Tony

ExamplesExamples

Students in an RSCH 7100 class were asked to specify the names of two other students with whom they would most like to work with on the midterm exam. Of the six students asked, the following are their selections.

Choices of students w/whom to Choices of students w/whom to work, on midterm examwork, on midterm exam

Herbert --- Sam, Julie John --- Fred, Julie Sam -- Julie, Herbert Fred -- Cynthia, Julie Julie -- Sam, Herbert Cynthia -- Fred, Julie

Number of times selectedNumber of times selected Herbert Twice (Julie, Sam) John None (isolate) Sam Twice (Herbert, Julie) Fred Twice (John, Cynthia) Julie 5X (all persons--star) Cynthia Once (Fred) Clique: Herbert, Julie, and Sam mutually

selected only each other

ObservationsObservations

Purpose: determine extent to which a particular behavior(s) is present

used to study classroom behaviors; natural or contrived settings; with infants, preschool, and elementary school children

in quantitative and qualitative research

ObservationsObservations

Disadvantages expensive to use in

terms of time and resources

presence of observer may alter behavior

observer bias

Advantages observe persons’

behaviors in natural surroundings

may use w/those who cannot communicate through language

InterviewsInterviews

form of data collection in which questions asked orally and participant’s responses are recorded

WE AREWE ARE DONE