Chapter 9 Correlation, Validity and Reliability. Nature of Correlation Association – an attempt to...
-
Upload
albert-ramsey -
Category
Documents
-
view
217 -
download
1
Transcript of Chapter 9 Correlation, Validity and Reliability. Nature of Correlation Association – an attempt to...
Chapter 9
Correlation, Validity and Reliability
Nature of Correlation
• Association – an attempt to describe or understand
• Not causal– However, many people will use terms such as
“predictor”
Correlation
• Association between 2 variables in its simplest form.
• Variable X and Variable Y
• Often times X = predictor variableY = criterion variable
Predictor/Criterion
• Height and shoe size
.73
Height = predictor
Shoe size = criterion
Could very well be reversed - explanatory
Predictor/Criterion
• Predictor = High school GPA
• Criterion = College GPA
• Predictor = belief about fixed intelligence
• Criterion = amount of study time per week
• Predictor = amount of time reading at home
• Criterion = grades in Literacy in 8th grade
Coefficient of Determination
• Indicated by r2 (r-squared)
• Indicates the amount of variance explained or accounted by the relationship between the variables
• Quick and dirty method of understanding the strength of the relationship
Common uses in Education
• Validity (e.g. Criterion related: predictive & concurrent)
• Reliability of instruments
• Inter-rater reliability
Validity
How well can you defend the measure?
– Face V
– Content V
– Criterion-related V
– Construct V
Face Validity
• Does instrument look like valid?– On a survey or questionnaire, the questions
seem to be relevant– On a checklist, the behaviors seem relevant– For a performance test, the task seems to be
appropriate
Content Validity
• The content of the test, the measure, is relevant to the behavior or construct being measured
• An expert judges or a panel of experts judge the content
Criterion Related
• Using a another independent measure to validate a test– Typically computing a correlation
• Two types– Predictive validity– Concurrent validity
Criterion-Related
Predictive• ACT achievement test
Correlated with College GPA
Concurrent• Coopersmith Self-esteem Scale
Correlated with teacher’s ratings of self-esteem
Construct Validity
• Construct – attempt to describe, name an intangible variable
• Use many different measures to validate a measure
• Self-esteem – construct– Instrument measure
Construct Validity
• Self-esteem – construct– Instrument measure e.g. coopersmith
– Correlated it with:• Behavioral checklist• Teacher’s comments• Another accepted instrument for Self-esteem• A measure of confidence• Locus of control measure
Reliability
• For an instrument –– Consistency of scores from use to use
• Types of reliability coefficients– Test – retest– Equivalent forms– Internal consistency
• Split-half• Alpha coefficient (Cronbach alpha)
Reliability Coefficient
• Value ranges from 0 to +1.00
• .70 considered the minimal acceptable
• .90 is very good
• .60 is sometimes acceptable but is really not very good
• Lower than .60 definitely unacceptable
Reliable but is it Valid?Valid but is it Reliable?
Invalid and UnreliableNo confidence you’ll get near the target; have no idea where it’s going to shoot.
Reliable but is it Valid?Valid but is it Reliable?
Invalid but ReliableNo confidence you’ll get near the target; but you know where it’s going to shoot (just not at the target!)
Reliable but is it Valid?Valid but is it Reliable?
Valid but UnreliableConfidence that when you hit something, it’s what you want, but you can’t depend upon consistency.
Reliable but is it Valid?Valid but is it Reliable?
Valid and ReliableConfident that when you hit a target, it’s what you want and you can depend upon consistent shots.
Inter-rater reliability
Example –
Two teachers reading same essay, scoring them in a similar manner – consistently
Using same checklist to make observations
Can be expressed as a coefficient
Often as percentage of agreement
A function of training, objectivity, and rubric or checklist, i.e., the operational definition!
• Norm-referenced tests– Comparison of individual score to others– Intelligence test– ISAT, Iowa Basic Skills Test– SAT aptitude test– Personality test
– Percentile’s - derived scores– Grading on a curve
• Criterion referenced test– Individual score is compare to a benchmark (a
criterion)
– If Raw Score used (no conversion): C-R test– Mastery of material– Earning a grade in my class
– Disadvantage is potential lack of variability
Measures of Optimum Performance
• Aptitude Tests– Predict future performance
• Achievement tests– Measure current knowledge
• Performance tests– Measure current ability to complete tasks
Measures of typical performance
Often impacted by “social desirability”– Wanting to hide
undesirable traits or characteristics
One way to work around sd is touse projective tests
Rorschach ink Blot
Thematic Apperception Test
• Paper/pencil measures of attitudes using Likert-type scales
• Strongly Agree – Strongly Disagree- Reverse scoring to prevent or identify
“response bias”