Education 3504 Week 3 reliability & validity observation techniques checklists and rubrics.
-
date post
19-Dec-2015 -
Category
Documents
-
view
217 -
download
1
Transcript of Education 3504 Week 3 reliability & validity observation techniques checklists and rubrics.
Education 3504 Week 3
reliability & validityobservation techniqueschecklists and rubrics
Reliability and Validity
• validity – extent to which you are measuring
what you think you are measuring
• reliability – extent to which information you are
gathering is consistent and unambiguous
Reliability
• chance factors– characteristics of student– characteristics of test / evaluation– conditions of administration– variations in scoring
Validity
• danger of thinking you are measuring one thing, actually measuring another– trivial knowledge– lowest-common denominator– regional knowledge/one
culture/gender
Validity
• face validity– appears important, relevant, interesting
• construct validity– appears to capture intended quality or
behaviour - e.g. what does “good citizenship” consist of?
validity and reliability are related
• but can vary independently
Checklists and Rubrics
checklist construction
• Checklist must be based on objectives of the course, unit, or lesson
• each item should be clear and precise
• each item should avoid generalities and focus on specifics
checklist construction
• checklist should be designed for feedback to students and parents – so need to be understandable – might want to distribute to students
before hand • must do so if intended for grading
checklist construction
• if students can use them, can be used for peer and self-evaluation
• focus on a few specifics, so keep checklist short
– if list needs to be longer, focus on only small section at a time
• anecdotal summary comments can be tied to checklists
rating scales
5 -Extremely active
4 -somewhat active
3 -average active
2 -somewhat active
1 -extremely active
rating scales
Extremely Active (5 points)
• always out of seat
• rarely sits without squirming
• makes noise, yells a good deal of the time
Rubricsscoring schemes
clear expectationsGeneric Essay
Subjectcontent
awareness of issues
critical thinking
Takes a position
EXCELLENT
A (85 - 100)
A Markedly Exceptional Performance
a comprehensive grasp of the subject matter is demonstrated, including an in-depth understanding of the relevant concepts, theories, and issues related to the topic addressed
an awareness of differing viewpoints is demonstrated and a rigorous assessment of these undertaken where relevant
an ability to think critically is demonstrated in the analysis, synthesis and evaluation of relevant information
a thoughtful statement of position is presented and defended through logical arguments and carefully selected supportive detail; the arguments presented build to a consistent conclusion
SUPERIOR B (70 - 84)
Clearly Above Average Performance
a thorough grasp of the subject matter is demonstrated
an awareness of differing viewpoints is demonstrated and an assessment of these attempted where relevant
the paper goes beyond description to interpretation, analysis, synthesis and evaluation
a position is adopted and logically argued; appropriate supporting
detail is supplied
SATISFACTORY
C (55 - 69)
A Fully Competent Paper
a basic grasp of the subject matter is
demonstrated
asserts viewpoint without acknowledging alternative viewpoints
accurate information incorporating relevant sources and references is conveyed
a position is adopted
and logically argued
…but flexibleGeneric Essay
Subjectcontent
awareness of issues
critical thinking
Takes a position
EXCELLENT
A (85 - 100)
A Markedly Exceptional Performance
a comprehensive grasp of the subject matter is demonstrated, including an in-depth understanding of the relevant concepts, theories, and issues related to the topic addressed
an awareness of differing viewpoints is demonstrated and a rigorous assessment of these undertaken where relevant
an ability to think critically is demonstrated in the analysis, synthesis and evaluation of relevant information
a thoughtful statement of position is presented and defended through logical arguments and carefully selected supportive detail; the arguments presented build to a consistent conclusion
SUPERIOR B (70 - 84)
Clearly Above Average Performance
a thorough grasp of the subject matter is demonstrated
an awareness of differing viewpoints is demonstrated and an assessment of these attempted where relevant
the paper goes beyond description to interpretation, analysis, synthesis and evaluation
a position is adopted and logically argued; appropriate supporting
detail is supplied
SATISFACTORY
C (55 - 69)
A Fully Competent Paper
a basic grasp of the subject matter is
demonstrated
asserts viewpoint without acknowledging alternative viewpoints
accurate information incorporating relevant sources and references is conveyed
a position is adopted
and logically argued
levels of masteryGeneric Essay
Subjectcontent
awareness of issues
critical thinking
Takes a position
EXCELLENT
A (85 - 100)
A Markedly Exceptional Performance
a comprehensive grasp of the subject matter is demonstrated, including an in-depth understanding of the relevant concepts, theories, and issues related to the topic addressed
an awareness of differing viewpoints is demonstrated and a rigorous assessment of these undertaken where relevant
an ability to think critically is demonstrated in the analysis, synthesis and evaluation of relevant information
a thoughtful statement of position is presented and defended through logical arguments and carefully selected supportive detail; the arguments presented build to a consistent conclusion
SUPERIOR B (70 - 84)
Clearly Above Average Performance
a thorough grasp of the subject matter is demonstrated
an awareness of differing viewpoints is demonstrated and an assessment of these attempted where relevant
the paper goes beyond description to interpretation, analysis, synthesis and evaluation
a position is adopted and logically argued; appropriate supporting
detail is supplied
SATISFACTORY
C (55 - 69)
A Fully Competent Paper
a basic grasp of the subject matter is
demonstrated
asserts viewpoint without acknowledging alternative viewpoints
accurate information incorporating relevant sources and references is conveyed
a position is adopted
and logically argued
multi-dimensionalGeneric Essay
Subjectcontent
awareness of issues
critical thinking
Takes a position
EXCELLENT
A (85 - 100)
A Markedly Exceptional Performance
a comprehensive grasp of the subject matter is demonstrated, including an in-depth understanding of the relevant concepts, theories, and issues related to the topic addressed
an awareness of differing viewpoints is demonstrated and a rigorous assessment of these undertaken where relevant
an ability to think critically is demonstrated in the analysis, synthesis and evaluation of relevant information
a thoughtful statement of position is presented and defended through logical arguments and carefully selected supportive detail; the arguments presented build to a consistent conclusion
SUPERIOR B (70 - 84)
Clearly Above Average Performance
a thorough grasp of the subject matter is demonstrated
an awareness of differing viewpoints is demonstrated and an assessment of these attempted where relevant
the paper goes beyond description to interpretation, analysis, synthesis and evaluation
a position is adopted and logically argued; appropriate supporting
detail is supplied
SATISFACTORY
C (55 - 69)
A Fully Competent Paper
a basic grasp of the subject matter is
demonstrated
asserts viewpoint without acknowledging alternative viewpoints
accurate information incorporating relevant sources and references is conveyed
a position is adopted
and logically argued
parallelismGeneric Essay
Subjectcontent
awareness of issues
critical thinking
Takes a position
EXCELLENT
A (85 - 100)
A Markedly Exceptional Performance
a comprehensive grasp of the subject matter is demonstrated, including an in-depth understanding of the relevant concepts, theories, and issues related to the topic addressed
an awareness of differing viewpoints is demonstrated and a rigorous assessment of these undertaken where relevant
an ability to think critically is demonstrated in the analysis, synthesis and evaluation of relevant information
a thoughtful statement of position is presented and defended through logical arguments and carefully selected supportive detail; the arguments presented build to a consistent conclusion
SUPERIOR B (70 - 84)
Clearly Above Average Performance
a thorough grasp of the subject matter is demonstrated
an awareness of differing viewpoints is demonstrated and an assessment of these attempted where relevant
the paper goes beyond description to interpretation, analysis, synthesis and evaluation
a position is adopted and logically argued; appropriate supporting
detail is supplied
SATISFACTORY
C (55 - 69)
A Fully Competent Paper
a basic grasp of the subject matter is
demonstrated
asserts viewpoint without acknowledging alternative viewpoints
accurate information incorporating relevant sources and references is conveyed
a position is adopted
and logically argued
high but attainable standards: cumulative criteria
Generic Essay
Originality Subjectcontent
awareness of issues
critical thinking
EXCELLENT
A (85 - 100)
A Markedly Exceptional Performance
originality, insight, and creativity are demonstrated; the paper goes beyond repeating what others have said and contributes something new to our understanding of the topic
a comprehensive grasp of the subject matter is demonstrated, including an in-depth understanding of the relevant concepts, theories, and issues related to the topic addressed
an awareness of differing viewpoints is demonstrated and a rigorous assessment of these undertaken where relevant
an ability to think critically is demonstrated in the analysis, synthesis and evaluation of relevant information
SUPERIOR B (70 - 84)
Clearly Above Average Performance
a thorough grasp of the subject matter is demonstrated
an awareness of differing viewpoints is demonstrated and an assessment of these attempted where relevant
the paper goes beyond description to interpretation, analysis, synthesis and evaluation
SATISFACTORY
C (55 - 69)
A Fully Competent Paper
a basic grasp of the subject matter is
demonstrated
asserts viewpoint without acknowledging alternative viewpoints
accurate information incorporating relevant sources and references is conveyed
allowance for part marksGeneric Essay
Subjectcontent
awareness of issues
critical thinking
Takes a position
EXCELLENT
A (85 - 100)
A Markedly Exceptional Performance
a comprehensive grasp of the subject matter is demonstrated, including an in-depth understanding of the relevant concepts, theories, and issues related to the topic addressed
an awareness of differing viewpoints is demonstrated and a rigorous assessment of these undertaken where relevant
an ability to think critically is demonstrated in the analysis, synthesis and evaluation of relevant information
a thoughtful statement of position is presented and defended through logical arguments and carefully selected supportive detail; the arguments presented build to a consistent conclusion
SUPERIOR B (70 - 84)
Clearly Above Average Performance
a thorough grasp of the subject matter is demonstrated
an awareness of differing viewpoints is demonstrated and an assessment of these attempted where relevant
the paper goes beyond description to interpretation, analysis, synthesis and evaluation
a position is adopted and logically argued; appropriate supporting
detail is supplied
SATISFACTORY
C (55 - 69)
A Fully Competent Paper
a basic grasp of the subject matter is
demonstrated
asserts viewpoint without acknowledging alternative viewpoints
accurate information incorporating relevant sources and references is conveyed
a position is adopted
and logically argued
allowance for part marks
A - EXCELLENT (85 - 100) - A Markedly Exceptional Performance
√ originality, insight, and creativity are demonstrated; the paper goes beyond repeatingwhat others have said and contributes something new to our understanding of thetopic
a comprehensive grasp of the subject matter is demonstrated, including an in-depthunderstanding of the relevant concepts, theories, and issues related to the topicaddressed
an awareness of differing view-points is demonstrated and a rigorous assessment ofthese undertaken where relevant
an ability to think critically is demonstrated in the analysis, synthesis and evaluationof relevant information
a thoughtful statement of position is presented and defended through logicalarguments and carefully selected supportive detail; the arguments presented build to aconsistent conclusion
√ a clear, fluent, and concise style highlights a well-written, tightly argued, andlogically structured essay
√ a virtually flawless mastery of all aspects of grammar, structure, and style isdemonstrated
B - SUPERIOR (70 - 84) - Clearly Above Average Performance
a thorough grasp of the subject matter is demonstrated√ an awareness of differing view-points is demonstrated and an assessment of these
attempted where relevant the paper goes beyond description to interpretation, analysis, synthesis and evaluation a position is adopted and logically argued; appropriate supporting detail is supplied a clear style which communicates well (but may contain occasional or minor flaws in
the mechanics of spelling, grammar, sentence structure, etc.) is evident in the logicalpresentation of a reasonable argument
C - SATISFACTORY (55 - 69) - A Fully Competent Paper
√ a basic grasp of the subject matter is demonstrated accurate information incorporating relevant sources and references is conveyed√ a position is adopted and logically argued an adequate attempt at analysis, synthesis, interpretation or evaluation is evident an acceptable style demonstrates an awareness of, and attention to, the principles of
paragraph development, sentence structure, grammar and spelling, etc.
correspondence to real worldGeneric Essay
Subjectcontent
awareness of issues
critical thinking
Takes a position
EXCELLENT
A (85 - 100)
A Markedly Exceptional Performance
a comprehensive grasp of the subject matter is demonstrated, including an in-depth understanding of the relevant concepts, theories, and issues related to the topic addressed
an awareness of differing viewpoints is demonstrated and a rigorous assessment of these undertaken where relevant
an ability to think critically is demonstrated in the analysis, synthesis and evaluation of relevant information
a thoughtful statement of position is presented and defended through logical arguments and carefully selected supportive detail; the arguments presented build to a consistent conclusion
SUPERIOR B (70 - 84)
Clearly Above Average Performance
a thorough grasp of the subject matter is demonstrated
an awareness of differing viewpoints is demonstrated and an assessment of these attempted where relevant
the paper goes beyond description to interpretation, analysis, synthesis and evaluation
a position is adopted and logically argued; appropriate supporting
detail is supplied
SATISFACTORY
C (55 - 69)
A Fully Competent Paper
a basic grasp of the subject matter is
demonstrated
asserts viewpoint without acknowledging alternative viewpoints
accurate information incorporating relevant sources and references is conveyed
a position is adopted
and logically argued
evolving
• changes if you discover papers that do not fit criteria – add descriptors that accurately describe
what you are seeing
evolving
• changes as your teaching changes
• changes as students change
• changes as environment changes
ease of useA - EXCELLENT (85 - 100) - A Markedly Exceptional Performance
originality, insight, and creativity are demonstrated; the paper goes beyond repeatingwhat others have said and contributes something new to our understanding of thetopic
a comprehensive grasp of the subject matter is demonstrated, including an in-depthunderstanding of the relevant concepts, theories, and issues related to the topicaddressed
an awareness of differing view-points is demonstrated and a rigorous assessment ofthese undertaken where relevant
an ability to think critically is demonstrated in the analysis, synthesis and evaluationof relevant information
a thoughtful statement of position is presented and defended through logicalarguments and carefully selected supportive detail; the arguments presented build to aconsistent conclusion
a clear, fluent, and concise style highlights a well-written, tightly argued, andlogically structured essay
a virtually flawless mastery of all aspects of grammar, structure, and style isdemonstrated
B - SUPERIOR (70 - 84) - Clearly Above Average Performance
a thorough grasp of the subject matter is demonstrated an awareness of differing view-points is demonstrated and an assessment of these
attempted where relevant the paper goes beyond description to interpretation, analysis, synthesis and evaluation a position is adopted and logically argued; appropriate supporting detail is supplied a clear style which communicates well (but may contain occasional or minor flaws in
the mechanics of spelling, grammar, sentence structure, etc.) is evident in the logicalpresentation of a reasonable argument
ease of useChecklist - Space Community Unit
1 = weak 2 = fair 3 = good 4 = very good 5 = superior
Name:
Date:
Grade:
The students demonstrate the ability to:
A Social 1 2 3 4 5 Comments
1. respond to stimuli
2. respect the space of others
3. discipline self
4. develop appreciation of thework of self and others
B Physical
1. become aware of body andvoice as instruments ofexpression
C Intellectual/Emotional
1. communicate through space
2. develop concentration
3. recognize form of drama(beginning, middle, end)
4. develop and exerciseimagination
5. explore emotion (role play)
KEY BEHAVIORS
A #1 Respond toStimuli
#2 Respect thespace of others
#3 Discipline Self #4 Developappreciation
B. #1 Becomeaware of Body andVoice
Superior
5
Acts on experienceand information in astrikingly co herentway
Respec ts the spaceof others every time
Readily takes ac tionto discipline self
Indica tes whole-hearted enthusiasm
Begins to use bodyand voice in a richvariety that reflec tspersonalknowledge, feelingsand values
Very Good
4
Act on experienceand information in aconsistentlycoherent way
Respec ts the spaceof othersconsistently
Takes action todiscipline self
Indica tesenthusiasm
Begins to use bodyand voice in avariety of ways thatreflects personalknowledge, feelingsand values
Good
3
Acts on experienceand information in agenerally coherentway
Respec ts space ofothers most of thetime
Sometimes takesaction to disciplineself
Indica tes interest Begins to use bodyand voice somedifferent ways thatreflect personalknowledge, feelingsand values
Fair
2
Acts on experienceand information in asomewhathaphazard way
Respec ts the spaceof others some ofthe time
With help takesaction to disciplineself
Indica tesacceptance
Begins to use bodyand voice withencouragementthat reflectspersonalknowledge, feelingsand values
Weak
1
Acts on experienceand informationrandomly;sometimes rambles
Respec ts space ofothers hardly ever
Even with help doesnot always disciplineself
Indica tesimpatience
Begins to use bodyand voice withdifficulty even whenencouraged toexpress personalknowledge, feelings
rubrics workshop
• Objective:Produce the best cookie possible
rubrics workshop
• Objective:Produce the best cookie possible
• Was it difficult to come to a consensus on criteria for “best”?
why we evaluate
• motivation • administration• accountability• certification
motivation
• problems: – test anxiety
Norm referenced
• describes student performance relative to other students– e.g.: ranks 11th in group of 30
Criterion referenced
• describes student performance compared to a clearly defined criteria or task– e.g.: “adds single-digit whole numbers”
motivation• problems:
– overemphasis distorts learning process
• emphasis on external motivation
motivation
• problems: – overemphasis distorts learning process
• not everything we teach can be assessed (e.g., non-cognitive skills, attitudes, etc.)
administration
• general trend towards greater emphasis on assessment
• general shift from selection to accountability
accountability
• government to taxpayers– 2003, K-12 = 3.8 billion dollars…
accountability
• teachers– accountable for mandated curriculum– accountable for student learning– to oneself over time
accountability
• students to their parents
certification
• right of passage• selection and
allocation (gatekeeper)