Part 2: Quantitative Methods
description
Transcript of Part 2: Quantitative Methods
![Page 1: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/1.jpg)
Part 2: Quantitative Methods
October 2, 2006
![Page 2: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/2.jpg)
A Scientific Process
Research Question
Defining the Problem
ReviewLiterature
ArticulateTheory
DefineHypothesis
Testing the Hypothesis
Subject sampling Instrumentation Research design Piloting
Collecting Data Choosing analyses Conducting analyses
Conclusions
Results or Findings
![Page 3: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/3.jpg)
Sampling
![Page 4: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/4.jpg)
Target vs. Accessible Population
• High School Sports Officials
• Students enrolled in 5th grade in NM schools
• Albuquerque Residents
• The U.S. Electorate
• National Association of Sports Officials Membership
• New Mexico 5th graders
• Albuquerque Phone Book
• Registered Voters
![Page 5: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/5.jpg)
PopulationValidity
size of sample
RandomProcesses
Prosand
Cons
Sampling Procedures
Simple Random Sampling
Systematic Sampling
Cluster Sampling
Convenience Sampling
Stratified Sampling
Biased Sampling
Sampling Error
The Inferential
Leap
volunteers
Selection and
Assignment
![Page 6: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/6.jpg)
The Inferential Leap
???
???
Target Population
Accessible or Operational Population
Sample
Sampling Unit
Sampling Unit
Sampling Unit
Sampling Unit
![Page 7: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/7.jpg)
Simple Random Sampling1 092725 012157 827052 297980 625608 9641342 104460 007903 484595 868313 274221 3671813 676071 388003 266711 323324 044463 7628034 881878 862385 203886 261061 096674 8115485 534500 336348 086585 241740 581286 0084356 094276 615776 242112 985859 075388 082003
1. Andrea2. Tina3. Paul4. Wilbur5. Sandra
6. Kathy7. Jim8. George9. Emir10. Becky
11. Sharon12. Gladys13. Jose14. Bill15. Sue
16. Erica17. Aaron18. Fred19. Pam20. Roger
![Page 8: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/8.jpg)
Stratified Random Sampling1 092725 012157 827052 297980 625608 9641342 104460 007903 484595 868313 274221 3671813 676071 388003 266711 323324 044405 7628034 881878 862308 203886 261061 096674 8115485 534500 336348 086585 241740 581286 0084356 094276 615776 242112 985859 075388 082003
1. Andrea2. Tina3. Kathy4. Sandra5. Becky
6. Paul7. Wilbur8. Jim9. George10. Emir
6. Sharon7. Gladys8. Sue9. Erica10. Pam
1. Jose2. Bill3. Aaron4. Fred5. Roger
![Page 9: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/9.jpg)
Systematic Sampling
• Say you have a target population that has a 100,000 members.– And:
• A list is available.• You need 1,000 cases for your sample.
• 100,000/1000 = 100.– Select a random number from table.
• Then select every 100th case.
![Page 10: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/10.jpg)
Cluster Sampling
• Naturally occurring groups.– State, district, school, classroom, student.– Randomly sample from one level then survey,
interview, etc. • Multistage sampling.
– Randomly select from one level.– Then randomly select within that level.
![Page 11: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/11.jpg)
Convenience Sampling
• Why convenient?– Sample that is located near the researcher– Connections with administrator or staff– Researcher is familiar with the setting– Data is already available
• Shortcomings of convenience samples?
![Page 12: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/12.jpg)
Volunteers in Sampling• How might volunteers differ?• Children having parental permission
– More academically competent– More popular with peers– More physically attractive– Less likely to smoke or use drugs– More likely to be white– More likely to come from two-parent household– More likely to be involved in extracurricular activities– Less likely to be socially withdrawn– Less likely to be aggressive
![Page 13: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/13.jpg)
Size of the Sample
• Bigger is (usually) better.– Unless?
• How big is big? Power analysis.• Practical issues.• Attrition.• Reliability.• Cost/ benefit.
![Page 14: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/14.jpg)
Correlation& Instrumentation
Reliability and Validity
![Page 15: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/15.jpg)
Correlation Coefficients
• Pearson product-moment correlation– The relationship between two variables of
degree.• Positive: As one variable increases (or decreases)
so does the other.• Negative: As one variable increases the other
decreases.– Magnitude or strength of relationship
• -1.00 to +1.00– Correlation does not equate to causation
![Page 16: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/16.jpg)
Positive Correlation
![Page 17: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/17.jpg)
Negative Correlation
![Page 18: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/18.jpg)
No Correlation
![Page 19: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/19.jpg)
Correlations
• Thickness of scatter plot determines strength of correlation, not slope of line.– For example see:
• http://noppa5.pc.helsinki.fi/koe/corr/cor7.html
• Remember correlation does not equate to causation.
![Page 20: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/20.jpg)
Negative Correlation
![Page 21: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/21.jpg)
Operationism vs. Essentialism
• According to Stanovich– What are they?– How do they differ?
![Page 22: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/22.jpg)
• Essentialist – Like to argue about the meaning of our terms
• “What does the theoretical concept really mean?”– Must have a complete and unambiguous
understanding of the language involved.• Operationism
– Link concepts to observable events that can be measured.
– Concepts in science related to a set of operations.• Several slightly different tasks and behavioral events are
used to converge on a concept.
![Page 23: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/23.jpg)
Validity and Reliability
• Validity is an important consideration in the choice of an instrument to be used in a research investigation– It should measure what it is supposed to measure– Researchers want instruments that will allow them to make
warranted conclusions about the characteristics of the subjects they study
• Reliability is another important consideration, since researchers want consistent results from instrumentation– Consistency gives researchers confidence that the results
actually represent the achievement of the individuals involved
![Page 24: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/24.jpg)
Reliability
• Test-retest reliability• Inter-rater reliability• Parallel forms reliability• Internal consistency (a.k.a. Cronbach’s
alpha)
![Page 25: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/25.jpg)
Validity
• Face– Does it appear to measure what it purports to
measure?• Content
– Do the items cover the domain?• Construct
– Does it measure the unobservable attribute that it purports to measure?
![Page 26: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/26.jpg)
Validity
• Criterion– Predictive – Concurrent
• Consequential
![Page 27: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/27.jpg)
Types of validity (cont.)
The construct
The instrument
Here the instrument samples some and only of the construct
![Page 28: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/28.jpg)
Types of validity
The instrument
The construct
Here the instrument samples all and more of the construct
![Page 29: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/29.jpg)
The construct
The instrument
Here the instrument fails to sample ANY of the construct
![Page 30: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/30.jpg)
The construct
The instrument
Here the instrument samples some but not all of the construct
![Page 31: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/31.jpg)
Perfection!
The construct and the instrument!
![Page 32: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/32.jpg)
Reliability and Validity
![Page 33: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/33.jpg)
In groups of 3 to 4
• Sampling– What is the target population?– What sampling procedure was used?– Do you think the sample is representative?
• Why or why not?
• Measurement– What types of reliability and validity evidence
are provided?– What else would you like to know?
![Page 34: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/34.jpg)
Ways to Classify Instruments• Who Provides the Information?
– Themselves: Self-report data – Directly or indirectly: from the subjects of the study – From informants (people who are knowledgeable
about the subjects and provide this information)
![Page 35: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/35.jpg)
Types of Researcher-completed Instruments
• Rating scales• Interview schedules• Tally sheets• Flowcharts
• Performance checklists
• Observation forms
![Page 36: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/36.jpg)
Excerpt from a Behavior Rating Scale for Teachers
Instructions: For each of the behaviors listedbelow, circle the appropriate number, using the following key: 5 = Excellent, 4 = Above Average, 3 = Average, 2 = Below Average,1 = Poor.
A. Explains course material clearly.1 2 3 4 5
B. Establishes rapport with students.1 2 3 4 5
C. Asks high-level questions.1 2 3 4 5
D. Varies class activities.1 2 3 4 5
![Page 37: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/37.jpg)
Excerpt from a Graphic Rating Scale
Instructions: Indicate the quality of the student’s participationin the following class activities by placing an X anywhere alongeach line.
AlwaysFrequently Occasionally Seldom Never
1. Listens to teacher’s instructions.
Always Frequently Occasionally Seldom Never
2. Listens to the opinions of other students.
Always Frequently Occasionally Seldom Never
3. Offers own opinions in class discussions.
![Page 38: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/38.jpg)
Sample Observation Form
![Page 39: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/39.jpg)
Discussion Analysis Tally Sheet
![Page 40: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/40.jpg)
Performance Checklist Noting Student Actions
![Page 41: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/41.jpg)
Types of Subject-completed Instruments
• Questionnaires• Self-checklists• Attitude scales• Personality
inventories
• Achievement/aptitude tests
• Performance tests• Projective devices
![Page 42: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/42.jpg)
Example of a Self-Checklist
![Page 43: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/43.jpg)
Example of Items from a Likert Scale
![Page 44: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/44.jpg)
Example of the Semantic Differential
![Page 45: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/45.jpg)
Pictorial Attitude Scale for Use with Young Children
![Page 46: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/46.jpg)
Sample Items from a Personality Inventory
![Page 47: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/47.jpg)
Sample Items from an Achievement Test
![Page 48: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/48.jpg)
Sample Item from an Aptitude Test
![Page 49: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/49.jpg)
Sample Items from an Intelligence Test
![Page 50: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/50.jpg)
Item Formats• Questions used in a subject-completed instrument can
take many forms but are classified as either selection or supply items.
• Examples of selection items are:• True-false items• Matching items• Multiple choice items• Interpretive exercises
• Examples of supply items are:• Short answer items• Essay questions
![Page 51: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/51.jpg)
Unobtrusive Measures• Many instruments require the cooperation of the respondent in one
way or another.• An intrusion into an ongoing activity could be involved which causes
a form of negativity within the respondent. • To eliminate this, researchers use unobtrusive measures, data
collection procedure that involve no intrusion into the naturally occurring course of events.
• In most cases, no instrument is used, however, good record keeping is necessary.
• They are valuable as supplements to the use of interviews and questionnaires, often providing a useful way to corroborate what more traditional data sources reveal.
![Page 52: Part 2: Quantitative Methods](https://reader036.fdocuments.in/reader036/viewer/2022081514/568167fd550346895ddd77b7/html5/thumbnails/52.jpg)
Norm-Referenced vs. Criterion-Referenced Instruments
• All derived scores give meaning to individual scores by comparing them to the scores of a group.
• The group used to determine derived scores is called the norm group and the instruments that provide such scores are referred to as norm-referenced instruments.
• An alternative to the use of achievement or performance instruments is to use a criterion-referenced test.
• This is based on a specific goal or target (criterion) for each learner to achieve.
• The difference between the two tests is that the criterion referenced tests focus more directly on instruction.