Post on 03-Jan-2016
Research MethodsContentContentAreaArea
ResearchableQuestions
ResearchResearchDesignDesign
MeasurementMeasurementMethodsMethods
SamplingSampling DataDataCollectionCollection
StatisticalStatistical AnalysisAnalysis
ReportReportWritingWriting??
Error component may be either:
• Random Error = Varaiation due to unknown or uncontrolled factors
• Systematic Error = variation due to systematic but irrelevant elements of the design
• Concern of scientific research is management of the error component
• Number of criteria by which to evaluate success
1 .Reliability
• Does the measure consistently reflect changes in what it purports to measure?
• Consistency or stability of data across• Time• Circumstances
• Balance between consistency and sensitivity of measure
2 .Validity
• Does the measure actually represent what it purports to measure?
• Accuracy of the data (for what?)• Number of different types:
A. Internal Validity
Semmelweis Pasteur
Lister
Semmelweis Pasteur
Lister
• Effects of an experiment are due solely to the experimental conditions
• Extent to which causal conclusions can be drawn
• Dependent upon experimental control• Trade-off between high internal
validity and generalizability of results
B. External Validity• Can the results of an experiment be
applied to other individuals or situations?
• Extent to which results can be generalized to broader populations or settings
• Dependent upon sampling subjects and occasions
• Trade-off between high generalizability and internal validity
C. Construct Validity
• Whether or not an abstract, hypothetical concept exists as postulated
• Examples of Constructs:• Intelligence• Personality• Conscience
Based on:
• Convergence = different measures that purport to measure the same construct should be highly correlated (similar) with one another
• Divergence = tests measuring one construct should not be highly correlated (similar) to tests purporting to measure other constructs
D. Statistical Conclusion Validity
• The extent to which a study has used appropriate design and statistical methods to enable it to detect the effects that are present
• The accuracy of conclusions about covariation made on the basis of statistical evidence
Can have a reliable, but invalidCan have a reliable, but invalidmeasuremeasure..
If measure is valid, thenIf measure is valid, thennecessarily reliablenecessarily reliable..
A. Efficient Methods provide:
• Precise, reliable data with relatively low costs in:
• time• materials• equipment• personnel
B. Generality
• Refers to the extent to which a method can be applied successfully to a wide range of phenomena
• a.k.a. Generalizability
Threats to Validity
• Numerous ways vailidity can be threatend
• Related to Design
• Related to Experimenter
Related to Design
1. Threats to Internal Validity (Cook & Campbell, 1979)
A. History = specific events occurring to individual subjectB. Testing = repeated exposure to testing instrumentC. Instrumentation=changes in the scoring procedure over time
D. Regression = reversion of scores toward the mean or toward less extreme scoresE. Mortatility = differential attrition across groupsF. Maturation = developmental processesG. Selection = differential composition of subjects among samples
H. Selection by Maturation interaction
I. Ambiguity about casual direction
J. Diffusion of Treatments = information spread between groups
K. Compensatory Equalization of Treatments = lack of treatment integrity
L. Compensatory Rivalry = “John Henry” effect on nonparticpants
2. Threats to External Validity (LeCompte & Goetz, 1982)
A. Selection = results sample-specific
B. Setting = results context-specific
C. History = unique experiences of sample limit generalizability
D. Construct efffects = constricts are sample specific
Related to Experimenter
1. Noninteractional ArtifactsA. Observer Bias = over/under estimation of phenomenon (schema)
B. Interpreter Bias = error in interpretation of data
C. Intentional Bias = fabrication or fraudulent interpretation of data
2. Interactional ArtifactsA. Biosocial Effects = errors attributable to biosocial attributes of researcher
B. Psychosocial Effects = errors attributable to psychosocial attributes of researcher
C. Situational Effects = errors attributable to research setting and participants
D. Modeling Effects = errors attributable to example set by researcher
E. Experimenter Expectancy Bias = researchers treatment of participants elicits confirmatory evidence of hypothesis
BasicAppliedPurpose
Context
Methods
•Expand Knowledge
•Academic Setting•Single Researcher•Less Time/Cost Pressure
•Internal Validity•Cause•Single Level of Analysis•Single Method•Experimental•Direct Observations
•Understand Specific Problem
•Real-World Setting•Multiple Researchers•More Time/Cost Pressure
•External Validity•Effect•Multiple Levels of Analysis•Multiple Methods•Quasi-Experimental•Indirect Observations