Selecting aSelecting a

SampleSample

SamplingSampling

• Select participants for studySelect participants for study

• Must represent a larger groupMust represent a larger group

• Picked from a populationPicked from a population

It’s all about convenienceIt’s all about convenience

• Total population too largeTotal population too large

• Would be too costlyWould be too costly

• Sample done well, results can Sample done well, results can become a generalizationbecome a generalization

Defining PopulationsDefining Populations

• Target – entire group (ex. All Target – entire group (ex. All Maryland 5Maryland 5thth grade students) grade students)

• Accessible – realistic number of Accessible – realistic number of participantsparticipants

Simple random samplingSimple random sampling

• All individuals in populationAll individuals in population– Have equal and independent chance of Have equal and independent chance of

being selectedbeing selected– Best way to obtain a representative Best way to obtain a representative

sample of your populationsample of your population– Table of random numbers may be usedTable of random numbers may be used

Stratified SamplingStratified Sampling

• Examples in textExamples in text

• Equal-sized stratified groupsEqual-sized stratified groups

• Proportional stratifiedProportional stratified

Cluster samplingCluster sampling

• Intact groups are selectedIntact groups are selected

• Example – instead of a sample of all Example – instead of a sample of all 55thth graders, one class would be used graders, one class would be used

• Similar characteristics and abilitiesSimilar characteristics and abilities

• Systematic SamplingSystematic Sampling– Rarely used (every Kth name) see pg. Rarely used (every Kth name) see pg.

108108

Sample size?Sample size?

• No agreement hereNo agreement here

• 30 is usual guide for some 30 is usual guide for some quantitativequantitative

• 60 needed however for causal-60 needed however for causal-comparative and experimentalcomparative and experimental

Qualitative samplingQualitative sampling

• Select small numberSelect small number

• Persons selected must help Persons selected must help researcher understand certain researcher understand certain phenomenonphenomenon

• Intensity, homogenous, snowball Intensity, homogenous, snowball sampling are a few examples from sampling are a few examples from texttext

MeasuringMeasuring

InstrumentsInstruments

DataData

• All types of research require the All types of research require the collection, analysis, and collection, analysis, and interpretation of data.interpretation of data.

• Data are pieces of evidence used to Data are pieces of evidence used to examine a research topic or test a examine a research topic or test a hypothesis.hypothesis.

ConstructsConstructs

• Mental abstractions such as Mental abstractions such as personality, creativity, and intelligence personality, creativity, and intelligence that cannot be measured or observed that cannot be measured or observed directly.directly.

• Constructs become variables when Constructs become variables when different levels or scores can be used different levels or scores can be used to measure the constructs (example: to measure the constructs (example: Personality / introverts vs. extroverts).Personality / introverts vs. extroverts).

Variables/InstrumentsVariables/Instruments

• Variable Variable – A concept that can assume any one of a A concept that can assume any one of a

range of values, e.g., intelligence, range of values, e.g., intelligence, height, aptitude.height, aptitude.

• InstrumentInstrument– Tool used to collect data (Iowa Test of Tool used to collect data (Iowa Test of

Basic Skills).Basic Skills).

Characteristics of Measuring Characteristics of Measuring InstrumentsInstruments

• Three main ways to collect data:Three main ways to collect data:– Administer an existing instrument.Administer an existing instrument.– Construct an instrument.Construct an instrument.– Observation(record naturally occurring Observation(record naturally occurring

events).events).– Collect existing data.Collect existing data.

Measurement ScaleMeasurement Scale

• Group of several related statements Group of several related statements that participants select to indicate that participants select to indicate degree of agreement or lack of degree of agreement or lack of agreement.agreement.

• There are four types of measurement There are four types of measurement scales and associated variables.scales and associated variables.

Nominal VariablesNominal Variables

• Also called categorical variable.Also called categorical variable.• Represents lowest form of Represents lowest form of

measurement.measurement.• Simply classifies persons.Simply classifies persons.

– Female; maleFemale; male– Public or private schoolPublic or private school

• Represented by a number (1-female; Represented by a number (1-female; 2-male) – doesn’t mean one category 2-male) – doesn’t mean one category is higher or lower than another.is higher or lower than another.

Ordinal variablesOrdinal variables

• Classifies persons or objects.Classifies persons or objects.

• In addition, it ranks themIn addition, it ranks them– Ex. – height, #1 = 6’ 5” ; #2 = 6’ 0”Ex. – height, #1 = 6’ 5” ; #2 = 6’ 0”

Interval variablesInterval variables

• Has all characteristics of nominal and Has all characteristics of nominal and ordinal variables.ordinal variables.

• Also has equal intervals.Also has equal intervals.– The difference between a score of 70 The difference between a score of 70

and 80 is basically the same as the and 80 is basically the same as the difference between a 90 and 100.difference between a 90 and 100.

– A score of zero does not mean a total A score of zero does not mean a total lack of knowledge.lack of knowledge.

Ratio variablesRatio variables

• Represents highest level of Represents highest level of measurement.measurement.

• In addition to having all In addition to having all characteristics of the three previous characteristics of the three previous variables, it has a true zero point.variables, it has a true zero point.

• Mainly used for physical measures.Mainly used for physical measures.– One person is 5 feet tall, and her friend One person is 5 feet tall, and her friend

is two thirds as tall as she.is two thirds as tall as she.

Qualitative and Quantitative Qualitative and Quantitative VariablesVariables

• Nominal variablesNominal variables– Show qualitative differences onlyShow qualitative differences only– i.e. religion, gender, political partyi.e. religion, gender, political party

• Ordinal, interval, and ratio variablesOrdinal, interval, and ratio variables– Show quantitative differencesShow quantitative differences– i.e. test scores, heights, age, class sizei.e. test scores, heights, age, class size

Dependent and Dependent and IndependentIndependent

• Dependent variableDependent variable– Depends on or caused by:Depends on or caused by:

• Independent variableIndependent variable– Manipulated by researcherManipulated by researcher

Testing InstrumentsTesting Instruments

• It is more time efficient and less It is more time efficient and less skillful to select an existing skillful to select an existing instrument than to create a new one instrument than to create a new one that measures the same thing.that measures the same thing.

Standardized TestsStandardized Tests

• There are thousands of standardized There are thousands of standardized tests that measure a broad range of tests that measure a broad range of variables.variables.

• A standardized test is one that is A standardized test is one that is administered, scored, and interpreted administered, scored, and interpreted in the same way no matter when and in the same way no matter when and where it is administered.where it is administered.

TestsTests

• Most tests are paper and pencil.Most tests are paper and pencil.

• Common formats are: Common formats are: – true/falsetrue/false– multiple choicemultiple choice– essay/short answeressay/short answer

Interpreting Test ResultsInterpreting Test Results

• Raw scores indicate the number of Raw scores indicate the number of items a person got right.items a person got right.

• They can be transformed into They can be transformed into derived scores (percentile ranks, derived scores (percentile ranks, stanines, and standard scores).stanines, and standard scores).

• Derived scores are most often used Derived scores are most often used with standardized tests.with standardized tests.

Norm Reference ScoresNorm Reference Scores

• Compares a students test Compares a students test performance to the performance of performance to the performance of other test takers.other test takers.

Criterion-Reference ScoresCriterion-Reference Scores

• Compares a student’s test Compares a student’s test performance to predetermined performance to predetermined standards of performance.standards of performance.

Self Reference ScoresSelf Reference Scores

• Compares a students performance Compares a students performance over time.over time.

• Looks for improvement or regressionLooks for improvement or regression– Similar or equal assessment is repeated Similar or equal assessment is repeated

over timeover time

Types of TestsTypes of Tests

• Achievement tests measure the Achievement tests measure the current status of individuals on current status of individuals on school taught subjects.school taught subjects.

• Aptitude tests are used to predict Aptitude tests are used to predict how well a test taker is likely to how well a test taker is likely to perform in the future.perform in the future.

Types of TestsTypes of Tests

• Affective tests measure characteristics Affective tests measure characteristics such as interest, values, attitude, and such as interest, values, attitude, and personality.personality.

• Projective tests are not self-reporting Projective tests are not self-reporting instruments. They present an ambiguous instruments. They present an ambiguous situation to the test taker that requires situation to the test taker that requires him or her to “project” his or her true him or her to “project” his or her true feelings on the ambiguous situation.feelings on the ambiguous situation.

Validity of Measuring Validity of Measuring InstrumentsInstruments

• Most important quality of a test. It is Most important quality of a test. It is the degree to which a test measures the degree to which a test measures what it is supposed to measure, and what it is supposed to measure, and consequently, permits appropriate consequently, permits appropriate interpretations of test scores.interpretations of test scores.

• A test is not valid per se; it is valid A test is not valid per se; it is valid for a particular interpretation and for for a particular interpretation and for a particular group. Each intended a particular group. Each intended score interpretation requires its own score interpretation requires its own validation.validation.

ValidationValidation

• Validation is a matter of degree; Validation is a matter of degree; tests are not valid or invalid, they are tests are not valid or invalid, they are highly valid, moderately valid, or highly valid, moderately valid, or generally invalid.generally invalid.

• The three main forms of validity are The three main forms of validity are content, criterion-related, and content, criterion-related, and construct. They are viewed as construct. They are viewed as interrelated, not independent interrelated, not independent aspects of validity.aspects of validity.

Content ValidityContent Validity

• The degree to which a test measures The degree to which a test measures an intended content area. It requires an intended content area. It requires both item validity and sampling both item validity and sampling validity.validity.

• Item validity is concerned with Item validity is concerned with whether the test items are relevant whether the test items are relevant to the intended content area.to the intended content area.

Content ValidityContent Validity

• Content validity is concerned with Content validity is concerned with how well the test sample represents how well the test sample represents the total content area.the total content area.

• Content validity is of prime Content validity is of prime importance for achievement tests.importance for achievement tests.

• Content validity is determined by Content validity is determined by expert judgment of item and sample expert judgment of item and sample validity, not statistical means.validity, not statistical means.

Criterion-Related ValidityCriterion-Related Validity

• Concurrent ValidityConcurrent Validity– the degree to which the scores on a test are the degree to which the scores on a test are

related to scores on another test related to scores on another test administered.administered.

• Predictive ValidityPredictive Validity– the degree to which scores can determine the degree to which scores can determine

how well someone will do in a future situation.how well someone will do in a future situation.

–

Construct ValidityConstruct Validity

• The most important form of validity.The most important form of validity.

• Seeks to determine whether the Seeks to determine whether the construct underlying a variable is construct underlying a variable is actually being measured.actually being measured.

Construct ValidityConstruct Validity

• Determined by a series of validation Determined by a series of validation studies that can include content and studies that can include content and criterion-related approaches.criterion-related approaches.

Construct ValidityConstruct Validity

• The validity of any test can be The validity of any test can be diminished by factors such as:diminished by factors such as:– unclear test directionsunclear test directions– inappropriate vocabularyinappropriate vocabulary– subjective scoringsubjective scoring– failing to follow administration failing to follow administration

procedures.procedures.

Reliability of Measuring Reliability of Measuring InstrumentsInstruments

• The degree to which a test measures The degree to which a test measures whatever it measures.whatever it measures.

• It is expressed numerically, usually It is expressed numerically, usually as a coefficient ranging from 0.0 to as a coefficient ranging from 0.0 to 1.0; a high coefficient indicates high 1.0; a high coefficient indicates high reliability.reliability.

ReliabilityReliability

• Scores obtained when administered Scores obtained when administered in November 2005 would be similar in November 2005 would be similar to results from the same test in to results from the same test in December 2005December 2005

• Same test-takers would be usedSame test-takers would be used