Copyright © Allyn & Bacon (2010) A Second Look at Field Research Graziano and Raulin Research...

Copyright © Allyn & Bacon (2010)Copyright © Allyn & Bacon (2010)

A Second Look atA Second Look atField ResearchField Research

Graziano and RaulinGraziano and RaulinResearch Methods: Chapter 13Research Methods: Chapter 13This multimedia product and its contents are protected under copyright law. The following are This multimedia product and its contents are protected under copyright law. The following are prohibited by law: (1) Any public performance or display, including transmission of any image prohibited by law: (1) Any public performance or display, including transmission of any image over a network; (2) Preparation of any derivative work, including the extraction, in whole or in over a network; (2) Preparation of any derivative work, including the extraction, in whole or in part, of any images; (3) Any rental, lease, or lending of the program.part, of any images; (3) Any rental, lease, or lending of the program.


Why Do Field Why Do Field Research?Research? Many reasons for doing field researchMany reasons for doing field research

– Test external validity of laboratory studiesTest external validity of laboratory studies Do the laboratory findings really indicate what Do the laboratory findings really indicate what

happens in the natural world?happens in the natural world?

– Study events that only occur in the fieldStudy events that only occur in the field Evaluate the effectiveness of social programsEvaluate the effectiveness of social programs

– Improve generalization across settingsImprove generalization across settings By testing your hypotheses in those settingsBy testing your hypotheses in those settings

Field research is becoming more Field research is becoming more commoncommon


Difficulties in Field Difficulties in Field ResearchResearch More difficult to control variables in field More difficult to control variables in field

studiesstudies– Sometimes we have no control over the Sometimes we have no control over the

independent variable (e.g., a natural disaster)independent variable (e.g., a natural disaster)– Often we cannot randomly assign participants Often we cannot randomly assign participants

to conditionsto conditions Even though we may not have complete Even though we may not have complete

control, we can often get sufficient control control, we can often get sufficient control to allow us to draw valid conclusionsto allow us to draw valid conclusions

Flexibility in ResearchFlexibility in Research

People don’t usually do research the People don’t usually do research the way people who write books about way people who write books about research say that people do research. research say that people do research. (Bachrach, 1981)(Bachrach, 1981)

Real research is a mixture of Real research is a mixture of – Hardheaded precision in research designHardheaded precision in research design– Creativity, flashes of insight, and alertness Creativity, flashes of insight, and alertness

to interesting phenomenato interesting phenomena



Quasi-Experimental Quasi-Experimental DesignsDesigns Similar to, but not quite equal to, an Similar to, but not quite equal to, an

experimental designexperimental design– Approximates the control of an experimentApproximates the control of an experiment– Researcher does not control key variablesResearcher does not control key variables

Manipulation of the independent variableManipulation of the independent variable Assignment of participants to conditionsAssignment of participants to conditions

Two types of quasi-experimental Two types of quasi-experimental designsdesigns– Nonequivalent control-group designNonequivalent control-group design– Interrupted time-series designInterrupted time-series design


NonequivalentNonequivalentControl-Group DesignControl-Group Design First introduced in Chapter 10 as the pretest-First introduced in Chapter 10 as the pretest-

posttest natural control group designposttest natural control group design The more similar the groups, the closer this The more similar the groups, the closer this

design is to a true experimentdesign is to a true experiment


Nonequivalent Nonequivalent Control-Group DesignControl-Group Design Sometimes participants are, for all Sometimes participants are, for all

practical purposes, randomly practical purposes, randomly assigned by nature (essentially a assigned by nature (essentially a natural experiment)natural experiment)

Other times, the groups are clearly Other times, the groups are clearly different, although careful analysis different, although careful analysis and cautious interpretation may and cautious interpretation may still produce valid conclusionsstill produce valid conclusions


NonequivalentNonequivalentControl-Group DesignsControl-Group Designs

Sometimes it is Sometimes it is possible to interpret possible to interpret the results from a the results from a nonequivalent nonequivalent control-group study, control-group study, even when the even when the groups are clearly groups are clearly different prior to the different prior to the interventionintervention

Only (b) and (f) Only (b) and (f) represent data represent data suggesting an suggesting an experimental effectexperimental effect


Interrupted Time-Interrupted Time-Series DesignSeries Design A single group of participants is tested A single group of participants is tested

repeatedly before and after a repeatedly before and after a manipulation or a natural eventmanipulation or a natural event– Essentially a single-group, pretest-posttest Essentially a single-group, pretest-posttest

design with multiple before and after design with multiple before and after measuresmeasures

The multiple measures allow us to The multiple measures allow us to detect many confounding variablesdetect many confounding variables– Regression to the mean, history, Regression to the mean, history,

maturation, etc.maturation, etc.


InterruptedInterruptedTime-Series DesignTime-Series Design


Relaxation TrainingRelaxation Trainingand Disruptive and Disruptive BehaviorBehavior


Adding a Control Adding a Control GroupGroup The interrupted time-series design The interrupted time-series design

does not need a control groupdoes not need a control group But adding a control group makes But adding a control group makes

the design even more powerfulthe design even more powerful– Guerin and MacKinnon (1985) added two Guerin and MacKinnon (1985) added two

natural control groups in their study of the natural control groups in their study of the effects of a child car seat law in Californiaeffects of a child car seat law in California

Older children not covered by the law in CaliforniaOlder children not covered by the law in California Children of the same age in TexasChildren of the same age in Texas


Guerin and MacKinnon Guerin and MacKinnon (1985)(1985)


Program EvaluationProgram Evaluation

One of the fastest growing areas of One of the fastest growing areas of researchresearch– Demands to evaluate the effectiveness of Demands to evaluate the effectiveness of

costly programs before they are renewedcostly programs before they are renewed– Program evaluation can provide field tests Program evaluation can provide field tests

of laboratory theoriesof laboratory theories Program evaluation uses the designs Program evaluation uses the designs

and controls already covered, with a and controls already covered, with a few twistsfew twists– Making the designs work in the field is both Making the designs work in the field is both

a research and political challengea research and political challenge


Control in Program Control in Program EvaluationEvaluation Selecting appropriate measuresSelecting appropriate measures

– Normally need several dependent measures to Normally need several dependent measures to evaluate all of the factors expected to changeevaluate all of the factors expected to change

– Should include at least one satisfaction Should include at least one satisfaction measuremeasure

Avoiding bias in the evaluationAvoiding bias in the evaluation– Use objective measures whenever possibleUse objective measures whenever possible– Measure community-wide effects, which are Measure community-wide effects, which are

less susceptible to response set biasesless susceptible to response set biases Use strongest research design Use strongest research design

possiblepossible


Program Evaluation Program Evaluation DesignsDesigns Randomized control-group designRandomized control-group design

– Excellent control, but may be impossibleExcellent control, but may be impossible Nonequivalent control-group designNonequivalent control-group design

– Natural control groups may add much to an Natural control groups may add much to an otherwise weak designotherwise weak design

Single-group, time-series designSingle-group, time-series design– Best alternative if no control group is Best alternative if no control group is

availableavailable Pretest-posttest designPretest-posttest design

– A weak design; not recommended A weak design; not recommended


Survey ResearchSurvey Research

A widely used research techniqueA widely used research technique– Provides information about people’s Provides information about people’s

attitudes, experience, and knowledgeattitudes, experience, and knowledge– Used extensively today by researchers, Used extensively today by researchers,

politicians, and news organizationspoliticians, and news organizations First introduced in the 1830s in First introduced in the 1830s in

England to study the impact of the England to study the impact of the Industrial Revolution on people’s Industrial Revolution on people’s liveslives


Types of SurveysTypes of Surveys

Status surveysStatus surveys– Descriptive survey about the current Descriptive survey about the current

status of the population sampledstatus of the population sampled– Descriptive information can guide Descriptive information can guide

policy and inform policy makerspolicy and inform policy makers Survey researchSurvey research

– Seeks to identify relationships Seeks to identify relationships among the variables studied in the among the variables studied in the surveysurvey


Steps in Survey Steps in Survey ResearchResearch

1.1. Determine what area of information Determine what area of information is to be sought.is to be sought.

2.2. Define the population to be studied.Define the population to be studied.

3.3. Decide how the survey is to be Decide how the survey is to be administered.administered.

4.4. Construct the first draft of the survey Construct the first draft of the survey instrument; edit and refine the draft.instrument; edit and refine the draft.


Steps in Survey Steps in Survey ResearchResearch

5.5. Pretest the survey with a subsample; Pretest the survey with a subsample; refine it further.refine it further.

6.6. Develop a sampling frame and draw Develop a sampling frame and draw a representative sample.a representative sample.

7.7. Administer the final form of the Administer the final form of the instrument to the sample.instrument to the sample.

8.8. Analyze, interpret, and communicate Analyze, interpret, and communicate the results.the results.


The Survey InstrumentThe Survey Instrument

Group-administered surveyGroup-administered survey– Written survey with instructions includedWritten survey with instructions included– Administered in groups, through the mail, Administered in groups, through the mail,

or on the Internetor on the Internet Individual survey scheduleIndividual survey schedule

– Administered in person or over the phoneAdministered in person or over the phone– Can be rather complex, provided there is a Can be rather complex, provided there is a

clear procedure for the interviewer to followclear procedure for the interviewer to follow– Allows clarification more easily than in Allows clarification more easily than in

group-administered surveysgroup-administered surveys


Developing a Survey Developing a Survey InstrumentInstrument Use clear and explicit instructionsUse clear and explicit instructions Types and number of questions Types and number of questions

will depend on the purpose and will depend on the purpose and the type of survey plannedthe type of survey planned

Types of itemsTypes of items– Open-ended itemsOpen-ended items– Multiple-choice itemsMultiple-choice items– Likert-scale itemsLikert-scale items


Sample Open-Ended Sample Open-Ended QuestionsQuestions How do you handle interpersonal How do you handle interpersonal

difficulties with your co-workers?difficulties with your co-workers? What are the most important What are the most important

values to instill into today’s values to instill into today’s children?children?

If you were president, what issues If you were president, what issues would be your highest priority?would be your highest priority?

What situations are particularly What situations are particularly stressful for you?stressful for you?


Sample Multiple-Sample Multiple-Choice QuestionsChoice Questions How frequently How frequently

do you take a do you take a sick day from sick day from work?work?a) nevera) never

b) once or twice a yearb) once or twice a year

c) 3 to 5 times a yearc) 3 to 5 times a year

d) 6 to 12 times a yeard) 6 to 12 times a year

e) at least once a e) at least once a monthmonth

Identify the issue Identify the issue that you believe that you believe is most critical to is most critical to this country’s this country’s future.future.a) the economya) the economyb) educationb) educationc) integrity in c) integrity in

governmentgovernmentd) national defensed) national defensee) some other issuee) some other issue


Sample Likert-Scale Sample Likert-Scale QuestionsQuestions

Rate each item on the scale shown to Rate each item on the scale shown to indicate your level of agreement:indicate your level of agreement:

I believe in the Bill of Rights.I believe in the Bill of Rights.strongly agree agree uncertain disagree strongly strongly agree agree uncertain disagree strongly disagreedisagree

I think that everyone should vote.I think that everyone should vote.strongly agree agree uncertain disagree strongly strongly agree agree uncertain disagree strongly disagreedisagree

Most politicians cannot be trusted.Most politicians cannot be trusted.strongly agree agree uncertain disagree strongly strongly agree agree uncertain disagree strongly disagreedisagree


Issues in SamplingIssues in Sampling

Need to define the population and Need to define the population and then develop a strategy for then develop a strategy for adequately sampling from the adequately sampling from the populationpopulation– Identify each member of the populationIdentify each member of the population– Sample from the comprehensive listSample from the comprehensive list

The more representative the sample, The more representative the sample, the more valid the conclusions from the more valid the conclusions from the surveythe survey


Sampling ProceduresSampling Procedures

Nonprobability samplesNonprobability samples– Convenience sample gathered quickly, but Convenience sample gathered quickly, but

with little interest in drawing strong with little interest in drawing strong conclusionsconclusions

Probability samplesProbability samples– Simple random sampling (Every person Simple random sampling (Every person

has an equal chance of being included)has an equal chance of being included)– Stratified random samplingStratified random sampling

Random sampling within clearly defined strata Random sampling within clearly defined strata (subdivisions of the population)(subdivisions of the population)


Sample SizeSample Size

Sample size is based on several Sample size is based on several factorsfactors– Costs and time constraintsCosts and time constraints– Degree of precision neededDegree of precision needed

Larger samples provide more precise estimates Larger samples provide more precise estimates of population parametersof population parameters

More heterogeneous populations require larger More heterogeneous populations require larger samplessamples

The larger the sample, the more The larger the sample, the more narrow the confidence intervalsnarrow the confidence intervals


Survey Research Survey Research DesignDesign Cross-sectional designsCross-sectional designs

– Administer the survey once to a sampleAdminister the survey once to a sample Longitudinal designsLongitudinal designs

– Repeatedly survey the same participantsRepeatedly survey the same participants– Allows us to see changes in attitudes, Allows us to see changes in attitudes,

experience, and knowledge over timeexperience, and knowledge over time– Difficult to get participants to submit to Difficult to get participants to submit to

such long-term researchsuch long-term research

Ethical PrinciplesEthical Principles

Several ethical issues in program Several ethical issues in program evaluation researchevaluation research– Must avoid any hint of coercionMust avoid any hint of coercion– Long baselines may be unreasonableLong baselines may be unreasonable– Random assignment may be Random assignment may be

unreasonableunreasonable– No treatment control groups may be No treatment control groups may be

unreasonableunreasonable



Field research is becoming more commonField research is becoming more common Field research need not be low constraintField research need not be low constraint Types of field researchTypes of field research

– Quasi-experimental designsQuasi-experimental designs– Program evaluationProgram evaluation– Survey researchSurvey research

Field studies: ultimate test of a hypothesisField studies: ultimate test of a hypothesis– Is this how nature really works?Is this how nature really works?

SummarySummary

Copyright © Allyn & Bacon (2010) A Second Look at Field Research Graziano and Raulin Research...

Documents

Transcript of Copyright © Allyn & Bacon (2010) A Second Look at Field Research Graziano and Raulin Research...