PSYB01 Final Exam Notes: chapters...
Transcript of PSYB01 Final Exam Notes: chapters...
PSYB01 Final Exam Notes: chapters 6-10
Chapter 6 - Observational Methods
Quantitative and Qualitative Approaches Observational methods can be broadly classified as primarily quantitative or qualitative.
Qualitative research focuses on people behaving in natural settings and describing their world in their own ways.
Quantitative research tends to focus on specific behaviors that can be easily quantified.
Naturalistic Observation Sometimes called field work or simply field observation.
Researchers makes observations in a particular natural setting (the field) over an extended period of time, using a variety
of techniques to collect info.
Reports includes observations and researchers interpretations of findings
Used to study many phenomena in all types of social and organizational settings (social sciences).
Has roots in anthropology and animal behavior.
A researcher uses natural observation when he/she wants to describe and understand how people in a social or cultural
setting live, work and experience the setting.
Description and Interpretation of Data
Natural observation demands that researchers immerse themselves in the situation.
Field researcher observes everything - the setting, the patterns of personal relationships, peoples reactions to events, and
so on.
Researchers needs to keep detailed field notes
Goal is to provide a complete/accurate picture rather than to test hypotheses formed prior to study.
Techniques:
observe people and events
interview key informants to provide inside info
talking to people about their lives
examining documents produced int he setting (newspapers, newsletters, or memos)
audio and videotape recordings
Natural observation = primarily qualitative, however could still be quantitative
Researcher must generate hypotheses that explain data and make it understandable = interpret
Analysis is done by building a coherent structure to describe the observations
Good natural observation report will support analysis by using multiple confirmations
Issues in Naturalistic Observation
Participation and Concealment
Is the researcher a participant or non-participant?
Will researcher conceal purposes from other people in the setting or not?
Qualitative Quantitative
- in depth info on few individuals or within a
very limited setting
- conclusions based on interpretations drawn by
investigator
- ex: focus groups (themes that arise) = non-
numerical
- larger samples
- conclusions based upon statistical analysis
- ex: surveys (numerical values assigned to
responses)
Potential problem with participating is losing objectivity necessary to conduct scientific observation. (especially if initially
a part of a group being observed).
Concealed observation may be preferable because presence of observer may influence and alter behavior of those being
observed = less reactive.
Still non-concealed observation may be preferable from an ethical viewpoint.
People often become used to observer and act naturally (ex: The Real World show)
The decision about concealment depends on both ethical concerns and nature of particular group/setting being studied.
There are degrees of participation and concealment = therefore researcher must carefully determine what their role in the
setting will be.
When anonymity is NOT threatened = exempt research = no informed consent necessary.
In non-concealed observation = IC verbal or written
Must be sensitive to ethical issues; whether observations are made in a public place with no clear expectations that
behaviors are private.
Defining the Scope of the Observation
Researchers must limit the scope of their observations to behaviors that are relevant to the central issues of their study.
Limits of Naturalistic Observation
Most useful when investigating complex social settings both to understand the setting and to develop theories based on
observations.
Less useful for studying well defined hypotheses under precisely specified conditions.
Field research is very difficult to do = ever changing pattern of events = time consuming.
Must ensure data is consistent with hypotheses.
Negative case analysis: an observation that does not fit the explanatory structure devised by the researcher = therefore
must revise hypothesis.
Systematic Observation Refers to the careful observation of one or more specific behaviors in a particular setting.
Much less global than natural observation.
This research is interested in only a few very specific behaviors, the observations are quantifiable and the research
frequently has developed prior hypotheses about the behaviors.
Coding Systems
Researcher must choose a setting, which behaviors are of interest and develop a coding system to measure behaviors.
Should be simple allowing observers to easily categorize behaviors.
Researchers can use coding systems that have been developed by others; advantage: system has been proven useful and
training materials are usually available.
Ex: FICS an SYMLOG
Methodological Issues
Equipment
Video recorders have the advantage of providing a permanent record of behavior that can be coded later.
Computer-based recording devices can be used to code the observed behaviors as well as to keep track of duration.
Reactivity
The possibility that the presence of observers will affect peoples behavior.
Reliability
Indicated by high agreement among raters (who code behavior).
Sampling
For many research questions, samples of behavior taken over a long period over time provide more accurate and useful
data than single, short observations.
Case Studies Provides a description of an individual = person or setting.
Case studies and natural observation overlap. However case studies do no necessarily involve natural observation.
a psychobiography is a type of case study in which a researcher applies psychological theory to explain the life of an
individual, usually an important historical figure.
Typically a case study is done when an individual possesses a particularly rare condition.
Case study presents: individuals history, symptoms, characteristic behaviors, reactions to situations or responses to
treatment.
Case studies are valuable in informing us of conditions that are rare or unusual and thus providing unique data about some
psychological phenomenon such as memory, language or social exchange.
Insights gained through case studies may also lead to development of hypothesis that can be tested using other methods.
Archival Research Involves using previously compiled info to answer research questions.
Analyze existing data
3 types: statistical record, survey archives and written records.
Statistical Records
Collected by many public and private organizations
The US Census Bureau maintain the most extensive set of statistical records available to researchers for analysis.
Public records can also be used as sources of archival data.
Survey Archives
Consist of data from surveys that are stored on computers and available tp researchers who wish to analyze them.
One very useful data set is the General Social Survey
Extremely important because most researchers do not have the financial resources to conduct surveys of randomly
selected national samples; the archives allow them to access such samples to test their ideas.
Written and Mass Communication Records
Written records are documents such as diaries and letters that have been preserved by historical societies, ethnographies,
public documents.
Mass communication records include books, magazine articles, movies, television programs and newspapers.
Archival data may also be used in cross-cultural research to examine aspects of social structure that differ from society to
society.
HRAF = Humans Relations Area Files = info from ethnographies.
Content Analysis of Documents
The systematic analysis of existing documents such as the ones described in this section.
Like systematic observation, content analysis requires researchers to devise coding system that raters can use to quantify
the info in the documents.
Researchers must define categories in order to code the info.
Use of archival data are a valuable supplement to traditional data collection methods.
Two problems with the use of archival data: 1) Desired records may be difficult to obtain
2) Can never be sure of accuracy of info collected by someone else.
Chapter 7: Asking People About Themselves - Survey Research
Why Conduct Surveys Common and important method of studying behavior
Methodology for asking people to tell us about themselves = society demands data about issues rather than only intuition
and anecdotes.
In basic research, many important variables are measured including attitudes, current emotional states and self-reports of
behaviors are most easily studied using questionnaires or interviews.
Survey method is an important way for researchers to study relationships among variables and ways that attitudes and
behaviors change over time.
Survey research complements experimental research findings.
Multiple methods are needed to understand any behavior.
A response set is a tendency to respond to all questions from a particular perspective rather then to provide answers that
are directly related to the questions.
Can affect usefulness of data.
Most common response set is called social desirability or “faking good”
most acute when questions concern sensitive topics such as violent/aggressive behavior.
If they trust the researcher, participants will provide honest answers.
Constructing Questions to Ask Defining the Research Objectives
What is it that the researcher wants to know? = research objectives
Survey questions must be tied to research questions.
Must decide what type of questions to ask: 1) Attitudes/Beliefs: the way people evaluate and think about issues
2) Facts/Demographics: things they know about themselves and their situation = necessary to describe sample = ex: age
and gender. Other factual info you might ask will depend on the topic of your survey.
3) Behaviors: past or intended future behaviors.
Question Wording
Cognitive psychologists have identified a number of potential problems with question wording (stem from a difficulty
with understanding the question): a) unfamiliar technical terms
b) vague or imprecise terms
c) ungrammatical sentence structure
d) phrasing that overloads working memory
e) embedding the question with misleading information
Simplicity
Avoid jargon and technical terms that people won’t understand.
Sometime you have to make question more complex to define a term or describe an issue.
Double-Barreled Questions
Avoid questions that ask two things at once.
Loaded Questions
A loaded question is written to lead people to respond in one way.
Questions that include emotionally charged words may influence the way that people respond and thus lead to biased
conclusions.
Negative Wording
Avoid phrasing questions with negatives.
Ex: Do you feel that the city should not approve the proposed women’s shelter?
“Yea-Saying” and “Nay-Saying”
When you ask several questions about a topic, there is a possibility that a respondent will employ a response set to agree
or disagree with all the questions = “yea-saying” or “nay-saying”.
Problem: Respondent may be agreeing with anything you say.
One way to detect this response set is to word the questions so that consistent agreement is unlikely.
Consistently agreeing or disagreeing with a set of related questions phrased in both standard and reversed formats is an
indicator that the individuals is “yea-saying” or “nay-saying”.
Graesser and his colleagues have developed a computer program called QUAID (Question Understanding Aid) that
analyzes question wording.
Responses to Questions Closed- Versus Open-Ended Questions
Close-ended questions, a limited number of response alternatives are given; with open-ended questions, respondents are
free to answer in any way they like.
Close ended = more structured= easier to code = response alternatives are same for everyone.
Open ended= need time to categorize and code= more costly = sometimes answers can’t even be categorized
However can still yield valuable insights into what people are thinking.
Open ended are most useful when researcher needs to know what people think and how they view the world. Close ended
are more useful when the dimensions of the variables are well defined.
The two approaches can lead to different conclusions -> therefore need to have a good understanding of topic when asked
closed-ended questions.
Number of Response Alternatives
In public opinion, “yes or no” or “agree or disagree” dichotomy is sufficient.
In more basic research, it is preferable to have a 5- or 7- point scale to allow people to sufficiently express themselves.
Rating Scales
Provide “how much” judgements on any number of dimensions, very commonly used in research.
Have many different formats, the format used depends on factors such as the topic being investigated.
Simplest, most direct scale = 5-7 point scale with endpoints on the scale defining the extremes.
Graphic Rating Scale
Requires a mark along a continuous 100-millimeter line that is anchored with descriptions at each end. A ruler is then
placed on the line to obtain the score on a scale that ranges from 0-100.
Semantic DIfferential Scale
A measure of the meaning of concepts that was developed by Osgood and associates.
Respondents rate ant concept-persons, objects, behaviors, ideas - on a series of bipolar adjectives using 7-point scales.
Ex: smoking cigarettes: good---- bad (evaluation), strong---weak (potency), active----passive (activity).
Research shows anything can be measured using this technique.
Concepts rated along 3 dimensions:
Evaluation (ex adjectives)
Activity
Potency
Nonverbal Scales for Children
young children can give ratings using smiley emoticons for example.
Labeling Response Alternatives
Sometimes researchers need to provide labels to more clearly define the meaning of each alternative.
Sometimes a perfectly balanced scale may not be possible or desirable.
High frequency scales and low frequency scales as indicated by response alternatives.
Schwartz points out that labels should be chosen carefully because people may interpret the meaning of the scale
differently, depending on the labels used.
Finalizing the Questionnaire Formatting the Questionnaire
Printed questionnaire = attractive and professional, neatly typed, no spelling errors, easy to identify questions and
responses.
If using a point scale, use it consistently throughout, don’t change it.
Consider sequence in which you will ask questions: more important/interesting questions should be at beginning to
capture attention, and similar themed questions should be grouped together.
Refining Questions
Before administering you should pilot questions by giving the questionnaire to a group of people and asking them to
“think aloud” while interpreting and responding to the questions.
Administering Surveys
Questionnaires
Written format
Advantages = less costly then interviews, anonymous.
Disadvantages= must be able to read/write, people find it boring.
Personal Administration to Groups or Individuals
Advantage: you have a captive audience that is likely to complete questionnaire once they start it and researcher is present
so people can ask questions if necessary.
Mail Surveys
Inexpensive way of contacting people who were selected for the sample.
Disadvantage: low response rates and researcher is not present for people to ask questions if they are confused.
Internet Surveys
Problem: how to sample people?
Internet is making it easier to obtain samples of people with particular characteristics.
One concern about INternet data collection is whether the results will be at all similar to what might be found using
traditional methods= research indicates that Internet results are in fact comparable.
Problem: ambiguity about characteristics of an individual providing info for the study.
Other Technologies
An interesting application is seen in studies aimed at sampling people’s behaviors and emotions over an extended period
of time. With cellphones and such you can contact person and ask them to report their current emotional
reactions/activities, etc. = “computerized experience sampling”.
Interviews
The fact that an interview involves an interaction between people has important implications: 1) People are likely to agree to answer questions for a real person than to answer a mailed questionnaire. = person is more
likely to answer all questions and complete survey.
Advantage: interviewer can clarify any problems/questions of interviewee, AND interviewer can ask interviewee for
clarification on their answers if needed.
Potential problem: interviewer bias =all the biases that can arise from the fact that the interviewer is a unique human
being interacting with another human.
Interviewer could subtly bias the respondent’s answers by inadvertently showing approval/disapproval of certain
answers.
If there are several interviewers, each could possess different characteristics (ex: level of attractiveness, age, race)
that might influence the way respondents answer.
Interviewers may have expectations that could lead them to “see what they are looking for”= could bias interviewers
interpretations of responses or probe certain people for answers. Solution: careful screening an training of
interviewers help to limit such biases.
Face-to-Face Interviews
Expensive and time consuming. Usually interviewer travels to persons home/office.
Most likely used when the sample size is fairly small and there are clear benefits to a face-to-face interaction.
Telephone Interviews
Almost all interview for large scale surveys are done via telephone.
Less expensive and allow data to be collected quicker because many interviewers can work on the same survey at once.
CATI system= computer-assisted telephone interview: questions are prompted on the computer screen and the data are
entered directly into the computer for analysis= lower cost by reducing labor and data analysis costs.
Focus Group Interviews
An interview with a group of about 6-10 individuals brought together for a period of usually 2-3 hours who have a
particular knowledge or interest in topic.
Usually some monetary or gift incentive.
Consists of open-ended questions.
Advantage: group interaction is possible.
Usually recored and may be transcribed = then analyzed to find themes and areas of group consensus/disagreement.
Prefer to conduct two/three discussion groups on a given topic to ensure info is not unique to one group = however it is
costly.
Survey Designs To Study Changes Over Time Surveys most frequently study people at one point in time, however researchers sometimes wish to make comparisons
over time = conduct same survey on multiple occasions to track changes in certain variables.
Another way to study changes over time is to conduct a panel study in which the same people are surveyed at two or
more points in time.Ex: “two-wave” panel study = people surveyed 2 times.
Panel studies are particularly important when the research question addresses the relationship between one variable at
“time one” and another variable at some later “time two”.
Sampling from a Population Most research projects involve sampling participants from a population of interest.
The population is composed of all individuals of interest to the researcher.
Although studying a whole population (if small) is possible, researchers usually opt to select a sample from the population
of interest.
Can use sample to predict characteristics of population as a whole.
Statistical theory allows us to infer what the population is like, based on data obtained from a sample.
Confidence Intervals
When researchers make inferences about populations, they do so with a certain degree of confidence.
Confidence interval: you can have 95% confidence that the true population value lies within this interval around the
obtained sample result. Best estimate of the population value is the sample value.
However because only using sample, there may be error = sampling error or margin of error (given by confidence
interval).
Sample Size
Larger sample size will reduce the size of the confidence interval. = increased accuracy
Most important factor in determing the size of the interval is sample size.
Larger samples are more likely to yield data that accurately reflects the true population value.
Sample size can be determined using a mathematical formula that takes into account the size of the confidence interval
and the size of the population you are studying.
Sample size is NOT a constant percentage of the population size.
Sample size needed does not change much even as the population increases (same degree of accuracy).
Sampling Techniques Two basic techniques for sampling individuals from a population:
1) Probability sampling: each member of the population has a specifiable probability of being chosen, this is very
important when you want to make precise statements about a specific population on the basis of the results of your
survey.
2) Non-probability sampling: we don’t know the probability of any particular member of the population being chosen.
Not as sophisticated, however common and useful in many circumstances.
Probability Sampling
Simple Random Sampling
Every member of the population has an equal probability of being selected for the sample.
Stratified Random Sampling
More complicated procedure: population is divided into subgroups (or strata) and random sampling techniques are then
used to select sample members from each stratum.
Dimensions chosen to divide population must be relevant to problem under study.
Advantage: built-in assurance that the sample will accurately reflect the numerical composition of the various subgroups.
Important when some subgroups represent a very small percentage of population 85
When important to represent a small group within a population, researchers will “oversample” that group to ensure that a
representative sample of the group is surveyed.
Cluster Sampling
Obtaining a list of all members of a population might be difficult, in that case we use cluster sampling: researcher
identifies individuals and then sample from these clusters. After clusters are chosen, all individuals in each cluster are
included in the sample.
Ex: classes = clusters, randomly sample from each class.
Most often, requires a series of samples from larger to smaller clusters - a “multistage” approach.
Main advantage: researcher does not have to sample from lists of individuals to obtain a truly random sample of
individuals.
Non-Probability Sampling
Quite arbitrary
Population may be defined but little effort is expended to ensure that sample accurately represents the population.
Cheap and convenient
Three types: 1) Haphazard Sampling
2) Purposive Sampling
3) Quota Sampling
Haphazard Sampling
“convenience” sampling
a “take them where you find them” method of obtaining participants
Likely to introduce biases into the sample= results do not accurately describe whole population.
Purposive Sampling
The purpose is to obtain a sample of people who meet some predetermined criterion.
Not a probability sample, limits sample to a certain group of people.
Quota Sampling
Researcher chooses a sample that reflects the numerical composition of various subgroups in the population. (Ex:19%
freshmen, 23% sophomores in whole population must be in sample as well).
Similar to stratified sampling however no random sampling occurs in quota sampling.
Problem remains: no restrictions are placed on how individuals in the various subgroups are chosen.
The sample does reflect the numerical composition of the whole population of interest but respondents within each
subgroup are selected in a haphazard manner.
Evaluating Samples A completely unbiased sample you need:
1) randomly sample from a population that contains all individuals in the population.
2) contact and obtain completed responses from all individuals selected to be in the sample.
However there can still be two sources:
the sampling frame used
poor response rates
Sampling Frame
The actual population of individuals (or clusters) from which a random sample will be drawn.
Ex: list of numbers to contact residents between 5PM and 9PM.
Rarely will this perfectly coincide with the population of interest- some biases will be introduced.
When evaluating the results of the survey, you need to consider how well the sampling frame matches the population of
interest. Biases are usually minor but could still be consequential.
Response Rate
The percentage of people in the sample who actually completed the survey.
Indicates how much bias there might be in the final sample.
The lower the response rate, the greater the likelihood that such biases may distort the findings and in turn limit the ability
to generalize the findings to the population of interest.
Many methods can be taken to maximize response rates.
Ex: researchers should attempt to convince people that the surveys purposes are important and their participation will be a
valuable contribution. Also incentives can be used.
Reasons for Using Convenience Samples Used a lot in research.
Cheap and not time consuming
Why aren’t researchers more worried about obtaining random samples from the general population for their research?
They are focused on studying the relationships between variables even though the sample may be biased (testing
hypotheses about behavior).
Research findings are still important even if they cannot be generalized.
Generalization in science is dependent upon replicating the results. the results of many studies (using multiple
samples/methods) can then be synthesized to gain greater insight into the findings.
Some non-probability samples are more representative than others.
Chapter 8 - Experimental Design
Confounding and Internal Validity A confounding variable is a variable that varies along with
the independent variable, confounding occurs when the
effects of the independent variable and an uncontrolled
variable are intertwined so you cannot determine which of the
variables is responsible for the observed effect.
When the results of an experiment can confidently be
attributed to the effect of the independent variable, the
experiment is said to have internal validity.
All other variables in an experiment should be kept constant
through direct experimental control or through
randomization. Then only can you conclude that the
independent variable is the cause of the different results.
Basic Experiments Simplest experimental design: two variables : IV and DV. The IV has 2 levels: experimental (receives treatment) and the
control group.
The basic, simple experimental design can take one of two forms: a posttest only or a pretest-posttest design.
Posttest only Design
Researcher must 1) obtain two equivalent groups of participants, 2) introduce the IV and 3) measure the effect of the IV
on the DV.
Procedures used must achieve equivalent groups to eliminate any potential selection differences: the people selected to
be in the conditions cannot differ in any systematic way.
Solution: randomly assign or have same participants participate in both conditions.
IV is measured using the same procedure for both groups, so that you can compare the two groups.
A statistical significance test would be used to assess the difference between the groups. An experiment must be well
designed and confounding variables must be eliminated before we can draw conclusions from statistical analyses.
Pretest-Posttest Design
The only difference between the posttest design and the pretest-posttest design is that in the latter a pretest is given
before the experimental manipulation is introduced. Makes it possible to ascertain that the groups were in fact equivalent
at the beginning of the experiment.
The larger the sample, the less likelihood there is that the groups will differ in any systematic way prior to the
manipulation of the IV. There is also an increasing likelihood that any difference between the groups on the DV is due to
the effect of the IV.
Advantages and Disadvantages of the Two Designs
With small sample sizes, a pretest is efficient to assess if they equivalent.
Sometimes a pretest is necessary to select the participants in the experiment.
Can use pretests to measure the extent of change in each individual.
A pretest is also necessary whenever there is a possibility that participants will drop out of the experiment (in studies that
take longer periods of time).
The dropout factor in experiments is called mortality.
Groups may become not equivalent due to different mortality rates. Mortality can become an alternative explanation for
the results.
Use of a pretest enables you to assess the effects of mortality; you can look at the pretest scores of the dropouts and
know whether mortality affected the final results.
However one disadvantage of a pretest is that it is time consuming and awkward to administer in the context of the
particular experimental procedures being used.
It can also sensitize participants, and they may figure out your hypothesis = may react differently.
Thus IV may not have an affect in the real world where pretests are not given.
Solution: disguise pretest.
It is possible to assess the impact of the impact of the pretest directly with a combination of both designs. In this design
half the participants receive only the posttest, and the other half receive both= Solomon four-group design.
IV
Pretest Condition Control Group Experimental Group
No pretest (posttest only)
Pretest and posttest
Assigning Participants to Experimental Conditions Independent groups design: participants are randomly assigned to the various conditions so that each participates in only
one group.
Repeated measures design: participants are in all conditions, each participant is assigned to both levels of the IV.
Independent Groups Design Different participants assigned to each of the conditions using random assignment = prevent systematic biases, and groups
will be equivalent in terms of participant characteristics such as income, intelligence, age, or political attitudes.
Researchers usually use a sequence of random numbers to determine assignment.
Repeated Measures Design Participants are repeatedly measured on the DV after being in each condition of the experiment.
Advantages and Disadvantages of Repeated Measures Design
Advantage: fewer research participants are needed.
Saves money as well.
Advantage: extremely sensitive to finding statistically significant differences between groups.
More likely to detect an effect of the IV on the DV, because its easier to separate the systematic individual
differences from the effect of the IV.
Ex: recall test.
Disadvantage: order effect - the order of presenting the treatments affects the DV.
There are several types of order effects: Order effects that are associated simply with the passage of time include practice
effects and fatigue effects. 1) Practice effect: an improvement in performance as a result of repeated practice with a task.
2) Fatigue effect: deterioration in performance as the research participant becomes tired, bored or distracted.
Other types of order effects occur when the effect of the first treatment carries over to influence the response to the second
treatment.
3) Contrast effect: when the response to the 2nd condition in the experiment is altered because the 2 conditions are
contrasted to one another.
Two approaches to deal with such problems:
Employ counterbalancing techniques.
Devise a procedure in which the interval between conditions is long enough to minimize the influence of the 1st
condition on the 2nd.
Counterbalancing
Complete Counterbalancing
In repeated measures design it is very important to counterbalance the order of the conditions. With complete
counterbalancing, all possible orders of presentation are included in the experiment.
By counterbalancing the order of conditions it is possible to determine the extent to which order is influencing the results.
Latin Squares
A technique to control for order effects without having all possible orders is to use a Latin square: a limited set of orders
constructed to ensure that 1) each condition appears at each ordinal position and 2) each condition precedes and follows
each condition one time.
The number of orders in a latin square is equal to the number of conditions.
Need 1 participant per row. The number of participants run in each order must be equal.
In an experiment in which individuals are tested over a series of trials (repeated measures variable), however in this case
counterbalancing is not needed because the order effect of changes in performance over trials is of interest to the
researcher.
Time Interval Between Treatments
Researchers need to carefully determine the time interval between presentation of treatments and possible activities
between them.
A rest period may counteract a fatigue effect.
Attending to an unrelated task between treatments may reduce the possibility of a contrast effect.
However the introduction of an extended time interval between treatments may lead to fewer people wanting to participate
and some may dropout.
Choosing Between Independent Groups and Repeated Measures Designs
Repeated measures design have two advantages over independent groups designs: 1) a reduction in the number of
participants required to complete the experiment and 2) greater control over participant differences and thus greater ability
to detect an effect if the IV.
A very different consideration in whether to use a repeated measures design concerns generalization to conditions in the
“real world”.
Any experimental procedure that produces a relatively permanent change in an individual cannot be used in a repeated
measures design. (Ex: psychotherapy treatment or surgery).
Matched Pairs Design Instead of simply randomly assigning participants to groups, the goal is to first match people on a participant
characteristic.
The matching variable will either be the dependent measure or a variable that is strongly related to the DV.
Goal is to achieve same equivalency of groups in a repeated measures design without the necessity of having the same
participants in both conditions.
Members of each pair are randomly assigned to the conditions in the experiment.
Groups are equivalent on the matching variable. Important for small sample sizes.
Therefore used when few participants are available or if its costly to run large numbers of participants.
Results in a greater ability to detect a statistically significant effect of the IV because it is possible to account for
individual differences in responses to the IV (just as with repeated measures design).
Matching procedures can be costly and time consuming (must measure participants on matching variable prior to
conducting study) and its only worthwhile when matching variable is strongly related to dependent measure.
Not commonly used.
Chapter 9- Conducting Studies
Selecting Research Participants The method used to select participants has implications for generalizing research results.
When it is important to accurately describe population, you must use probability sampling (crucial when conducting
scientific polls).
Ample evidence supports the view that we can generalize findings from haphazard samples to other populations and
situations.
Manipulating the Independent Variable To manipulate an IV you have to construct an operational definition of the variable and then turn a conceptual variable
into a set of operations: specific instructions, events, and stimuli to be presented.
In addition, the IV and the DV must be introduced within the context of the total experimental setting = setting the stage.
Setting the Stage
Provide participants with informed consent and explain to participants why the experiment is being conducted.
There are no clear cut rules for setting the stage except that the experiment must seem plausible to the participants nor are
there any clear cur rules for translating conceptual variables into specific operations.
How the variable is manipulated depends on the variable and the cost, practicality and ethics of the procedures being
considered.
Types of Manipulations
Straightforward Manipulations
Presenting written, verbal or visual material to participants; manipulate using instructions and stimulus presentations.
Stimuli may be presented verbally, written, via videotape or with a computer.
Studies on jury decisions often ask participants to read a description of a jury trial in which a crucial piece of information
is varied (ex: severity of injury).
Most memory research relies on straightforward manipulations.
Most manipulations of IV’s in all areas of research are straightforward. Researchers vary the difficulty of material to be
learned, motivation levels, the way questions are asked, characteristics of people to be judged and a variety of other
factors in a straightforward manner.
Staged Manipulations
Less straightforward. Sometimes it is necessary to stage events that occur during the experiment in order to manipulate the
IV successfully = staged or event manipulation.
Used frequently for two reasons:
Researcher may be trying to create come psychological state in the participants (ex: anger)
A staged manipulation may be necessary to simulate some situation that occurs in the real world.
Staged manipulations frequently employ a confederate (an accomplice) = usually appears to be another participant.
Staged manipulations demand a great deal of ingenuity and even some acting ability. Used to engage participants in an
ongoing social situation.
Such procedures allow for a great deal of subtle interpersonal communication that is hard to put into words; this may
make it difficult to interpret. (If many things happened during experiment, what one thing was responsible for the results?)
In general easier to interpret results when straightforward manipulations are used.
Strength of Manipulation
A general principle is to make manipulation as strong as possible = maximizes differences between two groups and
increases the chances that the IV will have a statistically significant effect on the DV.
Strong manipulation is most important when trying to demonstrate that a relationships exists between two variables.
The principle of using the strongest manipulation is tempered at two considerations:
strongest possible manipulation may involve a situation that rarely if ever occurs in the real world.
Ethics: a manipulation should be as strong as possible within the bounds of ethics.
Cost of Manipulation
Cost is another factor in the decision about how to manipulate the IV. Limited monetary resources.
Straightforward manipulations are less costly then staged.
Measuring the Dependent Variable Types of Measures
The DV in most experiments is one of three general types: self-report, behavioral, or physiological.
Self-Report Measures
Measure attitudes, liking for someone, judgements about someones personality characteristics, intended behaviors
emotional states, attributions about why someone performed well or poorly on a task, confidence in one’s judgements and
many other aspects of human thought and behavior.
Rating scales with descriptive anchors are most commonly used.
Behavioral Measures
Direct observations of behaviors.
Measurements of an almost endless number of behaviors is possible (just as with self-reports).
Often the researcher must decide whether to record the number of times a behavior occurs in a period of time = rate, how
quickly a response occurs after a stimulus = reaction time or how long a behavior lasts = duration. Or the
presence/absence of the behavior.
When both self report and behavioral measures could be used, a series of studies may be conducted to study the effects of
an IV on both types of measures.
Physiological Measures
Recordings of responses of the body. Physiological indicators of psychological variables.
Many such responses are available: galvanic skin response (GSR), electromyogram (EMG) and electroencephalogram
(EEG).
The GSR is a measure of general emotional arousal and anxiety, it measures the electrical conductance of the skin which
changes when sweating occurs.
The EMG measures muscle tension and is frequently used as a measure of tension or stress.
The EEG is a measure of electrical activity of brain cells. It can be used to record general brain arousal as a response to
different situations, activity in different parts of the brain as learning occurs, or brain activity during different stages of
sleep.
MRI’s (magnetic resonance imaging) provide an image of the brain structure of an individual
Functional MRI = allows researchers to scan areas of the brain while a research participant performs a physical or
cognitive task.
Provides evidence for what brain processes are involved in these tasks.
Other physiological indicators include: heart rate, temperature and blood/urine analysis.
Sensitivity of the Dependent Variable
The DV should be sensitive enough to detect differences between groups.
Issue of sensitivity is particularly important when measuring human performance.
When a task is too easy, everyone does well = ceiling effect - the IV appears to have no effect on the dependent measure
only because participants quickly reach the maximum performance level.
Opposite problem occurs when a task is too difficult that hardly anyone can perform well = floor effect.
Multiple Measures
Often desirable to measure more than one DV, because a variable can be measured in a variety of concrete ways
(operational definitions).
It is useful to know whether the same IV affects some measures but not others.
Researchers may also be interested in studying the effects of an IV on several different behaviors.
If there is a problem of order (regarding the measures) you could counterbalance. If there is no order problem, most
important measures go first.
Making multiple measurements in a single experiment is only valuable when it is feasible to do so. However may be
necessary to conduct a series of experiments to explore the effects of an IV on various behaviors.
Additional Controls Controlling for Participant Expectations
Demand Characteristics
Demand characteristics: any feature of an experiment that might inform participants of the purpose of the study.
When participants form expectations about hypothesis of study, they will do whatever to confirm it (if cooperative).
To control, you could use deception. Attempt to disguise the DV by using an unobtrusive measure placing it among a set
of unrelated filler items on a questionnaire.
Experiments conducted in field settings and observational research in which observer is concealed or unobtrusive
measures are used minimize demand characteristics.
Placebo Groups
A special kind of participant expectation aries in research on the effects of drugs.
We do not know whether an improvement in an experimental group was caused by the properties of the drug or by the
expectations about the effects of the drug = placebo effect.
To control for this possibility a placebo group can be added = they receive a pill that is not actually the drug but an inert
substance.
If experimental group shows greater improvement than placebo group we know its because of the drug itself.
Balanced placebo design: alcohol example.
Ethical implications: if drug has positive effects, must be given after study to those who were in the control groups.
Great deal of research and debate on the extent to which any beneficial effects of antidepressants such as Prozac are due to
placebo effects.
Controlling for Experimenter Expectations
Experimenter expectations can in turn bias the results = experimenter bias or expectancy effects.
Can occur when experimenter knows which condition participants are in. Two sources of bias:
Might unintentionally treat participants differently based on condition they are in.
Experimenter record the behaviors of the participants, there may be subtle differences in the way the experimenter
interprets and records behaviors.
Research on Expectancy Effects
Experimenter expectancies can be communicated to humans by both verbal and non verbal means.
Teacher expectancy = teacher’s expectations can influence students performance.
Expectations influence ratings of behavior.
Solutions to the Expectancy Problem
Experimenters should be well trained and should practice behaving consistently with all participants.
Run all conditions simultaneously so that experimenters behavior is the same for all participants.
Expectancy effects are also minimized when procedures are automated. (record responses using computers for example)>
Use experimenters who are unaware of the hypothesis. Person is blind to what is being studied or which condition the
participant is in.
In a single-blind experiment, the participant is unaware of whether a placebo or the actual drug is being administered.
In a double-blind experiment neither the participant or the experimenter knows whether the placebo or actual treatment is
being given.
Additional Considerations Research Proposals
Researcher write a research proposal = includes a literature review providing background of study.
Procedure, analysis procedures to be done, what and why the research is being done.
Proposals must be included in applications for research grants; ethics review committees require some type of proposal as
well.
Pilot Studies
Pilot study: researcher does a “trial run” with a small number of participants= reveals if instructions are understood, if
experimental setting is plausible, etc. Ask participants about experience of the experiment.
Another method: “think aloud”.
Manipulation Checks
A manipulation check is an attempt to directly measure whether the IV manipulation has the intended effect on
participants.
Provides evidence for the construct validity of manipulation.
Ask whether the IV manipulation was a successful operationalization of the conceptual variable being studied.
Should be done near the end of the experiment before debriefing = so no distraction during actual experiment.
Two advantages:
If manipulation is not working, saved expense of running experiment
Can identify whether nonsignificant results are due to a problem in manipulating the IV.
Debriefing
Discuss ethical and educational implications of study.
Learn what participants were thinking during experiment = useful i interpreting results and planning future studies.
Ask participants to refrain from discussing study with others.
Analyzing and Interpreting Results Statistical analyses of the data are carried out to allow the researcher to examine and interpret the pattern of results
obtained in the study; helps researcher decide if there is a relationship between the IV and the DV.
Communicating Research to Others Final step is to write a report that details why you conducted the research, how you obtained the participants, what
procedures you used, and what you found.
Research findings are then usually submitted as journal articles/papers to be read at scientific meetings = evaluated by two
or more knowledgable reviewers who decide whether paper is acceptable for publication/presentation or not (peer review).
Professional Meetings
Meetings sponsored by professional associations are important opportunities for researchers to present their findings to
others and learn about others research findings.
Verbal presentation of poster sessions.
Journal Articles
Almost 90% of articles submitted get rejected, very hard to publish.
Chapter 10- Complex Experimental Designs
Increasing the Number of Levels of an Independent Variable The simple design allows us to examine important parts of research such as internal validity.
Researcher might want to design an experiment with three or more levels of IV for several reasons.
First a design with only two levels of IV cannot provide very much info about the exact form of the relationship between
the independent and dependent variables.
If a curvilinear relationship is predicted, at least three levels must be used in order for the relationship to be detected.
Example: relationship between fear arousal and attitude change.
Finally, researcher frequently are interested in comparing more than two groups.
Increasing the Number of Independent Variables: Factorial Designs Researchers often manipulate more than one IV in a single experiment, typically 2 or 3 IVs are operating simultaneously
= closer approximation of real world conditions.
Factorial designs are designs with more than one IV (or factor).
All levels of each IV are combined with all levels of the other IVs.
Ex: 2 x 2 ; two IVs each having two levels. Has four groups
General format for describing factorial designs is: number of levels of 1st IV x Number of levels of 2nd IV x Number of levels of 3rd IV and so on.
Interpretation of Factorial Designs
Yields two kinds of information:
Info about the effect of each IV taken by itself: the main effect of an IV. (2 IVS= 2 main effects).
Second type of info is interaction, is there is an interaction between two IVs, the effect of one IV depends on the
particular level of the other variable. In other words, the effect that an IV has on the DV depends on the level of the
other IV.
Main Effects
A main effect is the effect each variable has by itself.
Main effect of each IV is the overall relationship between the IV and the DV.
Overall main effect means are obtained by averaging across all participants in each group.
Interactions
Interactions can be easily seen when the means for all conditions are presented in a graph. DV is always on the vertical
axis, one IV is placed on the horizontal axis, bars are then drawn to represent each of the levels of the other IV = useful
method of visualizing interactions in a factorial design.
Interaction between IVs indicates that the effect of one IV is different at different levels of the other IV. (Main effects of
the IVs must be qualified).
Factorial Designs with Manipulated and Non-manipulated Variables
One common type of factorial design includes both experimental (manipulated) and non experimental (measured or non-
manipulated) variables.
IV x PV designs (ex: IV by participant variable); allow researchers to investigate how different types of individuals
respond to the same manipulated variable.
“Participant variables” (aka subject or attribute variables) are personal attributes such as age, gender, ethnic group,
personality characteristics and clinical diagnostic category.
The simplest IV x PV designs include one manipulated independent variable that has at least two levels and one
participant variable with at least two levels.
Such experiments recognize that full understanding of behavior requires knowledge of both situational variables and the
personal attributes of individuals.
Interactions and Moderator Variables
In many research studies, interactions are discussed in terms of the operation of a moderator variable: influences the
relationship between two other variables.
Moderator variables may be particular situation or they may be characteristics of people.
Outcomes of a 2 x 2 Factorial Design
When analyzing the results there are several possibilities: 1) There may or may not be a significant main effect for IV A
2) There may or may not be a significant main effect of IV B
3) There may or may not be a significant interaction between the IVs.
Line graphs are usually used when the levels of the IV on the horizontal axis (IV A) are quantitative- low and high
amounts.
Bar graphs are more likely to be used when the levels of the IV represent different categories (ex: different types of
therapies).
Interactions and Simple Main Effects
A procedure called analysis of variance is used to assess the statistical significance of the main effects and the interaction
in a factorial design.
When there is a significant interaction, there is a need to statistically evaluate the individual means.
When there is a significant interaction, the next step is to look at the simple main effects. A simple main effect analysis
examines mean differences at each level of the IV.
Recall that the main effect of an IV averages across the levels of the other IVs; with simple main effect, the results are
analyzed as if we had separate experiments at each level of the other IV.
The simple main effect that you will be most interested in will depend on the predictions that you made when you
designed the study.
Assignment Procedures and Factorial Designs
The design can be completely independent groups, completely repeated measures or a mixed factorial design that is a
combination of the two.
Have different implications for the number of participants needed to complete the experiment.
Mixed Factorial Design Using Combined Assignment
Ex: IV A is an independent groups variable. 10 participants are assigned to level 1 of this IV and another 10 are assigned
to level 2. IV B is a repeated measures group, however the 10 participants assigned to A1 receive both levels of IV B.
Similarly the 10 assigned to A2 receive both levels of IV B.
Increasing the Number of Levels of an Independent Variable
The 2 x 2 is the simplest factorial design.
One way to increase complexity is to increase the number of levels of one or more of the IVs.
Increasing the Number of Independent Variables in a Factorial Design
A 2 x 2 x 2 contains three variables each with two levels.
Yields many effects for each of the IVs.
We can look at a three-way interaction that involves all three IVs, here we want to determine whether the nature of the
interaction between two of the variables differs depending on the particular level of the other variable.
When there are more than 3 or 4 IVs, many of the particular conditions that are produced by the combination of so many
variables do not make sense or could not occur under natural circumstances.