Lecture Research Design
-
Upload
zunaira-azhar -
Category
Documents
-
view
222 -
download
0
Transcript of Lecture Research Design
-
7/27/2019 Lecture Research Design
1/38
LECTURE
1
ELEMENTS OF
RESEARCH DESIGN
-
7/27/2019 Lecture Research Design
2/38
THE RESEARCH PROCESS
2
OBSRVATION
Board area
of research
interest
identified
1
PRELIMINARY
DATA GATHERING
Interviewing
Literature survey
2
PROBLEM
DEFINITION
Research
problem
delineated
3
THEORETICAL
FRAMEWORK
Variables clearly
identified and
labeled
4
GENERATION
OF
HYPOTHESES
5
SCIENTIFIC
RESEARCH
DESIGN
6
DATA COLLECTION
ANALYSIS AND
INTERPRETATION
7
DEDUCTION
Hypotheses
substantiated?
Research question
answered?
8
Report
Writing
Report
Presentation
Managerial
Decision
Making
No Yes
9 10 11
-
7/27/2019 Lecture Research Design
3/38
THE RESEARCH DESIGN
A research design is a plan, structure andstrategy of investigation so conceived asto obtain answers to research questions or
problems. The plan is the complete schemeor program of the research. It includes anoutline of what the investigator will dofrom writing the hypothesis and theiroperational implication to the finalanalysis of data.
3
-
7/27/2019 Lecture Research Design
4/38
4
THE RESEARCH DESIGN
Purpose of the
study
Exploration
Description
Hypothesis testing
Types of
Investigation
Establishing:
-Casual relationships
-Correlations
-Group differences,
Extent of researcher
Interference
Minimum: Studying events
as they normally occur
Moderate: Minimum
amount of interference
Maximum: High degree
of control and artificial
settings
Study setting
Contrived
Noncontrived
Measurement
and measures
Operational
definition
items (measure)
Scaling
Categorizing
Coding
Unit of analysis
(Population to
be studied)
Individuals
Dyads
Groups
Organizations
Machines
etc.
Sampling
design
Probability/nonprobability
Sample
Size (n)
Time
horizon
One-Shot
(cross-sectional)
Multishot
(longitudinal)
Data-Collection
method
ObservationInterview
Questionnaire
Physical
measurement
Unobtrusive
1. Feel for data
2. Goodness or
data
3. Hypotheses
testing
PROBLEMS
TATEME
NT
DATA
ANALYSIS
DETAILS OF STUDY MEASURMENT
-
7/27/2019 Lecture Research Design
5/38
5
An exploratory study is undertaken when notmuch is known about the situation in hand, or
no information is available on how similarproblems or research issues have been solved inthe past.
Exploratory studies are also necessary whensome facts are known, but more informationis needed for developing a viable theoreticalframework.
EXPLORATORY STUDY
PURPOSE OF THE STUDY
-
7/27/2019 Lecture Research Design
6/38
6
A descriptive study is undertaken in order to ascertain and be able
to describe the characteristics of the variables of interests in asolution. For instance, a study of a class in terms of thepercentage of members who are in their senior and junioryears, sex composition, age groupings, number of semestersleft until graduation, and number of business courses taken,can be considered as descriptive in nature.
DESCRIPTIVE STUDY
Example
A bank manager wants to have a profile of the individuals who haveloan payments outstanding for 6 months and more. It would include
details of their average age, earnings, nature of occupation, full-time/part-time employment status, and the like. This might help him toelicit further information or decide right away on the types ofindividuals who should be made ineligible for loans in the future.
-
7/27/2019 Lecture Research Design
7/38
7
Studies that engage in hypotheses testing usuallyexplain the nature of certain relationships, orestablish the differences among groups or theindependence of two or more factors in a solution.
HYPOTHESES STUDY
Example
A marketing manager wants to know, the sales of thecompany will increase, if he doubles the advertising
dollars. Here, the manager would like to know thenature of the relationship that can be established
between advertising and sales by testing the hypothesis:If advertising is increased, then sales will also go up.
-
7/27/2019 Lecture Research Design
8/38
8
TYPES OF INVESTIGATION
CAUSAL VERSUS CORRELATIONAL
Causal study: The study in which the researcher wants todelineate the cause of one or more problems is called a causal
study.
Correlational study: When the researcher is interested in
delineating the important variables associated with the problem,the study is called a correlational study.
Example
A causal study question:
Does smoking cause cancer?
A correlational study question:
Are smoking and cancer related?
OR
Are smoking, drinking, and chewing tobacco associated with cancer? If so, which ofthese contributes most to the variance in the dependent variable?
-
7/27/2019 Lecture Research Design
9/38
EXTENT OF RESEARCHER INTERFERENCE
9
The extent of interference by the researcherwith the normal flow of work at the
workplace has a direct bearing on whether
the study undertaken is causal orcorrelational. A correlational study is
conducted in the natural environment of
the organization with minimuminterference by the researcher with the
normal flow of work.
-
7/27/2019 Lecture Research Design
10/38
STUDY SETTING: CONTRIVED AND NONCONTRIVED
Noncontrived settings: If organizational researchbe done in the natural environment where work
proceeds normally, the research is in noncontrived
settings.
contrived settings: If organizational research be
done in artificial environment the research is in
contrived settings.
Correlational studies are invariably conducted
in noncontrived settings, whereas most rigorous
causal are done in contrived lab settings.
10
-
7/27/2019 Lecture Research Design
11/38
CONTRIVED AND NONCONTRIVED SETTINGS
1. FIELD STUDY: If various factors are examined in the naturalsettings in which daily activities going on as normal withminimal researcher interference, the study is field study(noncontrived).
2. FIELD EXPERIMENT: If cause and effect relationships arestudied with some amount of researcher interference, butstill in the natural settings where work continues in thenormal environment, the study is field experiment(contrived).
3. LAB EXPERIMENT: If the researcher explores cause andeffect relationship not only exercising a high degree ofcontrol but in an artificial and deliberately created settings(contrived).
11
-
7/27/2019 Lecture Research Design
12/38
12
A bank manager wants to analyze the relationshipbetween interest rates and bank deposit patterns of
clients. She tries to correlate the two by looking at
deposits into different kinds of accounts (such as
savings, certificates of deposit, and interest-bearing
checking accounts) as interest rates changed.
This is a field study where the bank manager has merely
taken the balances in various types of accounts andcorrelated them to the changes in interest rates.
Research here is done in a noncontrived setting with no
interference with the normal work routine.
EXAMPLE OF FIELD STUDY
-
7/27/2019 Lecture Research Design
13/38
13
The bank manager now wants to determine the cause-and-effect relationship betweeninterest rate and the inducements it offers to clients to save and deposit money in the
bank. She select branches within a 60-mile radius for the experiment. For 1 week only,
she advertise the annual rate for new certificates of deposit received during that week in
the following manner: the interest rate would be 9% in one branch, 8% in another, and
10% in the third. In the fourth branch, the interest rate remains unchanged at 5%. Within
the week, she would be able to determine the effects, if any, of interest rates on deposit
mobilization.
The above would be a field experiment since nothing but the interest rate in
manipulated, with all activities occurring in the normal and natural work environment.
Hopefully, all four branches chosen would be more or less compatible in size, number of
depositors, deposit patterns, and the like, so that the interest savings relationships are not
influenced by some third factors. But it is possible that some other factors might affect
the findings. For example, one of the areas may have more retirees who many not have
additional disposable income that they could deposit, despite the attraction of a good
interest rate. The banker may not have been aware of this fact while setting up the
experiment.
EXAMPLE OF FIELD EXPERIMENT
-
7/27/2019 Lecture Research Design
14/38
14
EXAMPLE OF LAB EXPERIMENT
The bank manager now wants to establish the causal connection between interest rates
and saving, beyond a doubt. Because of this she wants to create an artificial
environment and trace the true cause and effect relationship. She recruit 40 students who
are all business majors in their final year of study and are more or less of the same age.
She splits them into four groups and gives each one of them amount of $1,000, which
they are told they might utilize to buy their needs or save for the future, or both. She
offers them an incentive, interest on what they save but manipulates the interest rates by
offering a 6% interest rate on savings for group 1, 8% for group 2, 9% for group 3, andkeeps the interest at the lowest rate of 1% for group 4.
Here the manager has created an artificial laboratory environment and has manipulated
the interest rates for savings. She has also chosen subjects with similar backgrounds and
exposure to financial matters (business students). If the banker finds that the savings by
the four groups increase progressively, keeping in step with the increasing rates of
interest, she would be able to established a cause and effect relationship between interestand the disposition to save.
In this lab experiment with the contrived settings, the researcher interference has been
maximal, inasmuch as the setting is difficult, the independent variable has been
manipulated, and most external contaminating factors such as age and experience have
been controlled.
-
7/27/2019 Lecture Research Design
15/38
Decision points for embarking on an experimental design
15
Is tracing causal
effects necessary?
Yes
and ifNo
Internal validity is
more important than
external validity
Generalizability is
more important than
internal validity.
Both internal validity and
external validity are
important.
Engage in a lab
experiment.
Engage in a field
experiment.
First do a Lab experiment,
then, a FIELD experiment.
Are there cost
constraints?
NoYes
Engage in a simpler
experimental design.
Engage in a more
sophisticated design.
Do not undertake an
experimental design study
-
7/27/2019 Lecture Research Design
16/38
UNITS OF ANALYSIS
The unit of analysis refers to the level of aggregation (bunch) of the data collected during thesubsequent data analysis stage.
If the problem statement focuses on how to rates levels of employees in general, then we areinterested in individuals employees in the organization and would have to find out what we can doto raise their motivation. Here the unit of analysis is the individual.
If the researcher is interested in studying two-person interactions, then several two-person groups,also known as dyads.
If the problem statement is related to group effectiveness, then the unit of analysis would be at thegroup level.
If we compare different departments in the organization, then the data analysis will be done at thedepartmental level.
If we compare different organizations, then the data analysis will be done at the organizationallevel.
If we compare the different cities of any country, then the data analysis will be at the city level.
If we compare the different countries, then the data analysis will be at the country level. etc.etc.
16
-
7/27/2019 Lecture Research Design
17/38
TIME HORIZON
One Shot or Cross-Sectional StudiesIf data are gathered just once, perhaps over a period of days or weeks or
months, in order to answer a research question. are called one-shot or cross-
sectional studies.
17
EXAMPLES1. Data were collected from stock brokers between April and June of last year
to study their concerns in a turbulent (beyond control) stock market. Data with
respect to this particular research had not been collected before, nor will they
be collected again from them for this research.
2. A drug company desirous of investing in research for a new obesity
(reduction) pill conducted a survey among obese people to see how many of
them would be interested in trying the new pill. This is a one-shot or cross-
sectional study to assess the likely demand for the new product.
-
7/27/2019 Lecture Research Design
18/38
Multishot or Longitudinal Studies
If the researcher might want to study people or phenomena atmore than one point in time in order to answer the researchquestion or when data on the dependent variable are gathered
at two or more points in time to answer the research question,the studies are called longitudinal studies.
For instance, the researcher might want to study employeesbehavior before and after a change in the top management, so as
to know what effects the change accomplished. Here, becausedata are gathered at two different points in time, the study is notcross-sectional or of the one-shot kind, but is carriedlongitudinally across a period of time.
18
EXAMPLEOne could study the sales volume of a product before and after an advertisement, and
provided other environmental changes have not impacted on the results, one could
attribute the increase in the sales volume, if any, to the advertisement. If there is no
increase in sales, one could conclude that either the advertisement is ineffective or it
will take a longer time to take effect.
-
7/27/2019 Lecture Research Design
19/38
EXERCISE
In the following scenarios indicate how the researcher should proceed
in each case, that is, determine the following, give reason also:
1. The purpose of study,
2. The type of investigation,
3. The extent of researcher interference,
4. The study settings,5. The time horizon for the study,
6. The unit of analysis.
Scenario A
Ms. Joyce Lynn, the owner of small business (a womens dress
boutique), has invited a consultant to tell her how business is differentfrom similar small businesses within a 60-mile radius with respect touse of the most modern computer technology, sales volume, profitmargin, and staff training.
19
-
7/27/2019 Lecture Research Design
20/38
EXERCISE
Scenario BMr. pall Hodge, the owner of severalrestaurants on the East Coast, is concernedabout the wide differences in their profit
margins. He would like to try some incentiveplans for increasing the efficiency levels ofthose restaurants that lag behind. But beforehe actually does this, he would like to beassured that the idea would work. He asks aresearcher to help him on this issue.
20
-
7/27/2019 Lecture Research Design
21/38
21
EXPERIMENTALDESIGN
-
7/27/2019 Lecture Research Design
22/38
CONTROLLING OF CONTAMINATING FACTORS
When we postulate cause-and-effectrelationships between two variables X andY, it is possible that some other factor, saysA, might also influence the dependentvariable Y. In such a case, it will not bepossible to determine the extent to which Yoccurred only because of X, since we do
not know how much of the total variationof Y was caused by the presence of theother factor A.
22
-
7/27/2019 Lecture Research Design
23/38
EXAMPLE OF CONTROL
For instance, a Human Resource Development manager mightarrange for special training to a set of newly recruited secretariesin creating web pages, However, some of the new secretariesmight function more effectively than others, mainly or partlybecause they have had previous intermittent experience with
the web. In this case, the manager cannot prove that the specialtraining alone caused greater effectiveness, since the previousintermittent experience of some secretaries with the web is acontaminating factor. If the true effect of the training on learningis to be assessed, then the learners previous experience has tobe controlled. This might be done by not including in the
experiment those who already have had some experience withthe web. This is what we mean when we say we have to controlthe contaminating factors.
23
-
7/27/2019 Lecture Research Design
24/38
CONTROLLING THE CONTAMINATING EXOGENOUS OR
NUISANCE VARIABLES
Matching Groups
One way of controlling the contaminating ornuisance variablesis to match the various groups by picking the confoundingcharacteristics and deliberately spreading them across groups.
24
Randomization
In randomization, the process by which individuals are drawn(i.e., everybody has a known and equal chance of being drawn)
and their assignment to any particular group (each individualcould be assigned to any one of the groups set up ) are bothrandom.
-
7/27/2019 Lecture Research Design
25/38
EXTERNAL VALIDITY
25
To what extent would the result found in the lab
setting be transferable or generalizable to the actualorganizational or field settings? In other words, if we
do find a cause-and-effect relationship after
conducting a lab experiment, can we then
confidently say that the same cause-and-effect
relationship will also hold true in the organizational
setting?
Internal validity refers to the confidence we place in the cause-and-effect relationship with in the lab settings.
INTERNAL VALIDITY
-
7/27/2019 Lecture Research Design
26/38
FACTORS AFFECTING INTERNAL VALIDITY
26
Sales promotion Sales
Dairy
farmers advertisement
Independent variable Dependent variable
Uncontrolled variable
Time: t1 t2 t3
History Ef fects
Certain events or factors that would have an impact on the independent variable-dependent variable relationship might unexpectedly occur while the experimentis in progress, and this history of events would confound the cause-and-effectrelationship between the two variables, thus affecting the internal validity.
-
7/27/2019 Lecture Research Design
27/38
-
7/27/2019 Lecture Research Design
28/38
28
Testing Effects
Frequently, to test the effect of a treatment, subjects are givenwhat is called a pretest(say, a short questionnaire eliciting their
feelings and attitudes). That is, first a measure of the dependentvariable is taken (the pretest), then the treatment given, and afterthat a second test, called the posttest, administered. Thedifference between the posttest and the pretest scores is thenattributed to the treatment. However, the very fact that
respondents were exposed to the pretest might influence theirresponses on the posttest, which would adversely impact oninternal validity.
I nstrumentations Effects
Instrumentation effects are yet another source of threat tointernal validity. These might arise because of a change in themeasuring instrument between pretest, and posttest, and not
because of the treatments differential impact at the end.
-
7/27/2019 Lecture Research Design
29/38
29
Selection Bias Effects
The threat to internal validity could also
come from improper or unmatchedselection of subjects for the experimentaland control groups.
Mortality
Another confounding factor on the cause-and-effect relationship is the mortality or
attrition of the members in theexperimental or control group or both, asthe experiment progresses.
-
7/27/2019 Lecture Research Design
30/38
30
Statistical Regression
The effect of statistical regression are brought aboutwhen the members chosen for the experimental grouphave extreme scores on the dependent variable to beginwith. We know from the law of probability that thosewith very low scores on a variable have a greater
probability of showing improvement and scoring closerto the mean on the posttest after being exposed to thetreatment. This phenomenon of low scores tending tocloser to the mean is known as regression towards themean (statistical regression). Likewise, those with
very high abilities would also have a greater tendencyto regress towards the mean-they will score lower onthe posttest than on the pretest.
-
7/27/2019 Lecture Research Design
31/38
TYPES OF EXPERIMENTAL DESIGNS
Group Pretest score Treatment Posttest Score
Experimental group O1 X O2
Treatment effect = (O2-O1)
31
Pretest and Posttest Experimental Group DesignAn experimental group (without a control group) may be given a pretestexposed to a treatment, and then given a posttest to measure the effects of thetreatment. Where Orefers to some process of observation or measurement, Xrepresents the exposure of a group to an experimental treatment, and the Xand Os in the row are applied to the same specific group. Here, the effects of
the treatment can be obtained by measuring the difference between theposttest and the pretest (O2-O1). Note, however, that testing andinstrumentation effects might contaminate the internal validity. If theexperiment is extended over a period of time, history and maturation effectsmay also confound the results.
P tt t O l ith E i t l d C t l G
-
7/27/2019 Lecture Research Design
32/38
Group Treatment Outcome
Experimental group
Control group
X O1
O2
Treatment effect = (O2-O1)
32
Posttests Only with Exper imental and Control Groups
Some experimental designs are set up with an experimental and a controlgroup, the former alone being exposed to a treatment and not the latter. Theeffects of the treatment are studied by assessing the difference in the outcomes-that is, the posttest scores of the experimental and control groups. Here is acase where the testing effects have been avoided because there is no pretest,only a posttest. however, to make sure that the two groups are matched for allthe possible contaminating nuisance (unwanted) variables. Otherwise, thetrue effects of the treatment cannot be determined by merely looking at thedifference in the posttest scores of the two groups. Randomization would take
care of this problem.There are at least two possible threats to validity in this design. If the twogroups are not matched or randomly assigned, selection biases couldcontaminate the results. Mortality (the drop out individuals from groups) canalso confound the results,
-
7/27/2019 Lecture Research Design
33/38
33
Pretest and Posttest Exper imental and Control Group Designs
Two groups-one experimental and the other control-are both exposed to thepretest and the posttest. The only difference between the two groups is that theformer is exposed to a treatment whereas the latter is not. Measuring the
difference between the differences in the post-and pretest scores of the two groupswould give the net effects of the treatment. Both groups have been exposed toboth the pre-and posttests, and both groups have been randomized; thus we couldexpect that the history maturation, testing, and instrumentation effects have beencontrolled. This is so due to the fact that whatever happened with theexperimental group (e.g., maturation, history, testing, and instrumentation) also
happened with the control group, and in measuring the net effects (the differencein the differences between the pre-and posttest scores) we have controlled thesecontaminating factors. Through the process of randomization, we have alsocontrolled the effects of selection biases and statistical regression. Mortalitycould, however, pose a problem in this design. In experiments that take severalweeks, as in the case of assessing the impact of training on skills development, ormeasuring the impact of technology advancement on effectiveness, some of the
subjects in the experimental group may drop out before the end of theexperiment. It is possible that those who drop out are in some way different fromthose who stay on until the end and take the posttest. If so, mortality could offer aplausible (apparently valid) rival explanation for the difference between O2 andO1.
-
7/27/2019 Lecture Research Design
34/38
Group Pretest Treatment Posttest
Experimental group
Control groupO1
O3
X O2
O4
Treatment effect = [(O2-O1) - (O4-O3)]
34
Pretest and posttest experimental and control group
-
7/27/2019 Lecture Research Design
35/38
35
To gain more confidence in internal validity in experimentaldesign, it is advisable to set up two experimental groups andtwo control groups for the experiment. One experimentalgroup and one control group can be given both the pretest
and the posttest. The other two groups will be given only theposttest. Here the effects of the treatment can be calculatedin several different ways. To the extent that we come up withalmost the same results in each of the different calculations,we can attribute the effects to the treatment. This increases
the internal validity of the results of the experimental design.This design, known as the Solomon four-group design, isperhaps the most comprehensive and the one with the leastnumber of problems with internal validity.
SOLOMON FOUR GROUP DESIGN
-
7/27/2019 Lecture Research Design
36/38
.
Group Pretest Treatment Posttest
1. Experimental
2. Control
3. Experimental
4. Control
O1
O3
X
X
O2
O4
O5
O6
Treatment effect (E) could be judged by:
E= (O2-O1)
E= (O2-O4)
E= (O5-O6)
E= (O5-O3)
E= [(O2-O1) - (O4-O3)]
If all Es are similar, the cause-and-effect relationship is highly valid.
36
SOLOMON FOUR GROUP DESIGN MODEL
-
7/27/2019 Lecture Research Design
37/38
37
Solomon Four-Group Design and Threats to Internal Validity
Let us examine how the threats to internal validity aretaken care of in the Solomon four-group design. It is
important to note that subjects have been randomlyselected and randomly assigned to groups. This removesthe statistical regression and selection biases. Group 2,the control group that was exposed to both the pre-and
posttest, helps us to see whether or not history,maturation, testing, instrumentation, regression, ormortality threaten internal validity. If scores O3 and O4(pre-and posttest scores of group 2) remain the same, then
it is established that neither history, nor maturation, nortesting, nor instrumentation, nor statistical regression, normortality has had an impact. In other words, these havehad no impact at all.
-
7/27/2019 Lecture Research Design
38/38
THANK YOU
FOR YOURCONCENTRATION
38