EBN NOTEBOOK

If you could just provide me with a sample: examiningsampling in qualitative and quantitative research papers

When undertaking any research study, researchers mustchoose their sample carefully to minimise bias. This paperhighlights why practitioners need to pay attention to issues ofsampling when appraising research, and discusses samplingcharacteristics we should look for in quantitative and qualitative studies. Because of space restrictions, this editorialfocuses on the randomised controlled trial (RCT) as anexample of quantitative research, and grounded theory as anexample of qualitative research. Although these 2 designs areused as examples, the general principles as outlined can beapplied to all quantitative and qualitative research designs.

What is sampling?Research studies usually focus on a defined group of people,such as ventilated patients or the parents of chronically illchildren. The group of people in a study is referred to as thesample. Because it is too expensive and impractical to includethe total population in a research study, the ideal study sample represents the total population from which the samplewas drawn (eg, all ventilated patients or all parents of chronically ill children). This pointthat studying an entire population is, in most cases, unnecessaryis the key to the theory ofsampling. Sampling means simply studying a proportion ofthe population rather than the whole. The results of a studythat has assembled its sample appropriately can be moreconfidently applied to the population from which the samplecame. Using the examples of samples provided at the start ofthe paragraph, we can see that Chlan sampled 54 patientsfrom a population of patients who required mechanicalventilation,1(see EvidenceBased Nursing 1999 April, p49)whereas Burke et al sampled 50 children (and their parents)from a population of all children requiring admission to hospital for chronic health conditions.2(see EvidenceBased Nursing 1998 July, p79) In both studies the researchers wanted tosay something that would apply to the population by examining a small portion of those populations.

When reading a paper that sets out to say something abouta population by studying a sample, readers need to assess theexternal validity. External validity is the degree to which thefindings of a study can be generalised beyond the sampleused in the study. The ability to generalise is almost totallydependent on the adequacy of the sampling process. Nursesshould consider several possible threats to external validitywhen appraising a paper and deciding whether the resultscould be applied to patients in their care:x Unique sample selection: study findings may be applicable

only to the group studied. For example, the findings of astudy of a telephone support programme for caregivers ofpeople with Alzheimers disease that recruited a samplefrom local Alzheimers Association branches would notnecessarily apply to the population of caregivers as a wholebecause most caregivers do not belong to such localgroups, and therefore are different from the sample.

x Unique research settings: the particular context in which thestudy takes place can greatly affect the external validity ofthe findings. For example, Tourigny sampled 6 AfricanAmerican youths to learn about deliberate exposure toHIV.3 (see EvidenceBased Nursing 1998 October, p130) Thestudy took place in the context of extreme poverty in auniquely deprived US urban setting. The opportunity for asuburban district nurse in the UK to apply the theory generated by this study may be limited by the very differentsocial settings involved.

x History: the passage of time can affect the findings. Forexample, studies of mechanisms for implementing research findings within healthcare organisations might beaffected by the organisational and structural changes thatoccur at local and national levels over time. Examples ofsuch reorganisation include the effects of separatinghealthcare provision from purchasing through the NHSinternal market of the late 1980s and the creation ofhealth maintenance organisations in the US.

x Unique research constructs: the particular constructs, concepts, or phenomena studied may be specific to the groupsampled. For example, researchers evaluating the conceptof quality in healthcare should recognise that professionals and consumers may differ in their perceptions of theconcept and should be explicit about how they actuallymeasured it.

Sampling in quantitative researchQuantitative research is most often used when researcherswish to make a statement about the chance (or probability) ofsomething happening in a population. For example, aperson is 65% less likely to die a cardiac related death if sheeats a Mediterranean type diet compared with the recommended Step 1 diet of the American Heart Association.4 (SeeEvidenceBased Nursing 1999 April, p48.) Quantitative studiesusually use sampling techniques based on probability theory.Probability sampling, as it is known, has 2 central features:x The researcher has (in theory) access to all members of a

populationx Every member of the population has an equal and

nonzero chance of being selected for the study sample. Inother words they cannot have no chance of being sampled.

Three concepts relevant to probability samples are samplingerror, random sampling, and sampling bias. Each of these will bedescribed.

sampling errorProbability samples allow researchers to minimise samplingerror in that they give the highest chance of the sample beingrepresentative of the total population. Sampling error occursin all probability samples and is unavoidable because nosample can ever totally represent the population. There willalways be a gap between a samples representativeness and

Evidence-Based Nursing July 1999 Vol 2 No 368 EBN notebook

group.bmj.com on April 28, 2015 - Published by http://ebn.bmj.com/Downloaded from

the populations known or unknown characteristicsthesampling error. Readers of quantitative research should lookfor evidence that the researchers tried to combat samplingerror. Specifically, the authors should identify the study sample using a random selection process and should providesubstantiation for the sample size. The size of the samplingerror generally decreases as the size of the sample increases.

random samplingRandom selection works because as individuals enter thesample, their characteristics (which are different from thepopulation) balance the characteristics of other individuals.For example, in a randomly selected sample of users of mental health services, there will be users who are from uppersocioeconomic groups, and these will be balanced by thosewho are from lower socioeconomic groups.

Successful random sampling requires a sufficiently largesample. If the sample is large enough, then differences inoutcomes that exist between groups will be detected statistically, whereas if it is too small, important differences may bemissed. One of the clues that can alert the reader to a studythat is not large enough is the confidence interval around astudy finding. Although not all studies provide confidenceintervals, these are becoming increasingly popular in thereporting of quantitative studies. A confidence intervalprovides a statement on the level of confidence that the truevalue for a population lies within a specified range of values.A 95% confidence interval can be described as follows: ifsampling is repeated indefinitely, each sample leading to anew confidence interval, then in 95% of the samples, theinterval will cover the true population value.5 The larger thesample size, the more narrow the confidence interval, andtherefore the more precise the study finding. In the study byEgerman et al in this issue of EvidenceBased Nursing (p73),the relative risk of necrotising enterocolitis with oral versusintramuscular dexamethasone was 5.1 with a 95% confidence interval of 0.8 to 36.6. This range means that the truerelative risk could be as low as 0.8 or as high as 36.6 andbecause a relative risk of 1.0 (meaning no difference betweengroups) is included in this range, the conclusion is that thereis no difference between the 2 methods of dexamethasoneadministration in the risk of necrotising enterocolitis. Thereader should, however, take careful note that this confidenceinterval is very wide. It is possible that with a larger samplesize and, consequently, a narrower confidence interval, a statistically significant difference between groups may havebeen found because the confidence interval may no longerinclude the relative risk of 1.0.

sampling biasUnlike sampling error, which cannot be avoided completely,sampling bias is usually the result of a flaw in the researchprocess. It is systematic, and increasing the size of the samplejust increases the effect of the bias. Sampling bias occurswhen the sample is not representative of the population. Anexample of sampling bias has already been highlighted in thesection on external validity: sampling only caregivers fromAlzheimers Association groups introduces a sampling bias ifthe study aim is to generalise to the whole population ofcaregivers of people with Alzheimers disease. Bias, in thecontext of RCTs, can be thought of as ...any factor or process that tends to deviate the results or conclusions of a trialaway from the truth.6 Two important biases related to sampling that could affect generalisability of study findings arereferral filter bias and volunteer bias. In referral filter bias, the

selection that occurs at each stage in the referral processfrom primary to secondary to tertiary care can generatepatient samples that are very different from one another.7 Forexample, the results of a study of patients with asthma underthe care of specialists are not likely to be generalisable topatients with asthma in primary care settings. In volunteerbias, people who volunteer to participate in a study may haveexposures or outcomes (eg, they tend to be healthier) thatdiffer from those of nonvolunteers.8

When appraising a research report, readers can ask a simple set of questions to assess whether sampling bias exists:x Who was included in the sample?x What was the source of recruitment into the study?

Community or hospital? Specialised centres or generalhospitals?

x How were patients recruited? Approached by healthcareprofessionals or study researchers? Paid or unpaid?

x Which patients were approached to be in the study?Consecutive patients? Volunteers or only people theresearcher thought looked like useful candidates?

x What were the inclusion and exclusion criteria for thesample? What were the demographics (eg, age and sex) ofthe sample? Did they have any medical conditions or a history of previous interventions that may have a bearing onthe results of the study?

Some research designsspecifically, RCTsuse probabilitytheory slightly differently. In these studies, individuals mightbe initially sampled on a nonprobability basis (eg, all peopleattending an asthma clinic in 1 year) and then randomly allocated to an experimental or control group (eg, selfmanagement education or usual care). It is in the process ofallocation that probability theory comes into play. Throughrandom allocation, and provided that the sample is largeenough, all known and unknown confounders are equallydistributed among the groups and therefore, at the end of thestudy, any differences among the groups can be attributed tothe intervention. Randomisation alone, however, does notallow the researcher to generalise to a population. Questionsof sample size and the representativeness of the sample as awhole are still important. Readers should look for evidencethat the sample was large enough and that there were nobaseline differences between the experimental and controlgroups. If they are satisfied on these 2 counts, then they cango on to consider the similarity between the study sampleand their own patients. If they are similar, then the studysample can be said to be representative of their patients, andreaders can feel confident in generalising the study findings.

Sampling in qualitative researchNot all research designs are concerned with generalising froma sample to a population of people.Qualitative studies use richand deep description to inform our understanding ofconcepts and contribute to broader theoretical understanding.For example, Wilsons study of the experiences of older adultsmaking the transition to life in a nursing home says nothingabout the amount of experience or the probability ofadjustment to a nursing home; instead, the author simplydescribes what that experience might be like.9 (See EvidenceBased Nursing 1998 Jul, p96.) On the basis of this description,the author proposes a 3 stage theory about the process ofadjustment to a nursing home placement. What this means isthat nurses should consider the possibility that these stagesmight need to be addressed with patients undergoing thistransition. Good qualitative enquiry leads to theoretical generalisability rather than statistical inference.

Evidence-Based Nursing July 1999 Vol 2 No 3 EBN notebook 69

group.bmj.com on April 28, 2015 - Published by http://ebn.bmj.com/Downloaded from

Because we want to generalise about the shape or contentof a concept or contribute to a theoretical understanding ofan aspect of healthcare, sampling in qualitative research isdriven by a very different set of concerns than those of quantitative designs.

The purpose [of sampling in qualitative research] is not toestablish a random or representative sample drawn from a population but rather to identify specific groups of people who either possess characteristics or live in circumstances relevant to the socialphenomenon being studied. Informants are identified because theywill enable exploration of a particular aspect of behaviour relevantto the research.This allows the researcher to include a wide range oftypes of informants and also to select key informants with access toimportant sources of knowledge.10

Qualitative researchers therefore use nonprobabilitysampling techniques as the basis for their studies. For example, if a researcher wanted to know more about the content ofunpleasant experiences among people receiving primaryhealth care, it would be of no value to ask a random sampleof the population because most people have relatively pleasant experiences of consuming primary care. It would be farmore advantageous to interview people who were unhappywith the care they received (perhaps identified fromcomplainant records). If we wanted to know the extent ofunhappy experiences among consumers then, of course, thesampling strategies would be reversed and we mightrandomly select individuals from the whole population andattempt to measure the amount of dissatisfaction.

non-probability samplingJust as there are various approaches to sampling in quantitative studies, there are several approaches to nonprobabilitysampling used in qualitative research. The same questionsasked of probability samples can be asked of nonprobabilitysamples. Questions of sample size, however, are not asimportant here. Rather, it is the fitness for purposeorqualityof the sample that should be described in the paper.This is difficult to judge in qualitative studies because theadequacy of the sample can often only be judged by thequality of the analysis of the data generated. In some qualitative studies, sampling continues until no new themes emergefrom the data, a point called data saturation. The earlierquestions remain useful as a way of focusing your scrutiny.x Who was included in the sample? That is, are the

participants able to adequately contribute to knowledgeand understanding?

x What was the source of recruitment into the study? Is thiscontext likely to influence the shape of the analysispresented?

x How were patients recruited? Approached by healthcareprofessionals or study researchers? Paid or unpaid? Arethese factors likely to alter the stories, accounts, or behaviours presented, and if so how did the researcher accountfor this?

x Which patients were approached to be in the study?Consecutive patients? Volunteers or only people theresearcher thought looked like useful candidates?

x What were the inclusion and exclusion criteria for thesample? For example, what were the demographics (eg, ageand sex) of the study sample? Did patients have any medical conditions or history of previous interventions that mayhave a bearing on the results of the studyin qualitativeresearch this usually means the accounts people provide orbehaviours presented.

x All of these questions remain relevant, but clues thatshould help readers to assess the adequacy of a qualitative

sample include the believability of the final product andthe sense of completeness in the analysis presented. A solidqualitative paper should generate theory or descriptionthat encompasses a range of experiences or values andmakes explicit the limitations of qualitative research inregard to conventional generalisability.

Qualitative and quantitative sampling: a finalword of cautionThis overview has used the qualitativequantitative distinction to highlight the primary differences between probabilityand nonprobability sampling strategies. In reality, however,this distinction is often blurred and many studies use a combination of 2 or more approaches. This is not necessarily acriticismsampling is an integral part of the application ofresearch methodology and because of this it should beguided by the research question. Like so much in evidencebased health care, the key questions to ask about any studyssampling strategy relate to the fitness for purpose. Threequestions that capture the quality of any sample are:x Are the patients in the study similar enough to my patients? In

quantitative terms, this enables you to judge whether theresults of the research can be applied to your population. Inqualitative terms, the similarity of patients enables you tojudge whether the concept or experiences being exploredwould be meaningful to your practice.

x Was the sample selected in such a way that it could introduce biasinto the researchthis is as applicable to qualitative researchas it is to quantitative approaches. In qualitative research,however, the boundaries or limitations of theoretical generalisability are usually encompassed in the description ofthe participants.

x Was the sample large enough? In quantitative research, thismeans was it large enough to detect a difference betweengroups if a difference existed? In qualitative research, thismeans were there enough people (or enough time spentwith a few people) to provide rich meaningful descriptionand convincing analysis?

In summary, an understanding of how patients areassembled for a study, plus additional information abouttheir characteristics (eg, age, sex, and disease severity), willhelp readers to decide whether the study sample resemblestheir own patients enough for them to apply the results totheir own practice.7

CARL THOMPSON, RN, PhDCentre for Evidence Based Nursing,Department of Health Studies,University of York,York, UK

1 Chlan L. Effectiveness of a music therapy intervention on relaxation andanxiety for patients receiving ventilatory assistance. Heart Lung1998;27:16976.

2 Burke SO,HandleyDerry MH, Costello EA, et al. Stresspoint interventionfor parents of repeatedly hospitalized children with chronic conditions.Res Nurs Health 1997;20:47585.

3 Tourigny SC. Some new dying trick: African American youths choosingHIV/AIDS. Qual Health Res 1998;8:14967.

4 de Lorgeril M, Salen P, Martin JL, et al. Mediterranean dietary pattern in arandomized trial: prolonged survival and possible reduced cancer rate.Arch Intern Med 1998;158:11817.

5 Satin A, Shastry W. Survey sampling: a nonmathematical guide. Ottawa:Statistics Canada, 1985: 732.

6 Jadad A. Randomised controlled trials. London: BMJ Publishing Group,1998:28.

7 Sackett DL,Haynes RB, Guyatt GH, et al. Clinical epidemiology.A basic sciencefor clinical medicine. Second edition. Boston: Little, Brown and Company,1991.

8 Sackett DL. Bias in analytic research. J Chron Dis 1979;32:5163.9 Wilson SA. The transition to nursing home life: a comparison of planned

and unplanned admissions. J Adv Nurs 1997;26:86471.10 Mays N, Pope C.Qualitative research in health care. London: BMJ Publishing

Group, 1996:123.

Evidence-Based Nursing July 1999 Vol 2 No 370 EBN notebook

group.bmj.com on April 28, 2015 - Published by http://ebn.bmj.com/Downloaded from

quantitative research papersexamining sampling in qualitative and If you could just provide me with a sample:

Carl Thompson

doi: 10.1136/ebn.2.3.681999 2: 68-70 Evid Based Nurs

http://ebn.bmj.com/content/2/3/68Updated information and services can be found at:

These include:

References #BIBLhttp://ebn.bmj.com/content/2/3/68

This article cites 6 articles, 1 of which you can access for free at:

serviceEmail alerting

box at the top right corner of the online article. Receive free email alerts when new articles cite this article. Sign up in the

CollectionsTopic Articles on similar topics can be found in the following collections

(53)Sexual transmitted infections (viral) (23)Psychogeriatrics

(23)Psychiatry of old age (113)Memory disorders (psychiatry) (323)Immunology (including allergy)

(50)HIV/AIDS (240)Drugs: infectious diseases

(83)Drugs: CNS (not psychiatric) (85)Dementia

(89)Adolescent health (441)Child health

(35)Mechanical ventilation (35)Mechanical ventilation

Notes

http://group.bmj.com/group/rights-licensing/permissionsTo request permissions go to:

http://journals.bmj.com/cgi/reprintformTo order reprints go to:

http://group.bmj.com/subscribe/To subscribe to BMJ go to:

group.bmj.com on April 28, 2015 - Published by http://ebn.bmj.com/Downloaded from