Assessing the Implications of Publication Bias for Two Popular Estimates of between-StudyVariance in Meta-AnalysisAuthor(s): Dan JacksonSource: Biometrics, Vol. 63, No. 1 (Mar., 2007), pp. 187-193Published by: International Biometric SocietyStable URL: http://www.jstor.org/stable/4541314 .

Accessed: 24/06/2014 20:52

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact support@jstor.org.

.

International Biometric Society is collaborating with JSTOR to digitize, preserve and extend access toBiometrics.

http://www.jstor.org

This content downloaded from 62.122.77.83 on Tue, 24 Jun 2014 20:52:59 PMAll use subject to JSTOR Terms and Conditions

BIOMETRICS 63, 187-193 DOI: 10.1111/j.1541-0420.2006.00663.x March 2007

Assessing the Implications of Publication Bias for Two Popular Estimates of between-Study Variance in Meta-Analysis

Dan Jackson

Centre for Operational Research and Applied Statistics, University of Salford, Manchester M5 4WT, U.K.

email: d.j ackson@salford. ac.uk

SUMMARY. Perhaps the greatest threat to the validity of a meta-analysis is the possibility of publication bias, where studies with interesting or statistically significant results are more likely to be published. This obviously impacts on inference concerning the treatment effect but also has implications for estimates of between-study variance. Two popular and established estimation methods are considered and formulae for assessing the implications of the bias are provided in terms of a general process for selecting studies. Meta- analysts, concerned that publication bias may be present, can use these as part of a sensitivity analysis to assess the robustness of their estimates of between-study variance using any selection process that is likely to be used in practice. The procedure is illustrated using a meta-analysis of clinical trials concerning the effectiveness of endoscopic sclerotherapy for preventing death in patients with cirrhosis and oesophagogastric varices.

KEY WORDS: Between-study variance; Meta-analysis; Publication bias.

1. Introduction

Meta-analysis, the statistical process of combining results from separate studies concerned with the same treatment or issue, has become popular in medical applications. In or- der to combine apparently disparate results and still ob- tain a meaningful estimate of overall treatment effect some

(e.g., DerSimonian and Laird, 1986; Hardy and Thompson, 1996; Biggerstaff and Tweedie, 1997) have suggested the ran- dom effects model, described in detail in Section 2. This in- corporates a between-study variance, denoted by 72, which describes the variation in the studies' results that cannot be explained by within-study variation alone.

Perhaps the greatest threat to the validity of a meta- analysis is the possibility of publication bias. There has been an awareness of bias in published results for some time

(Sterling, 1959), including medical data (Begg and Berlin, 1988; Easterbrook et al., 1991). Although trial registration may lessen the implications of this type of bias in the near future, Carter, Griffin, and Carter (2006) recently found that publication bias remains a problem and Egger, Schneider, and Davey Smith (1998) conclude that meta-analyses of observa- tional studies are commonly used, for which no such registra- tion is currently possible.

The overall treatment effect is the parameter of primary importance but T2 is also estimated in a random effects meta- analysis and publication bias impacts on the assessment of this. It is hoped that this article will serve to draw attention to this underappreciated issue, as estimates of between-study variance are also important when interpreting the findings from random effects meta-analyses. In particular, a large value of T2 relative to the size of the studies indicates that there

are important differences in the studies' results, which should be investigated (Thompson, 1994; Glasziou and Sanders, 2002).

Jackson (2006) found that estimates of 72 that take into account publication bias can be either greater or less than the naive estimate, obtained under the implicit assumption that publication bias is not present, indicating that it is im- possible to make generalizations concerning the implications of the bias. This article builds upon this result by consider- ing an arbitrary selection process and adopts a novel strategy where the standard meta-analysis model is embedded into a wider class of models that incorporate the bias. How esti- mates change as we move from the standard model to one which models the bias can, therefore, be assessed.

The bias will be modeled using weight functions, as fre- quently used in the publication bias literature (e.g., Iyengar and Greenhouse, 1988; Dear and Begg, 1992; Hedges, 1992; Cleary and Casella, 1997; Larose and Dey, 1998). We assume a standard model for the distribution of the studies' results, yi, denoted by Pi(yi), and a model for the selection pro- cess. This is the probability of publishing a study given yi, Pi(pub I yi), where "pub" denotes that the study is published. The function Pi (pub I yi) is referred to as the weight function and hence the distribution of yi, given that the study is pub- lished, is obtained as

Pi (Yi)Pi(pub I yi)

Pi (yi I pub) = (1) Pi (pub)

Because the studies in the meta-analysis are published, the observed studies' results are distributed as in equation (1). The denominator is obtained by integrating yi out of the

© 2006, The International Biometric Society

This content downloaded from 62.122.77.83 on Tue, 24 Jun 2014 20:52:59 PMAll use subject to JSTOR Terms and Conditions

188 Biometrics, March 2007

numerator, numerically if necessary. Throughout this article, equation (1) will be used with Pi (yi) =

12 ( )/

that is yi r N(p, ai + 72), so that the random effects model, described in detail in Section 2, is adopted. This is an estab- lished way of making use of weight functions (Dear and Begg, 1992; Hedges, 1992).

The rest of this article is set out as follows. In Section 2 the random effects model is discussed, and in Section 3 suitable

properties of the weight functions are described. In Sections 4 and 5 formulae for assessing the robustness of estimates of T2 are developed. The use of these for assisting with a sensitivity analysis is illustrated in Section 6, using a meta-analysis of clinical trials concerning the effectiveness of endoscopic scle-

rotherapy for preventing death in patients with cirrhosis and

oesophagogastric varices. Sections 7 and 8 provide some con-

cluding remarks and details of the supplementary materials associated with the article.

2. The Random Effects Model

The random effects model assumes that the studies' estimates of treatment effect are distributed as yi ti N

i• N(ji, a), where o2 is the within-study variance and pi is the true treat- ment effect for the ith study. It is further assumed that pi ~

N(p, T2), where p denotes the overall treatment effect, and that the studies are independent. The parameter T2 denotes the between-study variance. Together, these assumptions pro- vide the marginal distributions yi . N(t, 2 + T2), where the

yi are independent. If T72 is determined to be zero, then all studies are assumed to have the same true treatment effect and the model simplifies to a fixed effects model. The val-

ues of a? are estimated in practice but are usually assumed fixed and known (Hardy and Thompson, 1996; Biggerstaff and

Tweedie, 1997). One way to estimate the value of 72 is to use the statistic

Q= (y, - Y)2,(2) i=1

where wi = 1/a?, - =1WiYi/ =1 wi, and n is the num- ber of studies in the meta-analysis. Under the assumptions of the random effects model, it can be shown that the expecta- tion of Q is given by

i1 2

E(Q) = (n - 1)+ wi L 2

i=1 n

i

i=1

which provides the DerSimonian and Laird (1986) estimate of 72

D2L = max 2

.

i=-1 i=l i=l

Another popular approach for estimating 72 is to use max- imum likelihood. Under the random effects model the log likelihood is

1 (2y -)2 (3) L(p, T2)= -2

log (27r( T + 72)} 2( +2 (3)

i=l

Hence by numerically maximizing this likelihood we can obtain an estimate of T2. As with the DerSimonian and Laird estimator, the maximum likelihood estimator of 72 is constrained to be nonnegative. Once T2 has been estimated, the pooled estimate of overall treatment effect is given by f i=

w /i= w where =

1/oa + 72. Assuming that there is a sufficiently large num-

ber of studies, the distribution of f is approximately

ft N(p, 1/Zi=

w*). Hence approximate inferences involv-

ing hypothesis tests and confidence intervals are read-

ily obtained. In addition to this standard approach, an alternative method for meta-analysis has been proposed (Hartung, 1999; Hartung and Knapp, 2001). Because this pro- cedure involves estimating T2, this is also subject to the con- clusions of this article.

3. Publication Bias and Weight Functions

In order to model publication bias, let us parameterize our

weight function in terms of a single parameter 0, Pi(pub I Yi) cx pi(P, yi). We need only specify our weight function in this way as it provides relative publication probabilities (Hedges, 1992) and any constant of proportionality cancels

immediately from equation (1). So that this provides an in-

terpretable model, it will be defined as having the following properties:

(i) The parameter / measures the severity of the bias. A value of 3 = 0 indicates no publication bias and that the usual random effects model applies, that is, p,(0, yi) = 1. More generally, a weight function providing pi(0, yi) = c, where c is a constant, also means that 3 = 0 indicates no publication bias. Such a weight function can be multiplied by 1/c, however, without changing the distributions of the studies' results, providing the assumed property that pi(0, yi) = 1.

(ii) As / gets larger, so does the amount of publica- tion bias. This property might be interpreted as

(&pi(/, y0)/O) < 0 for all yi, so that as the bias in- creases the probability of publishing studies decreases.

Alternatively, this might mean that as / increases, studies with more optimistic estimates of treatment effect become more likely to be published relative to those with less promising results. Exactly how /P mea- sures the severity of the bias is unimportant in the analysis that follows, but it should do this in an inter- pretable way.

(iii) The subscript i is used in pi(/, yi) to indicate that the weight functions will typically depend upon study i's within-study variance, oa, and possibly other study specific variables.

Note that these properties say very little about the nature of the bias, in particular they do not specify whether this is one or two sided. Here one sided refers to bias where studies

indicating that the treatment is beneficial are more likely to be

This content downloaded from 62.122.77.83 on Tue, 24 Jun 2014 20:52:59 PMAll use subject to JSTOR Terms and Conditions

Assessing Implications of Publication Bias 189

published, and two sided means that studies indicating that the treatment is either beneficial or harmful are more likely to be published. See Cleary and Casella (1997) and Preston, Ashby, and Smyth (2004) for a wide variety of simple but plausible weight functions suitable for modeling publication bias.

The intention is, therefore, not to work with any partic- ular weight function but to follow the approach of Jackson

(2006) and ascertain how estimates that take into account the possibility of publication bias compare with their naive

counterparts. This is considered preferable to an entirely estimation-based procedure, as the estimation of parameters determining the degree of publication bias is generally very imprecise (Copas and Shi, 2000; Preston et al., 2004). Fol- lowing Jackson (2006), the derivative of estimates of T2 with respect to 3 will be evaluated at 0 = 0. By examining the magnitude of this derivative, we can assess how robust esti- mates are to the assumption of no publication bias.

4. The DerSimonian and Laird Estimate

Jackson (2006) considered only a special case to illustrate that conclusions for the DerSimonian and Laird estimate depend on the form of the weight function. The results obtained in this section and the next build on this and show how to as- sess the implications of publication bias for estimates of T2 for any weight function that is likely to be used in practice. The necessary passing of derivatives through integrals inher- ent in the proofs of Theorems 1 and 2 below can be justified under very general conditions by applying Lebesgues's Dom- inated Convergence Theorem (e.g., Casella and Berger, 2002, Section 2.4). All that is required in addition to this for the analysis of the DerSimonian and Laird estimate is that the

weight function provides a differentiable expectation of Q (equation (5) below), as this is assumed when using the stan- dard result for implicit differentiation. For convenience, in terms of standardized scores, zi = ,• - equation (2) becomes

S(ni)2

= zi - i= (4) i=1

where w = wi. The procedure suggested by DerSimonian and Laird assumes the random effects model, that is, with no publication bias. However, we can find the expectation of Q using a particular weight function and data set, numerically if necessary, and find an estimate of T2 by matching this moment, again probably numerically. Taking expectations of equation (4) and a little rearrangement gives E(Q Ipub), the expectation of Q for a meta-analysis of published studies, of

Z 1 ) E (Z1 pub) i=1

Zw/iE(Zi p {Z

' E(Zi I pub) + i=1 i=1(5)

. W

where, from equation (1), the probability density functions of the yi are

Yi -

Ai)

f(yi I pub) = 0 Yi - p Pi (, Yi) dyi

so that

f(zi Ipub) = .

(6)

P0 i

A '-i

+ p dzi

Hence E(Zi I pub) and E(Z| I pub) can be obtained by inte-

grating the product of equation (6) and zi and z2, respectively. From these expectations E(Q I pub) in equation (5) can be ob- tained. Estimates of T2 which take into account publication bias can be obtained in the way developed in Jackson (2006). Let E(Q I pub) = f(T2, /3, p), SO that for estimation purposes we equate f(2, /, pt) to its observed value which, for a given value of 1L, depends only on the two variables / and T2. We therefore have a function of these set equal to a constant (the observed value of Q). Implicit differentiation, as shown in the proof in Web Appendix Al (see Section 8 for details), can be used to obtain the following result.

4.1 Theorem 1

For the DerSimonian and Laird estimate of 72, assuming that this has not been truncated to zero,

d2 (wi-.)Cov Z , qi ( i +t)

=n, (7)

d/3 =02 - 2

i=1

where qj(y) = (Opi(, y)/00)Ip=o and ZijrN(0, 1 +

The denominator of equation (7) is positive and hence the sign of the derivative is determined by the covariances in the numerator. If the estimate has been truncated to zero, then a small positive /3 will also provide 2L = 0, and the derivative

(7) is zero.

5. The Maximum Likelihood Estimate

A limitation of extending the DerSimonian and Laird ap- proach to the case where there is publication bias is that conclusions depend on the value of ,, which is unknown. Max- imum likelihood estimation seems a preferable approach, as

# is estimated simultaneously with 72 and there is no need to resort to using particular values of overall treatment effect. Hence this method will be adopted for the remainder of the article. With publication bias the log likelihood, L(/3, , 72), is given by

This content downloaded from 62.122.77.83 on Tue, 24 Jun 2014 20:52:59 PMAll use subject to JSTOR Terms and Conditions

190 Biometrics, March 2007

L(,, r2) + •og{pi•(,

y)}

- log

pi (o,

yi)

- dy i=1

/a

2 dy

(8) where L(p, T2) is given in equation (3). Using equation (8) we have

dL 72 { 2(0)} = 0

and

dL

oP P{ , (p)} = 0,

where 4(/) and 2(o) denote the maximum likelihood esti- mates for a given value of 0. Differentiating these two expres- sions with respect to 3 results in two simultaneous equations

in and Solving these and substituting / = 0 gives

d2L d2L d2L d2L d22 2a2 4p aI2 a72

d•p=o d2L d2L ( 2L

2

dp ()2(2)2 dOO2 0=0,72=f2(0),qp=A(0) (9)

Of course, P(0) and r2 (0) are the naive maximum likelihood

estimates, that is, the estimates under the assumption that the random effects model applies and there is no publication bias. Assuming that i2(0) # 0, the denominator is positive; it concerns derivatives with respect to 1t and T2 evaluated at

0 = 0. Because 3 plays no role in obtaining these derivatives, except taking the value 0, these are in effect derivatives of equation (3): note that pj(0, yi) = 1, so that partial deriva- tives of the second summation in equation (8) with respect to p and r2 are identically 0. These derivatives are evaluated at the maximum likelihood estimates of the naive model, and we know that the naive likelihood is at a maximum at this

point. Hence by the test for stationary points of functions of two variables (see, e.g., Anton, 1988, p. 1022) the denomi- nator is positive and is easily evaluated from equation (3). The numerator, therefore, determines the sign of the deriva- tive. Evaluating equation (9), as shown in the proof in Web Appendix A2 (see Section 8 for details), gives Theorem 2.

5.1 Theorem 2

For the maximum likelihood estimate of 7•2,

assuming that

#2(0) 0,

dp

Cov{Ye, q% (Y)} 1 _

Cov[{Y•

- t(0)}2,

Qi (i)]

i=1 "'i +2(0) 2

i= .2

(0)1

C2C3- Cj2 (10)

where Y N ^{(0), ai + 72(0)}, n n

CY= - (0) C2 1

i=1 { ?+ 2(0) }1 i=(0)

and

3- 2{fy -

(0)}2 - I -2(0) i=1 2f ?+-2(0) }

If we write the second covariance in the numerator of equation (10) in terms of its integral and change vari- ables zi = {y- 4(0)}/ai, we obtain an expression pro- portional to Cov[Z2, qi{(Zi/(wi)1/2) + f(0)}], where Zi N{0, 1 + wii2(0)}. This has obvious similarities with the co- variances that feature in Theorem 1, indicating that this type of criterion plays a key role for both estimation methods.

6. A Sensitivity Analysis The usefulness of Theorems 1 and 2 is that, having conducted a random effects meta-analysis but suspecting publication bias, we can assess the robustness of estimates of -r2 to the as-

sumption of no publication bias in a relatively straightforward manner. Models for the publication process are not verifiable, so a variety of plausible models should be examined, and The- orems 1 and 2 facilitate a thorough sensitivity analysis.

The sclerotherapy dataset has been used as an example sev- eral times (Thompson and Sharp, 1999; Sharp and Thompson, 2000; Jackson, 2006). Briefly, this involves 19 two by two ta- bles summarizing results concerning the number of deaths in

assessing the effectiveness of endoscopic sclerotherapy for pre- venting death in patients with cirrhosis and oesophagogastric varices. The data are shown in Table 1, where yi and ai are the log odds ratios and their variances obtained, after adding halves to all table entries, in the usual way (Sutton et al., 2000). Note that as calculated here, a negative log odds ratio indicates that the treatment is beneficial. The DerSimonian

Table 1 The sclerotherapy data

Study (i) Yi a 1 -2.156 0.543 2 -1.455 0.177 3 0.174 0.483 4 -0.535 0.488 5 -0.568 0.193 6 0.067 0.189 7 -0.712 0.157 8 -0.717 0.139 9 -0.956 0.223

10 -1.760 2.501 11 -0.042 0.191 12 0.024 0.238 13 0.852 0.083 14 -0.511 0.156 15 0.296 0.158 16 -0.504 0.789 17 0.011 0.385 18 0.303 0.448 19 0.108 0.472

This content downloaded from 62.122.77.83 on Tue, 24 Jun 2014 20:52:59 PMAll use subject to JSTOR Terms and Conditions

Assessing Implications of Publication Bias 191

cO c 0- o

o

o 0 0 0

Sc 0 00

0 . ooo

oo

-2.0 -1.5 -1.0 -0.5 0.0 0.5

log odds ratio

Figure 1. Funnel plot for the sclerotherapy data.

and Laird and maximum likelihood estimates of 72 are 0.303 and 0.244, respectively, which result in 95% confidence inter- vals for p with an upper limit of approximately zero, providing some evidence of an overall treatment effect.

The funnel plot in Figure 1, a plot of precision, 1/ai, against yi is rather asymmetric. This suggests that one sided pub- lication bias may be present, where studies indicating that the treatment is effective are more likely to be published. Of course there are other explanations for this apparent asym- metry including a type of heterogeneity where the quality of studies depends on their size. If this is the case then smaller studies may provide more optimistic results in the absence of publication bias and it is very difficult to distinguish between this heterogeneity and publication bias in practice. The inten- tion here, however, is to assess the potential impact of publi- cation bias on the estimate of T2, using suitable models for the bias. As the DerSimonian and Laird procedure is somewhat unsatisfactory, as noted above, results will be obtained to as- sess the robustness of the estimate of T2 using the method of maximum likelihood alone. Findings for the weight function used below, and using the DerSimonian and Laird estimation method, have already been illustrated in Jackson (2006) to which the interested reader should refer.

Although results may be obtained for any suitable weight function, perhaps the simplest type will be used to serve as an illustration. This is given by

iW, () 1 if Yi < S

(1 - )) if y

>

where 3 < 1 and S is a "step" or "cut-off' point that is re-

garded as fixed. Hence (11) describes a family of weight func- tions and, as S is fixed, we have that L(0, p, T2) is differen- tiable. This model provides

qj (Yi)= {0 if y < S (12

Although this partial derivative is discontinuous, it is defined for all yi. Substituting equation (12) into equation (10) and evaluating the covariances analytically gives

di2

d3=0

-CT {2 +

+2(0) ) - C21{S - (0)} S

- P(0)

i=1 2?+-2(0) 2 (0) L c2C3 _ C2

(13)

Alternatively, for this weight function the log likelihood, L( ,

p, 72), is given by

L(p, -r2)

+ Y(yi) log(1 -

- log 04) -P +0+1

(14) where 4P denotes the standard normal cumulative distribu- tion function, y(yi) = 0 if y, < S and y(yi) = 1 otherwise.

Obtaining the necessary derivatives required in equation (9) directly from equation (14) also provides equation (13). The role played by S in equation (13) shows that S can be se- lected to provide both positive and negative derivatives just as Jackson (2006) concluded for the DerSimonian and Laird estimate.

The derivative in equation (13), evaluated for the scle-

rotherapy data for a range of values of S, is shown in Figure 2. The derivatives are not particularly large relative to the mag- nitude of the maximum likelihood estimate of T2, indicating that this estimate is not particularly sensitive to the assump- tion of no publication bias when considering this family of

weight functions. A particular value of S can be investigated more thoroughly by plotting 72 against 0. Such a plot for S = 0 is shown in Figure 3. Also shown in this figure as a

ao

-1.0 -0.5 0.0 0.5 1.0

Value of S

Figure 2. How the maximum likelihood estimate of T2 changes as publication bias is taken into account: derivatives obtained from equation (13) against S for the sclerotherapy data.

This content downloaded from 62.122.77.83 on Tue, 24 Jun 2014 20:52:59 PMAll use subject to JSTOR Terms and Conditions

192 Biometrics, March 2007

c;V

U)

a)

-0.2 0.0 0.2 0.4 0.6 0.8 1.0 Beta

co E

co

Bet

Figure 3. Maximum likelihood estimates of 72 for the scle- rotherapy data, using the weight function in equation (11) with S = 0.

dotted line is the first-order Taylor series of T2 about 0 = 0.

Figure 3 illustrates the lack of sensitivity of the estimate of T2 for moderate values of 3 when using S = 0, as indicated

by the small derivative evident from Figure 2. However, as

/ becomes very large the estimate of T2 becomes sensitive to the value adopted and increases rapidly. Clearly Figure 3 provides far more insight into the model with S = 0 than

Figure 2. This example illustrates that although evaluating the derivative considered in this article assists with a sensitiv-

ity analysis, considering this in isolation of other investigative techniques can hardly be recommended.

The advantage of Figure 2 is that this allows a quantita- tive assessment of the whole family of weight functions. Hence a full sensitivity analysis can be efficiently performed by pro- ducing a series of figures akin to Figure 2, for a variety of fam- ilies of weight functions, with some illustrative figures such as Figure 3. Figure 3, which examines a range of possible degrees of publication bias using a particular model for the bias, pro- vides a similar approach to Copas and Shi (2000) and follows the suggestion of Preston et al. (2004).

7. Conclusions

Estimates of between-study variance are of considerable in- terest when making inferences from meta-analyses and are

subject to publication biases just as estimates of treatment effect are. It is hoped that by focusing attention on 2, this article will serve to emphasize this underappreciated issue.

By considering the derivative discussed in this article, entire families of weight functions can be examined simultaneously, facilitating a thorough sensitivity analysis in an efficient way. Particular members of these families should also be examined more closely, as illustrated using the sclerotherapy data and the weight function suggested with S = 0. Hence the theorems are intended to assist with, rather than provide, a sensitivity analysis.

This article provides a practical way to assess the implica- tions of publication bias for estimates of T2 for a wide range of

possible models, and a similar approach could also be adopted

for p. Hence the ideas presented provide a framework for sen- sitivity analyses to assess the potential impact of publication bias.

8. Supplementary Materials

A document containing Appendices Al and A2, which pro- vide proofs of the theorems, can be found at http: //www. tibs. org/biometrics. This is also available from the author on request.

REFERENCES

Anton, H. (1988). Calculus with Analytic Geometry, 3rd edi- tion. New York: Wiley.

Begg, C. B. and Berlin, J. A. (1988). Publication bias: A prob- lem in interpreting medical data (with discussion). Jour- nal of the Royal Statistical Society, Series A 151, 419-463.

Biggerstaff, B. J. and Tweedie, R. L. (1997). Incorporating variability of estimates of heterogeneity in the random effects model in meta-analysis. Statistics in Medicine 16, 753-768.

Carter, A. O., Griffin, G. H., and Carter, T. P. (2006). A

survey identified publication bias in the secondary liter- ature. Journal of Clinical Epidemiology 59, 241-245.

Casella, G. and Berger, R. L. (2002). Statistical Inference, 2nd edition. North Scituate, Massachusetts: Duxbury.

Cleary, R. J. and Casella, G. (1997). An application of Gibbs

sampling to estimation in meta-analysis: Accounting for publication bias. Journal of Education and Behavioural Statistics 22, 141-154.

Copas, J. and Shi, J. Q. (2000). Meta-analysis, funnel plots and sensitivity analysis. Biostatistics 1, 247-262.

Dear, K. B. G. and Begg, C. B. (1992). An approach for assess-

ing publication bias prior to performing a meta-analysis. Statistical Science 7, 237-245.

DerSimonian, R. and Laird, N. (1986). Meta-analysis in clin- ical trials. Controlled Clinical Trials 7, 177-188.

Easterbrook, P. J., Berlin, J. A., Goplan, R., and Matthews, D. R. (1991). Publication bias in clinical research. Lancet 337, 867-872.

Egger, M., Schneider, M., and Davey Smith, G. (1998). Meta-

analysis spurious precision? Meta-analysis of observa- tional studies. British Medical Journal 316, 140-144.

Glasziou, P. P. and Sanders, S. L. (2002). Investigating causes of heterogeneity in systematic reviews. Statistics in Medicine 21, 1503-1511.

Hardy, R. J. and Thompson, S. G. (1996). A likelihood ap- proach to meta-analysis with random effects. Statistics in Medicine 15, 619-629.

Hartung, J. (1999). An alternative method for meta-analysis. Biometrical Journal 8, 901-916.

Hartung, J. and Knapp, G. (2001). A refined method for the

meta-analysis of controlled clinical trials with binary out- come. Statistics in Medicine 20, 3875-3889.

Hedges, L. V. (1992). Modelling publication bias selection ef- fects in meta-analysis. Statistical Science 7, 246-255.

Iyengar, S. and Greenhouse, J. B. (1988). Selection models and the file-drawer problem. Statistical Science 3, 109- 135.

This content downloaded from 62.122.77.83 on Tue, 24 Jun 2014 20:52:59 PMAll use subject to JSTOR Terms and Conditions

Assessing Implications of Publication Bias 193

Jackson, D. (2006). The implications of publication bias for meta-analysis' other parameter. Statistics in Medicine 25, 2911-2921.

Larose, D. T. and Dey, D. K. (1998). Modelling publication bias using weighted distributions in a Bayesian frame- work. Computational Statistics and Data Analysis 26, 279-302.

Preston, C., Ashby, D., and Smyth, R. (2004). Adjusting for publication bias: Modelling the selection process. Journal of Evaluation in Clinical Practice 10, 313-322.

Sharp, S. J. and Thompson, S. G. (2000). Analysing the re- lationship between treatment effect and underlying risk in meta-analysis: Comparison and development of ap- proaches. Statistics in Medicine 19, 3251-3274.

Sterling, T. D. (1959). Publication decisions and their possi- ble effects on inferences drawn from tests of significance. Journal of the American Statistical Association 54, 30-34.

Sutton, A. J., Abrams, K. R., Jones, D. R., Sheldon, T. A., and Song, F. (2000). Methods for Meta-Analysis in Med- ical Research. New York: Wiley.

Thompson, S. G. (1994). Why sources of heterogeneity should be investigated. British Medical Journal 309, 1351-1355.

Thompson, S. G. and Sharp, S. J. (1999). Explaining het- erogeneity in meta-analysis: A comparison of methods. Statistics in Medicine 18, 2693-2708.

Received May 2005. Revised May 2006. Accepted May 2006.

This content downloaded from 62.122.77.83 on Tue, 24 Jun 2014 20:52:59 PMAll use subject to JSTOR Terms and Conditions