SLAM Effect

download SLAM Effect

of 17

  • date post

    03-Apr-2018
  • Category

    Documents

  • view

    225
  • download

    0

Embed Size (px)

Transcript of SLAM Effect

  • 7/28/2019 SLAM Effect

    1/17

    Why Authors Believe That Reviewers Stress LimitingAspects of Manuscr ipts:The SLAM Effect in Peer Review'

    PAUL A. M. VAN A N C E ?Departme171o/'.soc;ocia/ ' . ; ? ~ ' h o / o g~

    Free L'nri*rr.sit>,Am.v/erdam The . c.theriutidr

    This manuscript describes a prelirninap stud) examining judgments of authors andreviewers regarding manuscripts that ha\e been either accepted or rejected for publication.Consistent with hjpotheses, results reveal that participants believe that their o\vn manu-scripts are superior to others' manuscripts in terms of general. theoretical. and mrthod-ological qualit?. Relevant to the presumed tendencq among reviewers to .s / re .n /imir/n,qmpeclc of n f a n i m r i p f ( (S L A M). revieuers exhibited greater agreement 14ith editorialdecisions favoring rejection. relative to those favoring acceptance. These findings suggestthat authors' beliefs in reviewers' tendencies to S L A M can be partially understood interms o f authors' unrealisticall) favorable and optimistic beliefs regarding their nianu-scripts and in reviewers' actual tendencies to be quite critical-at least more critical thaneditors.

    Virtually all sciences rely on the peer-review system. a practice that has beendiscussed by v arious scientists. W hile there is a good deal of agreem ent a mon gscientists of different dis ciplines regarding the overall utility of the peer-re viewsystem , few (if an y) believe that the peer-review system is without limitations(e.g., Laband & Piette, 1994;Peters & Ceci. 1985).

    Recently, Epstein ( 1995) fostered further debate regarding the peer-reviewsystem, noting that reviewers tend to place particular emphasis on (often se em-ingly correctable) imperfections and limitations, rather than on the innovativeaspects of a manuscript. He summ arized a discussion with fellow sc ient ists (se v-eral of whom we re associate editors who had m any pub lication credits i n APA

    'This research \\as supported in part by a grant from the Netherlands Organization for ScientificResearch (N WO ; R-57-178). I thank the organizers o ftw o conferences. Chuck Stangor and \VolfgangWagner, for their help in conducting this research; Wilma Otten and two anonymous reviewers fortheir constructive comments on an earlier draft o f this manuscript; and all o f the social psychologistswho generously participated in this research.

    ?Correspondence concerning this article should be addressed to P aul Van Lange. Department o fSocial Psychology, Free University at Amsterdam. Van der Boechorststraat I, 1081 B T Amsterdam,The Netherlands. e-mail: pam.van.Langetiilps).vu.nl

    2550Journal of Applied Social Psychology, 1999, 29,12, pp 2550-2566Copyright 0 1999 by V H Winston 8, Son, Inc All rights reserved.

  • 7/28/2019 SLAM Effect

    2/17

    THE SLAM EFFECT IN PEER REVIEW 2551journ als and w ho did a considerable amount o f reviewing themselves) by notingthat I was im pressed by the widespread discontent with the journ al review pro-cess in this select group. So me of the terms people used to describe the reviewswere arbitrary, biased, self-serving, irresponsible, and arrogant (p. 884).Epstein (1995) further illustrated such biases among reviewers by suggest-ing that reviewers frequently exhibit an I gotcha mentality, a frame of mind inwh ich reviewers are tempted to recommend rejection on the basis of minor limi-tations, even in case of an otherwise excellent manuscript (for similar observa-tions, se e Ra bino vich, 1996). Are reviewers really that bad? Do es this belief inwhat I refer to as reviewers tendenc ies to SLAM (an acronym for stressing limit-ing aspects ofm anu scripts) reflect an unbiased, accurate jud gm en t? Or might thisbelief be colored by authors tendencies to interpret reviews (and reviewersinclinations) in w ays so as to maintain a favorable view of ones own m anu-script?3While the belief in reviewers tendencies to SLAM seems to be shared bymany researchers, Levenson (1996) pointed ou t that it is common for authors toengage in rationalization after reading reviews of their manuscripts, particularlythose review s that favor rejection rather than acceptance. He referred to this phe-nomenon as the sour-grapes hypothesis,stressing the notion that reviewers ten-den cies to S LA M are, to some degree, constructed beliefs. Indeed, i t is unlikelythat review ers themselves describe their reviews in terms o f SLA M . It is evenless l ikely that reviewers themselves describe their reviews as arbitrary,biased, self-serving, irresponsible, and arrogant (Epstein, 1995, p. 884),as Epsteins fellow scientists characterized reviews. Instead, it is more likely thatreviewers believe that their (negative) appraisals are entirely justified, in light ofwhat ar e believed to be serious limitations underlying the work they review ed(e . g ., T he theo ry w as poor ly deve loped , Th e s tud ie s were so poor lydesigned; cf. Levenson, 1996). Thus, it is possible that the belief in reviewerstendencies to SL AM represents an objective reality, a constructed reality, or both.

    A Preliminary S tudy of Author and Reviewer JudgmentsIn light of the preceding discussion-and given that the accum ulation of

    knowledge relies to a significant degree on the peer-review system-it becom esimportant to examine the experiences of authors and reviewers. Th e present3Unfortunately, the acronym SLA Mmay have some specific connotations that I do not intend to

    convey. For example, one connotation might be that reviewers are merely interested i n providingauthors with negative feedback. Of course, I do not use this acronym because of such harsh connota-tions, which in my view provide a very poor and inaccurate description of reviewers motivations.Nevertheless, I elieve that this acronym captures the global meaning that I wish to convey: (authorsbelieving that) reviewers stress limitations more than they do strengths. I thank an anonymousreviewer for suggesting this acronym.

  • 7/28/2019 SLAM Effect

    3/17

    2552 PAUL A . M. VAN LANGEmanuscript describe s a preliminary study. designed to enhan ce our understandingof (a ) ho u authors jud ge the qual ity of thei r own manuscripts that have beeneither accepted or rejected for publication. (b) how such judgm ents relate to Jud g-ments of others manuscripts that have been either accepted or rqjected for publi-cation, and ( c ) the extent to which authors and reviewers agree or disagree witheditorial decisions favoring acceptance and re.jection. Specifically, this researchasks several social p sycho logists to think of the most recent manuscript that theysubmitted (vs. review ed) and that has been accepted (vs. rejected) for publica-tion. Thereafter. they are asked to judge the general quality, theoretical quality,and methodological quali ty of the manuscript and to indicate how much theyagree with the editorial decision .O ur frameLvork for understanding experiences of authors and reviewersregarding the peer-review system proposes that evaluations of own and otherswork to some extent are socially defined and colored by relatively stable beliefsof superiority. assuming that ones o m u.ork is better than and not as bad as oth-ers wor k. Th is proposi tion m ay be der ived f rom class ic and contemporaryassum ptions underlying theories of social compa rison and self-other judg me nt,which emphasize the social and self-enhancing nature ofju dgm ents , particularlyin contex ts in w hich perfectly objecti1.e standard s for jud gm ent are not a vailable(cf . Festinger. 1954: Suls & Wills. 1991:Taylor & Brown. 1988 ). Accordingly,the perceived quality of o u n m anusc ripts may be partially affected by the goodand bad features believed to be associated w ith others manu scripts (e.g .. T hispaper is actually quite good. when I look at what is being published these da ys).Similarly, the perceived quality of others manuscrip ts may be partially affectedby the good and bad features believed to be associated with ow n man uscripts( e . g . . C o m p a r e d to m y o wn w o r k . th i s \vork does not seen1 to be all thatnovel).

    Congruent u i th these theoretical approaches. there is considerable e\ idencethat judgm ents of others are affected by beliefs about the self (an d vice \ . m a ) , andthat individuals tend to regard themselves as being better than average on severalcompetence-related attributes (Suls & Wills, 199 1:Taylor & Bromn. 1988; seealso Allison, Messick, & Goethals. 1989; Van Lange & Rusbult. 199 5). Th is evi-dence suggests that people have developed relatively stable views of the self, fre-quently referred to as positiLz self-scl7ema.r Taylor & Brown, 1988),14 hich maybe used to interpret, filter. and colo r ne u self-relevant info rm ation . In part. suchinformation processing is guided by sr!fl,,7/7,i7cer?ier7f.he motivation to elexitethe positivity of ones self-conceptions and to protect ones self-concepts fromneg ativ e eva luat ion. Also. such information processing might be guided by self-\.erification, the motivation to maintain consistency between self-conceptions andnew self-relevant information (SLvann. 1983, 1990; see also Se dikides & Strube,1997). Indeed. i f people already hold beliefs of self-other superiori ty , seff-enhancement and self-verification are highly complementary mechanisms.

  • 7/28/2019 SLAM Effect

    4/17

    THE SLAM EFFECT IN PEER REVIEW 2553Importantly, given that beliefs of superiority m ay to s om e degree serv e as an

    anchor for evaluating others work-and given that it is not easy to compete withsuc h standards-we as reviewers are likely to take a fairly critical approach to thework of othe rs, thereby stressing the limitations rather than the stre ngth s of themanu scripts we review. For exam ple, consciously or uncons