0.CSE 2018 Questions combined€¦ · Cochran Associate Publisher and Journals Director •American...

5/14/2018

1

Mary Warner

•Senior Director, Periodicals

•American Pharmacists

Association (APhA)

Jill Jackson

•Managing Editor & Publishing Administrator

•Annals of Internal

Medicine, American College of Physicians

Jody Plank, PhD

•Manager, Products

& Analytics, Editorial Services

•American Chemical Society

Angela Cochran

Associate Publisher and Journals Director

•American Society of Civil Engineers

Jenny Hamlin

•Content Technical Lead

•PLOS

Moderator: Diane Scott-Lichter

•Senior Vice President, Publishing

•American College of Physicians

Editorial and Publishing Questions – Data Informed Decisions#CSE2018

American Pharmacists Association

How do we ensure that we have

the best Editorial Advisory Board

members for our journal?

Mary Warner, CAE

Senior Director, Periodicals, American Pharmacists Association

2018 CSE Annual Meeting, New Orleans, LA

May 8, 2018

Session 5.2: Editorial and Publishing Questions — Data-Informed Decisions


Editorial Advisory Boards

• Subject matter experts to advise the journal editors and staff

• Size varies from ~15 to over 500 members depending on the function of the EAB

• What makes a good Editorial Advisory Board member?

• Subject matter knowledge

• Engagement

• Contacts in the field

5/14/2018

2


Data

• Looked at performance of EAB members over the last 5 years for an EAB with rotating 3-year terms (70 total members)

• Reviewer performance• Were they available to review when asked?

• Did they return their review on time?

• Engagement with EAB activities• Attendance at EAB meetings and calls

• Participation

• Feedback• Responsiveness to ad hoc questions


Results

• Younger/early career members were more engaged with the editorial team and other EAB members

• Reviewed more often/faster

• Attended EAB meetings and calls regularly

• Provided good feedback when requested


Lessons Learned

• Mix of age, experience, backgrounds makes the best EAB

• Actively look for early career scientists and those in emerging subject areas to join the EAB

• Don’t rely only on nominations from the Journal’s editors

• Make expectations for service on the EAB clear to new members– it’s not just a ceremonial position

5/14/2018

3

Are reviewers being confidential with

manuscripts during the peer review

process?

Jill Jackson

Managing Editor & Publishing Administrator

American College of Physicians

2018 CSE Annual Meeting, New Orleans, LA, May 8, 2018

Editorial and Publishing Questions — Data-Informed Decisions

8

Annals of Internal Medicine

Annals of Internal Medicine is ranked 5th

among 154 general medicine journals.

It is one of the most highly cited

and influential journals in the world.

Annals sends 35% of submitted

manuscripts to external peer review.

9

Case of Plagiarism

In August 2016, Annals was contacted regarding a case

of plagiarism.

A peer reviewer who had evaluated the paper for

Annals subsequently published the work in another

journal as if it was his own.

5/14/2018

4

10

Scientific Misconduct Hurts

11

Dear Plagiarist

12

Confidentiality

� We decided to assess peer reviewers’ self-reported use of the

manuscripts they receive for review.

� It is not known how frequently peer reviewers use the information

gained from received manuscripts in a manner that is inconsistent

with the goals of peer review.

5/14/2018

5

13

Survey

� All recipients of Annals papers sent for external

review in 2015 and 2016 were invited to complete

an anonymous online survey.

� 1,431 of 3,275 invited reviewers (44%) completed

our survey.

14

Survey Questions

� Have you ever shown a paper you agreed to review to other

colleagues without seeking the permission of the journal’s

editor?

� Have you ever kept a copy of manuscript you reviewed after

submitting your comments?

� Have you ever used the information in a paper you reviewed

for personal and/or academic benefit?

� In what way(s) have you used the information in a paper you

reviewed for personal benefit prior to the paper’s publication?

15

Have you ever kept a copy of a manuscript you

reviewed after submitting your comments to the journal?

Yes No

0%

10%

20%

30%

40%

50%

60%

70%

476

5/14/2018

6

16

Have you ever shown a paper you agreed to review to

other colleagues without permission of the journal’s

editor?

Yes No

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

153

17

Use of information in a paper you reviewed

Reported Uses

Altered my research plans 14

Discussed results with research colleagues 7

Sped submission of my related work 8

Delayed submission of my review 2

Copied a part of manuscript for my own

work 1

Other* 6

18

Conclusion

Trusting that reviewers will treat manuscripts received for peer review

as confidential communications is an essential tenet of peer review.

Although self-reported and from a small sample, these results suggest

that abuse of such trust does occur.

5/14/2018

7

Editorial and Publishing Questions — Data-Informed Decisions

How do you make the author of a rejected manuscript happy?

Jody Plank, PhDManager, Products & Analytics

Global Journals Development | Publications Division

[email protected]


Rolling Author Survey Responses Provide a Means to Understand Author Experience and Perception

Manuscript SubmittedManuscript Submitted

Editorial/Peer Review

Editorial/Peer Review

Manuscript Accepted

Manuscript Accepted

Manuscript PublishedManuscript Published

Accepted Survey

Accepted Survey

Manuscript Rejected or Transferred

Manuscript Rejected or Transferred

Rejected or Transferred

Survey

Rejected or Transferred

Survey

20

Manuscript

Database

• 110,000+ surveys distributed each year

• Corresponding Authors only

1 Day

2 Days

Sections of the Author Surveys

21

Published Rejected Transferred

Demographics � � �

Motivations � � �

Submission & Peer Review Experience � � �

Production Experience �

Overall Satisfaction � � �

Future Plans for this Manuscript �

General Future Publishing Plans � � �

Marketing (New Journal Discovery) � � �

Start Date July 2015 July 2015 June 2017

Number of Responses (Jan, 2018) 16,900 15,600 1,500

5/14/2018

8

Sections of the Author Surveys

22

Published Rejected Transferred

Demographics � � �

Motivations � � �

Submission & Peer Review Experience � � �

Production Experience �

Overall Satisfaction � � �

Future Plans for this Manuscript �

General Future Publishing Plans � � �

Marketing (New Journal Discovery) � � �

Start Date July 2015 July 2015 June 2017

Number of Responses (Jan, 2018) 16,900 15,600 1,500

Specific aspects of the peer review experience surveyed

23

Peer Review Experience

Ease of form completion (e.g. Copyright)

Clarity of Author Guidelines

Overall experience with submission system

Contact with the Editor

Contact with editorial staff

Speed of peer review decision

Clarity with which peer review outcome and feedback was shared

Comments received from Editors

Comments received from peer reviewers

Appropriateness of expertise demonstrated by Editor's feedback

Appropriateness of expertise demonstrated by peer reviewers’ feedback

Editor communication is strongly correlated with Author satisfaction

24

0.262

0.281

0.357

0.588

0.511

0.474

0.652

0.718

0.631

0.730

0.655

0.0 0.2 0.4 0.6 0.8







Clarity of peer review and feedback communication



Appropriateness of expertise shown by the Editor

Appropriateness of expertise shown by the peer reviewers

Pearson’s Correlation Coefficient with Overall Satisfaction

5/14/2018

9

Rejected Authors are satisfied if they feel the Editor displayed appropriate expertise

0

1

2

3

4

5

0 1 2 3 4 5ExcellentPoor Fair Average Good

Extremely

satisfied

Somewhat

satisfied

Neutral

Somewhat

dissatisfied

Extremely

dissatisfied

Appropriateness of expertise demonstrated by Editor's feedback

Ov

era

ll S

ati

sfa

cti

on

There is a strong link between dissatisfaction and interest in future submissions

26

“When do you expect to submit another manuscript to this ACS journal?”

Overall Satisfaction

0%

10%

20%

30%

40%

50%

60%

70%

Extremelydissatisfied

Somewhatdissatisfied

Neutral Somewhatsatisfied

Extremelysatisfied

Pe

rce

nt

of

Au

tho

rs s

tati

ng

“N

ev

er”

Overall Satisfaction

Rejected Author Satisfaction Since the Editor’s Conference

27

4.25

4.32

4.204.24

4.094.15

3.70

3.76

3.873.92

3.83

3.87

3.393.50

3.113.24

3.37

3.49

3.073.22

3.35

3.51

3.22

3.34

1 2 3 4 5












Overall Experience

Average Score ±Standard Error

0.0059

0.0603

0.0310

0.1540

0.1510

0.2322

0.0082

0.0054

0.0113

0.0016

0.0020

0.0034

P-Value*

*Values in gray are > 0.01 (1%)

� Six Months Prior � Three Months Since

5/14/2018

10

Rejected Author Satisfaction Since the Editor’s Conference

28

4.25

4.32

4.204.24

4.094.15

3.70

3.76

3.873.92

3.83

3.87

3.393.50

3.113.24

3.37

3.49

3.073.22

3.35

3.51

3.223.34

1 2 3 4 5












Overall Experience

Average Score ±Standard Error

0.0059

0.0603

0.0310

0.1540

0.1510

0.2322

0.0082

0.0054

0.0113

0.0016

0.0020

0.0034

P-Value*

*Values in gray are > 0.01 (1%)

� Six Months Prior � Three Months Since

Conclusions

• Rolling author surveys can be a powerful tool to generate convincing and actionable data

– Plan the surveys very carefully to ensure they will provide meaningful

data over time

– Continuously monitor and communicate the results to the editors

• Even rejected authors can be very satisfied with their editorial experience

– Dispel the idea that rejected authors will just be unhappy because

they were rejected

• Group discussions/brainstorming among Editors can be very

productive

29

SESSION 5.2: EDITORIAL AND PUBLISHING QUESTIONS — DATA-INFORMED DECISIONS

Angela Cochran, Associate Publisher, ASCE


@acochran12733

Can we improve

turnaround time by

tweaking auto-reminders?

5/14/2018

11

Can we improve turnaround time by tweaking auto-reminders?

�Asked editors why it took so long to get to a final decision.

�Blamed reviewers for taking a long time.

�Anecdotal not evidence based.

�Created Editor and Reviewer Performance Report

@acochran12733

@acochran12733

@acochran12733

5/14/2018

12

�Delay not with reviewers• Given 45 days, completing in ~36 days

�Delays with editors

�Ed office asked to change 45 day reviewer deadline to 30

�Editors reluctant

@acochran12733

Reviewer Reminder Structure

�Review 1 due in 45 days

�Pre-reminder (your review is due in 5 days) would go out

�Most reviews came in the day after the reminder

�Changed Review 1 pre-reminder to 10 days before due date

@acochran12733


Deadline Pre-reminder Time taken

45 days 5 days before

deadline

36 days


deadline

29 days


deadline

22 days

21 days (a few

journals)

10 days before

deadline

15 days

@acochran12733

5/14/2018

13


� Invite reminder: 5 days after Invite

�Uninvited: 10 days after invite

�System reviewer reminders 1. 10 days before due

2. 3 days before due

3. 0 (day it is late)

4. 5 days late

5. 10 days late

6. Editor intervention

@acochran12733

Editor and Reviewer Performance Report Benefits: Accountability

�Editors and AEs know report will be sent out twice a year

�Some EICs shifted their focus on managing AEs better

�AEs started assigning alternate reviewers to keep things moving

�Some competition between some editors

�Reports are provided to oversight committees and used in consideration for editor awards

@acochran12733

Reductions in TAT

�Average time to first decision for all ASCE Journals reduced by 50 days since 2011

�Other reductions since the report:• 7 day reduction in average time for AEs to make a recommendation to

editor

• 3 day reduction in average time for reviewers to respond to invite

• 37 day reduction in time to first decision when looking at papers in full review (no ed rejects)

@acochran12733

5/14/2018

14

Lesson Learned

� Look beyond your canned reports

�Find data points to make your argument

�Share data to encourage peer pressure and competition for high performance

�Review your automatic reminders annually

�Use actual performance to push for efficiencies

@acochran12733

Implementing PACE

Jenny Hamlin

Content Technical Lead, Production

Public Library of Science (PLOS)

2018 CSE Annual MeetingNew Orleans, LA, May 8, 2018

How can we resolve common

figure errors more efficiently?

Editorial and Publishing Questions —Data-Informed Decisions

Background

• Most common issue in Production related to

graphics failures (~40%)

• Figures must meet PLOS’ quality and

formatting requirements before publication

• We had one in-house graphics specialist and

a large number of graphics issues

� led to a large backlog

• Lack of author resources in figure editing

5/14/2018

15

• Assist with the resolution of most common

figure issues and remove backlog

• Decrease of the number of image-related

tickets Apex had to create

• Shorten vendor turnaround times

• Keep development costs down

• Improve author experience

Goals

What is PACE?

• PACE is a ‘Preflight Analysis and Conversion

Engine’ image tool

• Allows authors to effortlessly transform images

to exact publisher specifications in minutes

• Resolves most common figure issues, provides

an issues/fix report, and offers PDF previews of

fixed figures

• Apex developed the tool for free; no cost for

PLOS or authors!

5/14/2018

16

File Requirements

These are the criteria that PACE accesses based off the

PLOS Figure Requirements:

Upload

• Accepts 9 common file types, incl. DOCX,

JPG, PDF, PNG, PPT, etc. Zip files also allowed.

• Converts figures to TIFF Format with a

downloadable report in minutes

• Easy to use drag/drop UI

• PACE keeps your originally uploaded files, Figure

File Quality Reports, and any PACE generated

figures for seven days

File Storage

10.1371/journal.pone.0196108

For each figure, PACE provides:

• Downloadable versions of original and edited figures

• Renamed figure files according to PLOS convention

• Lists changes made as well as errors that it can’t resolve

• A preview of the figure within a PLOS formatted PDF

5/14/2018

17

Example Report

PACE Limitations

• External system that authors have to be

directed to; not integrated into submission

system

• Not a one-stop solution for resolving all figure

issues

o Cannot edit content (typos, cropping, etc.)

o Vendors and in-house staff handle more

complicated issues

Workflow Changes

• Introduced to PLOS/vendors in Dec 2015; available to

authors in early 2016

• Trained in-house production staff to resolve graphics

issues

• Feb 2016: Included PACE instructions in outgoing

communication to authors

• Oct 2017: Allowed us to turn off a troublesome image

review tool in our submission system

5/14/2018

18

Users

• Over 55,600 users and more than 120,000 sessions

• Average of 2,100 new users per month in the last year

• Average session duration of 5:25

…and continuing to grow!

0

500

1000

1500

2000

2500

3000

3500

Jan

-16

Feb

-16

Ma

r-16

Ap

r-16

Ma

y-1

6

Jun

-16

Jul-16

Aug

-16

Sep

-16

Oc

t-16

No

v-1

6

De

c-1

6

Jan

-17

Feb

-17

Ma

r-17

Ap

r-17

Ma

y-1

7

Jun

-17

Jul-17

Aug

-17

Sep

-17

Oc

t-17

No

v-1

7

De

c-1

7

Jan

-18

Feb

-18

Ma

r-18

New Users (2016-2018 YTD)

Users by Location

• Users in 180 countries

• US has biggest user base followed by China

Results

• Reduced Salesforce cases by 50%, and JIRA tickets by 30%

• Saved ~$55K in resource/process costs from 2016 – 2017

• Helped reduce turnaround times to meet aggressive

publication schedules

0

1000

2000

3000

4000

5000

6000

7000

0

50

100

150

200

250

300

Co

un

t o

f PA

CE S

ess

ion

s

Effect of PACE on Number of Articles with

Image Quality Issues

Articles with Image Quality Source Discrepancies PACE Sessions

5/14/2018

19

Looking Forward

• Apex is continuing to develop PACE

o Process additional file types

o New figure cropping tool

o Cleaner UI

Questions?

[email protected]

Thank you!

0.CSE 2018 Questions combined€¦ · Cochran Associate Publisher and Journals Director •American...

Documents

Transcript of 0.CSE 2018 Questions combined€¦ · Cochran Associate Publisher and Journals Director •American...