The State of science, journals, peer-review,

thoughts on Open Science, reproducibility,

and Science 2.0?

Dr. Benjamin Laken

University of Oslo

@benlaken

We are now on the brink of an achievable aim: for all science literature to be online, for all of the data to be online and for the

two to be interoperable.

Source XKCD

Essentially, the volume of published work is constantly increasing.

A rough estimate of the rate of increase is …

Unsurprisingly, the number of PhDs is also increasing

Nature 472, 276-279 (2011) | doi:10.1038/472276a

Trends in PhDs granted per year for all disciplines,

values x103

The number of science doctorates earned each year grew by nearly 40% between 1998 and 2008, to

some 34,000, in countries that are members of the Organisation for

Economic Co-operation and Development (OECD). The growth

shows no sign of slowing

Systems Research and Behavioral Science Syst. Res (2014) Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/sres.2324

(Sta

ts h

ere

for U

SA b

iom

edic

al s

cien

ces)

However, there is no matching rise in permanent positions

This has created ‘The Postdoc Pile-Up’

Nature 520, 144–147 (09 April 2015) doi:10.1038/520144a

In essence a saturated Postdoc market. This means that postdocs who want to stay in academia are usually destined to travel from one place to another with no

long-term prospects. (One of the reasons you may hear postdocs referring to themselves as ‘hobos with a PhD’.)

In the context of the increasing pressures on individual scientists to compete—largely through publications—enters

the problem of Academic publishers as for-profit entities

The first journal, Proceedings of the Royal Society, was created by Henry Oldenburg (1619--1677), in 1665. The idea was to drive science forward through rapid

communication and sharing to a broad audience.

However, things have gone slightly astray over the years…

The role of journals: have they changed from a force of innovation to a hinderance?

The academic publishing business model explained by Nature’s Publishing Director

Apple: Exxon:

Profits (% of revenue) in 2011

Elsevier:

Springer:

John Wiley & Sons:

~25%~8.5%

36% (£724m)

33.9% (£866m)

42% ($106m)

Are journals overcharging?

Some opinions on academic journals…

Cushing, 2014, https://www.techdirt.com/articles/20140605/05175627467/academic-publisher-fights-publication-paper-criticizing-publishers-price-increases-profits.shtml

Elsveier a particularly bad example

Is a good solution to science communication simply traditional journals but as open-access?

Open access does not mean free, often the authors still pay high prices to publish their research.

The growth of Open Access (OA) journals in the last 10 years opened the door for the creation of hundreds of new Publishing company and new

journals, who engage in behaviour often best described as ‘predatory’…

They claim their articles are peer-reviewed, however independent research and many examples have repeatedly shown that the real barrier

to publication in their journals is simply the publication fee…

The Dark-side of the OA journal explosion

In 2005, following journal submission request emails, Peter Vamplew sent a 10 pg paper (with two figures) to International Journal of Advanced Computer Technology…

…they accepted the paper. An anonymous reviewer rated it as “excellent”, and the journal requested a fee of $150.

Tom Spears submitted a completely incoherent paper featuring a made-up study area (Nepean Desert)

Several famous examples of OA journal ‘stings’

Sent to 18 journals. 8 responded quickly, accepting, asking a $1k - 5$ fee.

More famous sting: John Bohannon

J Bohannon Science 2013;342:60-65

AbstractDozens of open-access journals targeted in an

elaborate Science sting accepted a spoof research article, raising questions about peer-review practices in much of the open-access world.

Peer review reviewed.Few journals did substantial review that identified the paper's flaws.

Another problem: the valuable reviews are usually wasted

It is not a question of if an article will be published, but where.

In this context, the expert reviews, community feedback, and reproducibility are the hallmarks of quality, and a real factor in

the value of the research.

We need to develop systems which focus on these aspects of the peer system, rather than see peer-review as the ‘behind

the scenes’ work.

Source: PhD comic

Explosion in OA journals over last decade (and rapid growth of predatory OA journals)

Published science doubles every ~9 yrs

New PhD’s awarded annually increasing (simultaneous decrease in TT and Permanent positions)

‘Publish or perish’, saturated PhD-level job market, and resulting competitiveness has combined with journal profiteering

…and if a paper gets a hard review, rejection usually means a paper will be submitted (unchanged) to new journal, until accepted

Many Journals = opaque review process, expensive, & behind the scenes politics/motives

End result (*at least for me) = scepticism, cynicism, frustration, time loss, high-blood pressure, and madnessPublication ≠ quality guarantee

Favour systems totally open (inc. review) and communal New paradigm: publication not the end of a funding cycle but the

start of a collaborative communal conversation

Summary

My fantasy model of Science 2.0

Data, code, paper: all in one open repository (like ArXiV)Version control system (like Github)

All researchers have unique ID profile (like RG/ORCID)

ID-tagged comments. Can be general, or specific: specific comments tagged to sections, as a thread, including up-

vote system (like StackOverflow) on paper and comments.Copying, and contributing (pull-requests) allowed

to create rapid-iterations and community refinement (like Github)

Vote-based score system, with scores contributing to different levels of community privileges (like StackOverflow).

Score a measure of a researchers overall contributions (Like RG), > than citation metrics [e.g. as bad papers often cited on purpose]

Needs shared super-computer-scale resources to re-run experiments (Like Google Earth Engine)

(This model already exists for the open software we use every day. The model can be used for research, and for lecture/course/textbook material too.)