Amidst a flurry of retractions of research articles from high level journals and growing concerns about the non-reproducibility of research findings, the time-honored (some say old-fashioned) closed pre-publication mode of peer review has come under critique. Major issues concern the quality of reviews and lack of transparency. A number of modifications and alternative models have been proposed (e.g. Front. Comput. Neurosci. 6:94;2012), including open post-publication review. Most publications are no longer read in printed and bound issues of a journal, but rather accessed in digital form via the internet. This allows for novel forms of readership participation, such as post-publication review and online commenting and discussion of articles. Several journals are experimenting with such novel features (e.g. PLOS One), and some are based on it (e.g. F1000Research or eLife). Nevertheless, most established journals are hesitant to give up their time honored modes of publishing. They argue that closed pre-publication review may not be perfect, but that the alternatives are untested, and may actually be worse. Post-publication commenting requires software upgrades to journal websites, as well as monitoring and moderation of content, and there may be legal issues. Another problem relates to the troubling fact that a substantial fraction of the biomedical literature is not read at all (even if cited!), which means that we may not be able to solely rely on processes that take place after publication.

An alternative to changes at the level of individual journals is offered by websites such as PubPeer , and since 2013 by PubMed. PubPeer allows scientists to comment on any published article that has a DOI. Registration is necessary, but identities are not verified, enabling anonymous posts. An anonymous post on PubPeer has recently set in motion a process which ultimately lead to the highly publicized retraction of two articles in Nature  on so called STAP (Stimulus-Triggered Acquisition of Pluripotency) cells (Nature 501:4;2014).  PubMed Commons provides a platform for all comments to Pubmed listed articles. To comment on an article one needs to be an author of a PubMed listed article (email addresses are verified), registered users (‘members’) can invite other researchers to comment. Comments can be searched and cited. Both platforms are not moderated for content or quality, but inappropriate comments may be deleted. PubMed Commons also addresses a problem which may severely affect the usefulness and acceptance of scientific discourse via this format: Fragmentation of comments on one article over many different sites. In PubMed articles with comments are tagged in PubMed searches, and are therefore directly visible and accessible to readers who search the literature via this univeral portal. Contrarily, comments on PubPeer and similar services  must be either proactively searched by visiting their website, or additional programs (e.g. PubChase) need to be installed or integrated into the webbrowser. Journals that allow commenting articles on their websites (e.g. PLOS One, F1000Research) link article and comment, but PubMed does not display that a comment has been made. All this inevitably leads to a fragmentation of comments and discourse.

At present, there are only a few thousand entries on PubMed Commons, on more than 23 million PubMed indexed articles. However, those numbers will rapidly increase. PubMed, a search engine accessing primarily the MEDLINE database of the The United States National Library of Medicine (NLM) at the National Institutes of Health (NIH), has revolutionized access to the medical literature and is still the kingpin around which biomedical publishing revolves. The trust of the scientific community into PubMed, as well as its ease of use, stability, and the fact that it verifies commenters makes PubMed a natural choice for post-publication commenting. This will create pressure on journals to tag and link articles on their websites which have received PubMed commons.

Post-publication commenting can do much more for science than proxy for pre or-post-publication review, as in alerting readers to potential flaws which have been overlooked in the review process. Post-publication commenting may open up a new avenue of scientific discourse, in which readers and authors exchange arguments almost like at a scientific meeting. Whether this also eventually will lead to more reliable publications is unclear at the moment. At present, the future of post-publication commenting and other forms of readership participation in scientific publishing to a large extent depends on whether and how the scientifice community will utilize these novel tools. The future is in our hands, lets shape it!

PS: Here is an interesting and prescient reflection on commenting and post-publication reviewing in Michael Nielsen’s blog post on the future of science (2008!)

‘It seems that people want to read reviews of scientific papers, but not write them.

The problem all these sites have is that while thoughtful commentary on scientific papers is certainly useful for other scientists, there are few incentives for people to write such comments. Why write a comment when you could be doing something more “useful”, like writing a paper or a grant? Furthermore, if you publicly criticize someone’s paper, there’s a chance that that person may be an anonymous referee in a position to scuttle your next paper or grant application.

[…]

The contrast between the science comment sites and the success of the amazon.com reviews is stark. To pick just one example, you’ll find approximately 1500 reviews of Pokemon products at amazon.com, more than the total number of reviews on all the scientific comment sites I described above. The disincentives facing scientists have led to a ludicrous situation where popular culture is open enough that people feel comfortable writing Pokemon reviews, yet scientific culture is so closed that people will not publicly share their opinions of scientific papers. Some people find this contrast curious or amusing; I believe it signifies something seriously amiss with science, something we need to understand and change.’