SciPost two years on

There are two reasons why we should care about the journal SciPost Physics: it practices open peer review, by publishing the exchanges between authors and reviewers, and it is free to authors and readers, at a time when academia is struggling to escape the stranglehold of predatory publishers such as Elsevier.

Two years after it was launched, SciPost Physics is alive on well, with 155 published articles at the time of this writing. The journal has managed to attract articles of high quality, some of them by well-known physicists such as Cardy, Verlinde, Seiberg, or Rychkov. The main challenges are now to attract many more authors, and sustainable funding. The journal is funded by institutional subsidies, an economic model that works for arXiv but that is rare for journals. If the finances worked out, the journal could ultimately hope to become a megajournal, and publish thousands of articles a year like PLOS One or PeerJ. For the moment, the stated aim of the “Expansion and Sustainability Drive” is to publish 500 articles a year at a cost of 200.000 euros. So each published article should cost 400 euros, a very reasonable price for a lean electronic journal: an order of magnitude more than arXiv, an order of magnitude less than Elsevier. (Unlike Elsevier, SciPost does not pay executives, lobbyists, shareholders, salespersons, lawyers, etc.)

In this post I will comment on SciPost’s workflow and platform, updating my earlier assessment in light of two years’ worth of experience. The aims are to help authors decide whether this journal suits them, to determine which features of SciPost should be emulated (or avoided) by other journals, and to provide feedback to SciPost itself.

Is SciPost only a journal?

 

SciPost’s stated ambition is to be not just a journal, but a portal where articles are publicly discussed. It is possible to comment on any arXiv preprint or journal article, whether it was submitted to SciPost Physics or not. However, very few people take advantage of this: in 2018 there were only 2 such Commentaries so far. Two reasons might plausibly explain why:

• There is no obvious way for readers of an article to discover that there is a SciPost Commentary on that article. While a post on this blog that mentions an arXiv preprint is linked back from arXiv, a SciPost commentary on that preprint is not.
• Commentaries are systematically moderated by SciPost, so it takes time and effort to have them appear online.

In order for commentaries to have a chance of becoming popular, they would need to be linked from arXiv. And moderation should be done a posteriori, not a priori, or even replaced with a StackExchange-like quality control mechanism. Also, the author of a commentary should be allowed to easily amend it.

 

Reviewers: anonymous or not?

SciPost reviewers have the choice of remaining anonymous, or publicly disclosing their names alongside their reports. However, quickly browsing the latest 20 published articles, I find that only about 10% of reports are signed. This is not surprising, as writing anonymous reports is business as usual for most researchers. However, PeerJ manages to do a much better job at inciting reviewers to disclose their identities, and 40% of PeerJ reports are signed. SciPost could take the very simple step of having reports signed by default, rather than anonymous by default. This could come with a message explaining why signing reports is better in most cases.

 

The reviewer’s tale

I have recently experienced SciPost as a reviewer. A few comments on the substance of the process:

• SciPost requires that I give marks to the article in no fewer that 6 areas: Validity, Significance, Originality, Clarity, Formatting, Grammar. And in each area, there is a wide scale of possible marks, with distinctions that are not clear. (What is the difference between ’High’ and ’Top’?) But I may not have an opinion on all of the artice’s aspects, in particular when it comes to the highly subjective areas of Significance and Originality. Being unable to give meaningful marks, I have given the highest mark in each area. At the very least, giving these marks should be optional, not mandatory. Of course, it would be preferable to have a more meaningful rating system. And SciPost’s marks are not exploited: I cannot search for articles with top Originality and high Clarity, for example.
• As a reviewer, I have to assess my own qualifications: expert, very knowledgeable, knowledgeable, generally qualified, or not qualified. This assessment is not publicly displayed. I do not see the point: the editor should know how qualified I am.
• As a reviewer, I have to say whether the article should be published, and in which Tier: Tier I (top 10% in this journal), Tier II (top 50%), or Tier III. But I should have the option of giving my opinion on technical matters only, and leaving editorial judgements to editors.
• Reports by invited reviewers have to be vetted before they appear online, and then they cannot be amended by their authors. More flexibility would be welcome.

Technical comments on the platform:

• It is good that I can save my unfinished report, and not have to to complete it in one go. This would be even better if I could save the report without having filled all mandatory fields.
• It is good that there is a preview of the report, but the preview curiously does not have line breaks. Also, it is not clear which fields will be shown to the editor only, and which fields will be publicly displayed. It would be better for the preview to accurately reflect what will be publicly displayed. On the other hand, it is not really necessary for the preview to appear side-by-side with the report.
• When entering reports, why not allow Latex for enumerations and other formatting, in addition to formulas? Or allow me to enter a report in some standard format, after conversion from Latex using Pandoc.

 

Miscellaneous suggestions

 

• Better display the history of each submitted article: not just its current status, but the past and future steps in the process, with the dates of the past steps.
• Have a Contact email address or web interface for technical queries, and quickly answer such queries. At the moment, the web interface allows the authors and reviewers to write to the Editor in charge only.
• Make it easy to print the reviewer reports that appear on the website, possibly by providing PDF versions.

One more suggestion (update on October 23)

When asking potential reviewers to accept or decline an invitation to review, SciPost could emulate JHEP in the following ways:

• Declining the invitation could be done via a pre-written email to the editor, where the decliner could suggest alternative reviewers, and explain his reasons for declining.
• When accepting the invitation, the reviewer could be asked to declare a self-imposed deadline. (No more than 30 days at JHEP.) The reviewer could then receive an automatic reminder email when approaching the deadline. If the report is not submitted at the deadline, the reviewer could be asked to declare a new deadline by another automatic email.