Open peer review!
The main reason I was attracted to SciPost in the first place is that it practises open peer review, which means that the referee reports are publicly viewable. (The referees can choose to remain anonymous.) If one wants to improve the communication of research results, publishing referee reports is the obvious first step, as it requires no extra work, and has potentially large benefits on the quality of the process. Actually, publishing reports on a rejected article can even save some work if the article is later submitted elsewhere. (SciPost however erases reports on rejected articles.)
In closed peer review, the only indication of quality is the reputation of the journal. Using the reputation (or the impact factor) of a journal as a proxy for the quality of an article is however stupid. In contrast, open peer review guarantees the quality of the peer review process at the level of the article. Readers should prefer open peer reviewed articles. Therefore, authors should prefer journals that do open peer review.
In the case of SciPost, the referees not only write reports, but also give some qualitative grades in six domains: validity, significance, originality, clarity, formatting, grammar. While this is in principle a good idea, I do not see why each referee should comment on each aspect on the article. For example, I could complain that the first referee wrote a narrowly technical report, and has no understanding of the significance and originality of the work. Fortunately, readers can see that for themselves.
SciPost’s procedure is not for the faint-hearted: I and my coauthors were a bit shocked by the first referee report, which came without comments from the editor, and without indication whether there would be other reports. Fortunately there was a second report, and it was more insightful.
On the use of arXiv
According to the procedure, articles are submitted to SciPost from arXiv. While this is good and normal in the case of the initial submission, this also applies to subsequent revisions. Posting each revision on arXiv sounds strange, because in principle there could be many revisions until the article is published. But arXiv is not designed to host intermediate steps in a peer review process. ArXiv is designed to communicate articles to readers, and having too many versions can confuse those readers. Actually, arXiv has technical features that make it awkward to have many versions: there is a delay of about one day between the submission and the appearance of each version, and only the first four revisions of an article are announced in the daily “Replacemements” list.
While SciPost requires the posting on arXiv of intermediate versions, but not of the published version, the publishing platform Episciences does just the opposite. However, authors who are not fully happy with the published version, should not have to inflict it to arXiv readers. Journals should respect arXiv’s role in scientific communication, and avoid treating it as a technical platform.
SciPost has its own format for articles, and takes great care of references. In practice, I found that the amount of required formatting work was reasonable. In principle however, I do not see why journals should apply their own cosmetics to the articles they publish. Moreover, having technically perfect references is useful not to readers, but to bibliometric robots, whose work needs maybe not be encouraged. So far, SciPost has missed the opportunity to free itself from needless formatting rules.
The platform: possible improvements
The platform for submitting and tracking articles works well, but could be improved on a number of points:
- It would be good to know in advance what the submission form looks like, without having to actually fill it. In particular, SciPost’s subject classes differ from arXiv’s, and should be made explicit somewhere.
- I would greatly appreciate if all the data that I enter into the platform were accessible to me from my account. This would include submission forms, comments, and correspondence with editors.
- It would be good to be able to preview texts such as comments on articles or replies to referee reports. Previewing before sending is all the more relevant, as these texts can contain Latex formulas. Being able to save them and send them later would also be good.