A strong editorial college
The most distinctive feature of SciPost is its editorial college, made of well-known theoretical physicists. These people do not just lend their names to the project. Given how SciPost functions, they have a lot of work:- Approve registration of new contributors.
- Approve requests for new Commentary pages. Such requests need to be sent by contributors in order to be able to comment on arXiv or journal articles.
- Approve comments themselves.
- Oversee the refereeing of submitted articles, according to a quite complicated procedure.
An adequate platform
Having tested the website by looking around and writing a small comment, I have found that it works well. My registration, request for a Commentary page, and comment, were approved in less than 24 hours, sometimes much less. Latex formulas are allowed. The search function works as expected.While this is a good start, I hope that the platform’s capabilities will improve in due time:
- A potentially crucial point would be to have Commentaries linked from arXiv using trackbacks.
- Contributors who comment on an article could have the option to inform the authors by an automatically written email. And readers could have the option to follow a number of articles and contributors.
- Different versions of the same article are apparently treated as unrelated objects: it would be nicer to have one Commentary for all versions, with the possibility to indicate which comments apply to which version(s).
- More generally, SciPost could do less categorizing, and more tagging. There need not be essential distinctions between different versions of the same article, between Reports, Replies and Comments on an article, between authors and other contributors. These distinctions could be indicated by tags, rather than woven into the structure of the platform.
A new flavour of open peer review
SciPost’s organization of open peer review is disctinctive for- the strong involvement of editors,
- the principle that reports are publicly viewable, while reviewers can remain anonymous,
- the deletion of reports when articles are rejected.
Ultimately, one could even eliminate the notions of accepted and rejected articles. While not being that radical, SciPost takes steps in this direction by having reviewers rate articles in a more nuanced way than just accepting or rejecting. Reviewers are supposed to rate the validity, significance, originality and clarity of articles, with six possible grades (from “poor” to “top”) in each case. (The originality rating might be a bit subjective.) It would be good if such ratings were available to all commentators, and not just reviewers.
The problem of attracting contributors
The main difficulty for any initiative like SciPost is to attract enough contributors, and to become a standard tool for a large enough community. With its editorial college, focus on quality, and intitutional backing, SciPost has a chance of attracting quality-conscious contributors.However this comes at the expense of ease of use. New contributors wanting to post a comment should not have to wait for three successive approvals from moderators (for registering, opening a Commentary page, and posting the comment itself). Wikipedia and StackExchange have shown that allowing new contributors to bring content online immediately is not necessarily detrimental to quality control. In the case of SciPost, weak contributions need not be blocked: what matters is that strong contributions are recognized, displayed more prominently, and easy to find.
Many thanks for the observations, comments and suggestions.
ReplyDeleteA few clarifications:
SciPost also goes much beyond commenting on papers (both preprints on the arXiv and papers published anywhere at any time). The initiative also aims at starting a set of high-quality journals, with a very robust open form of refereeing.
SciPost is also by no means limited to theoretical physics. Fellows of the Editorial College will be recruited from all fields of physics (we are currently recruiting). They will carry the decisional power on publishing. The work of Fellows is in fact relatively limited and not very time consuming, and focuses on the processing of submissions (inviting referees, and formulating an editorial recomendation at the end of a refereeing round). Vetting of new contributors, of requests for commentary pages, of comments etc. is a one-click affair. This vetting part is indeed a means of ensuring high-quality content, and to avoid blog-style degeneration.
Let me address your suggestions:
- arXiv trackback:
arXiv has been contacted about this, but no answer has yet been received. We will strive for seamless linking with it, and hopefully we will hear from them soon.
- auto emails:
This is currently being implemented, with opt-in automatic email facilities for all forms of delivered content.
- versions:
Indeed a Commentary pertains to a given version (this applies only to preprints of course). If other versions of the same preprint are discussed, a link to the other versions appears in the top part of the page.
- categorizing/tagging:
The reason why different version of preprints are kept separate is that reports and comments can lead to substantial modifications (improvements hopefully!) in the manuscript from one version to the other. Keeping things compartmentalized thus helps visualize the `history’ more easily. Reports are also expected to be more substantial than comments, and are displayed more prominently. Author replies are also `attached’ to the relevant object in order to give prominence to author responses and make the flow while reading more linear. The idea of using more tags is a good one, which we will consider.
On open peer review, more precisely the principle of removing material associated to a rejected paper: note that this remains the authors’ discretion: if the authors want the reports, comments and replies to be kept online or forwarded on to a subsequent journal, then SciPost will do so. The default of removing content is as a sign of respect for the authors: open refereeing is quite stringent, and can lead to very substantial rewriting. Authors might feel that any subsequent version of their paper is sufficiently `divorced’ from the previous that older critical material might affect objective evaluation of the new version. In any case, SciPost is positioned very much on the side of openness: if the community prefers to have material always remain online, then we will do so.
The problem of attracting contributors is indeed the greatest challenge in the beginning phase. Most critically, attracting submissions for the journals to help establish them is high priority. Members of the community who share the values of SciPost and who are willing and able to give it a try, should!
Ease of use will be improved. Note that registration is a one-off thing which also needs to occur on other platforms (remember that only active, professional scientists are allowed to be contributors on SciPost; this requires a verification step). One-click opening of Commentary Pages is a good suggestion which will be implemented. We have implemented the vetting procedure for all content, in order to prevent any possibility of abuse of the system, but perhaps this is indeed not needed. Time will tell.
Thanks for looking into it!
J-S Caux
This comment has been removed by a blog administrator.
ReplyDelete