The organization of traditional academic journals arose at a time when their most important task was to disseminate information, and when the printed word was expensive and therefore scarce. The electronic word is no longer expensive, but readers' time still is, and there remains some justification for imposing conciseness in articles. But disseminating articles can now be done for free, and subscription-based journals now hinder the flow of information using paywalls and legal threats, rather than helping it. Not to mention the very useful information which they never disclose -- the reviewers' reports.
There are many ideas on how to improve scientific communication. Articles themselves might be replaced by wikis or something else. Keeping articles as the primary means of communication, the natural idea is to build, as Gowers puts it,
"a cross between the arXiv, a social networking site, Amazon book reviews, and Mathoverflow". However, it is difficult to start a new system from scratch.
Any new system must be able to compete with the established journals, which not only are entrenched in the minds of academics, but also hold sway over their careers. Being able to compete limits the radicality of possible innovations: the new system must be officially recognized and indexed as a journal, and must play the game of bibliometrics. Of course, it is possible to solve each problem in turn, and to start winning over academics with services such as social networking, before trying to be recognized as a journal.
So, which innovations have been or are being tested?
The simplest idea is to start a traditional journal using modern tools, thereby reducing costs to a minimum. One can use free journal-management software. One can even rely on the arXiv for hosting and distributing the articles, and do only the selecting and reviewing: this is the idea of the overlay journals.
A more ambitious proposition is to start journals with advanced features such as open peer review or a new economic model. This was done by eLife, and PeerJ. These examples show that, when given the choice, most authors will make reviewers' reports public. Moreover, PeerJ has introduced a new economic model where authors pay of the order of 100 euros for a lifetime membership -- negligible costs when compared with the thousands of euros per article of typical Gold open access publishers.
Finally, the more radical idea of allowing the public to comment on articles, instead of picking one or more reviewers, is being tested by the Selected Papers Network and Publons. For the time being, these systems are not supposed to replace journals, but they can be used to publicly discuss articles. The community would certainly benefit if email exchanges between readers and authors, as well as reviewers' reports, were thus made public.