All entries for Friday 08 June 2012

June 08, 2012

Unpublished papers: what happens to them? How do we tell a quality paper?

Writing about web page

Researchers are discussing some interesting topics on ResearchGate. I have linked to a discussion on what happens to papers that they don't manage to get published. I like the answer that they keep refining papers until they are good enough to get accepted for publication eventually. Academic authors need to be persistent and resilient, and published content needs to be of high quality!

However, there is a hierarchy of publication quality out there, and as well as improving their articles, authors can approach less selective or rigorous journals, if their aim is simply to get their work published and out in the public domain. The discussion on ResearchGate was started by someone who has established a website for "Unpublished Articles In Science" ( It's a new site and I didn't find any content. I am also not sure if they want any kind of article or only review articles, and they are asking for donations so I will wait and see if it takes off in any way.

This is not a new idea, to provide a "mop up" place online for academic work that would otherwise not be discoverable. Some institutional repositories were set up as places to make all kinds of research outputs available, including unpublished work. I guess that some authors might just get frustrated enough that their work cannot be published in any other way and put their article(s) somewhere like that, but I wonder what the value is in doing this? Perhaps some really important scholarly works are being missed by the world, but perhaps some works are just not of high enough quality and so should not be publicly available: they might even be misleading.

Surely the peer review and editorial processes of journals exist for a reason? If the work is not of high enough quality to be published, then should it really be in the public domain? Would it not damage an author's career, to have lesser quality work attributed to him/her in such a public way? It is possible that a particular piece of research is just too different for there to be an appropriate journal or publishing outlet, even though it is of high quality and importance. The "differentness" of the output could be that it is in an unusual format or that the subject is highly unusual, and the researcher might be glad of a place that simply makes the work publicly available.

Institutional repositories which accept any kind of output, whether published or not, rely on their academics' judgement about what is a good output. In a way, there is a quality filter of some sort because the author must be employed by the institution, so there is some likelihood that s/he will be able to select what should and should not be in the public domain and associated with his/her name. Some institutions even introduce internal peer review in some way, for unpublished outputs.

At the same time, publishers are introducing new journal titles which appear a little less selective. I am thinking of their author-pays open access (OA) titles, since authors' letters of rejection from their journal of choice sometimes include a suggestion that the article could now be submitted to the publisher's open access journal. The concept of the author paying a publication fee has always been an argument against the open access model of publication, because it interferes with the quality filters in the existing reader-pays model. Of course, the article could still be rejected from the OA journal and there is no reason why peer review could not operate just as rigorously in OA journals as in any other journals.

New, online journals or article collections are not bound by the same format and issue restrictions as traditional journals, and that does allow them to accept more "different" content. The selectiveness or otherwise of an online journal need not be dictated by the amount of print space and paper, but it can use selective criteria based on quality alone. That sounds like a "good thing"!

OA journals can still be of very high quality, as evidenced by the PLoS journals' high impact factor. (I have blogged about these kinds of publication before, and of course I know that impact factors are not infallible measures of quality.) PLoS ONE is the largest, most inclusive journal from PLoS and it has some really interesting features and filters, to enable people to discover high quality research outputs.

With online journals and collections of academic outputs, as with all online materials, the reader needs to be more aware than ever of the features that indicate the quality of the work s/he is reading. Hopefully, the reader will read widely and so be aware of the field, when reading academic content, but beyond the readers' own expertise and academic insight, websites and online journals have features to help readers to assess quality.

Here is a little list of clues on academic quality, including traditional as well as new online features:

  1. place of publication: eg journal title, or special collection within a wider collection like PLoS ONE
  2. information about the authors: institution employed at, membership of organisations, etc
  3. information about who funded the work - they value it, but you might also want to ask why.
  4. when the work was published: this might be recent or it might be before a defining discovery altered academic understanding, so dates are important clues!
  5. whose work is referenced and acknowledged, and therefore this work builds upon
  6. news & media coverage
  7. reviews or comments by other readers, either on the collection site or on readers' blogs.
  8. ratings/scores by other readers
  9. tweets about the article (or other social network discussions)
  10. number of "likes" or bookmarks by other readers
  11. number of views or downloads of an article
  12. citations of the article by others (NB citation sources might matter, i.e. who is citing the work)

Publishers can help readers to access these clues, and providing trackback URLs and ways for readers to bookmark articles in their tool of choice, in a way that the publisher can monitor and publicise are important contributions that a publisher can make. And bullet points 7 onwards in my list are numerical scores, and as such should be taken in context. What is a high score for an article in one discipline might seem low in another discipline: publishers could also provide that context, if they want to help readers to appreciate the quality of the work that they are publishing.

Sites like the UNAIS, institutional repositories and even authors' own sites need to be as good as publishers at providing clues as to quality, if not better, since their quality filters are less well known and understood. And readers do need to be aware.

Subscribe to this blog by e-mail

Enter your email address:

Delivered by FeedBurner

Find out more...

My recently bookmarked sites

Tweet tweet

Search this blog

Most recent comments

  • Oh yes, I'm writing that too! And tidying up my paperwork, plastering each piece with post–it notes … by Jenny Delasalle on this entry
  • A useful list, thanks Jen. I would add "it's never too early to start writing your handover document… by Emma Cragg on this entry
  • Yes, Google does find things very fast: I use it a lot to find sites that I know and regularly visit… by Jenny Delasalle on this entry
  • Mac OS has the ability to share Safari www bookmarks and other data, securely across multiple machin… by Andrew Marsh on this entry
  • Hi Peter, I see that you practice what you preach… and indeed the point that you make about being … by Jenny Delasalle on this entry

Blog archive

Not signed in
Sign in

Powered by BlogBuilder