All entries for Thursday 07 November 2024

November 07, 2024

Discussing Reviewer Suitability & Citation Justice with Early Stage Scholars

The first training session of the new academic session brought discussions around reviewer quality.

This week I had the pleasure of contributing to Harriet Richmond’s early-stage researcher session focussing on publication. As blog readers will be aware, I’ve been feeding into these sessions for almost three years now, around once a term generally. As always it was a pleasure to collaborate with Harriet, who is a great teacher and all round good-egg. Certainly I always come away with some thoughts on improving my own teaching, or new piece of information to ruminate upon, so being involved is always a win-win for me.

I also enjoy participating because these workshop sessions are also a wonderful opportunity to share some of my personal experiences as an editor and reviewer with researchers who are a little earlier in their career track than I. The hope is – like this week’s cohort – that it sparks some conversation and perhaps demystifies the area a little. I do often feel it’s a shame that in the post-pandemic period we’ve kept the workshops online only, which perhaps slightly curtails a more free-flowing exchange of thoughts and ideas. That might be a personal preference for ‘live’ performance in my teaching, and I can see why the delegates enjoy not having to schlep into campus for the sessions: less time away from the lab and office probably pays them dividends in a busy schedule. No matter, as always, this week’s exchange online was still good enough to pick up on a few new threads of conversation.

This time around I introduced ideas of citation justice, as discussed in our recent podcast, into the debate. This arose out of a delegate discussion about questionable citation habits and bias within them. It was handy to be able to talk a little about these concepts and highlight some of the ideas around introducing a ‘beneficial bias’: although, I should note that for now Exchanges has no hard and fast policy in this area. All the same, I’ll be looking forward to reading the Reinvention report, and perhaps kicking off some policy discussions with my Board as a result. Getting better and broader representation within article citations can only benefit readers, researchers and authors alike in my opinion.

We also got into an interesting debate over journals/editors and their ability to pick ‘quality’ reviewers. Now, if you’re an editor you probably don’t need me to tell you how challenging it can be, with the hundreds of thousands [1] of manuscripts annually in need of multiple reviewers, especially with academics’ time increasingly stretched across so many research, teaching and administration priorities. With this initial hill to climb just to get >any< reviewers on board, conceptualisations of also judging their quality beyond a simple visit to their personal/professional website [2] sounds like an even bigger slope to ascend. As journal editors we offer lots of guidance to reviewers on how to do a quality job, as well as advising reviewers about those times when they feel they might be out-of-scope for a particular paper [3]. Truth be told though, much of the responsibility for invalidating a reviewer does seem to fall on the prospective reviewers’ shoulders themselves. It’s a bit of a sector norm, and perhaps one which deserves some greater consideration – I’ve certainly been inspired to look around and see what other journals are doing. Perhaps a small debate among my editorial team is also worthwhile following up on these conversations too – we do have a Board meeting scheduled for next month after all!

This week’s session also saw the second running of my ‘you are the reviewer!’ case study challenge, wherein the delegates had to make a judgement on several not-so-hypothetical reviewing problems. Once again the delegates rose to the challenge well, and it was good to see that the approach we had adopted in each case – the examples being anonymised but real ones – seemed to chime with the views of the scholars present. I might tweak the examples for the next running of the course in the new year though, just to avoid things becoming too staid or stale.

So, a good session from my perspective, and I hope from the delegates too. I think my next task is to take that annual look at my teaching notes and think about how my contribution could be tweaked or improved. I certainly got the feeling this time, that another refresh of the slides and accompanying interactions could do with another polish!

---

Endnotes

[1] Almost certainly more when you consider how many articles are rejected annually. With a ~50% decline rate for Exchanges the 68 articles we published this years therefore equates to ~140 submissions as a back of the envelope calculation. And we’re a relatively small journal with a small ingest of article submissions!

[2] Or of course knowing them in person.

[3] Doubly tricky for interdisciplinary papers where few, if anyone other than the author, will be entirely conversant with the paper’s themes, methodologies and methods.