Reviewer Roundup 2022 – Getting a better dataset

Writing about web page https://exchanges.warwick.ac.uk/index.php/exchanges/FAQ#reviewers

The editor talks about one of those tasks that invariably ends up being tackled during the summer – data clean up!

Some time ago, I went through the database of all our registered reviewers for Exchanges. You see, one key bit of information we ask of each reviewer is to provide is a few keywords covering their areas of expertise as part of their profile creation. In Open Journal System (OJS) parlance this is known as their reviewing interests. Reviewers can update, revise and amend these at any time by logging on to their profile, although I suspect in practice few actively do this unless promoted [1]. When my editors and myself come to look for suitable reviewers to examine a newly submitted manuscript, one of the first things we do is consult this reviewer database. In this way we hope to find someone with matching or closely related reviewing interests; although given the variety and variance in topics we tackle on the journal more often than not we have to do

Sadly, due to the way OJS operates it is entirely possible to create a new reviewer profile without this information being completed. It’s a technical oversight I’ve long hoped might be fixed, but for now it means over time as our database of reviewers swells, some will be running with incomplete profiles. Technically functional, but lacking that crucial interest information we need! Even more frustratingly as editor, there’s no easy way for me to press a couple of buttons and locate all the reviewing accounts which lack this information [2]. Which is why once again I’m deeply grateful to resident ITS OJS Guru Rob T for managing to capture it for me.

As last time there were quite a lot of accounts lacking this information, albeit not quite so many. I’m hoping that represents the improvement in the dataset from when I last ran this exercise. So, as of a few minutes ago I’ve written to all of these reviewers asking if they could take a couple of minutes to provide this missing, vital information. It’ll not only make our lives easier, but will also help ensure we’re more likely to ask reviewers to consider papers which correspond closely to their research and professional expertise.

There’s also been a couple of knock-on benefits from this process, alongside hopefully a better reviewer dataset [3]. Firstly, in offering instructions on how to update this information in the mailshot, I noticed the guide we provide on our FAQ was slightly outdated by the most recent system update. So, that’s now be rectified and clarified. Secondly, ever since I ran the mail merge to send out the messages my laptop has kept pinging every 30 seconds for quite a while. This is mostly bounce back messages from dead, defunct and otherwise formerly functional email accounts. Which means one of my follow up tasks will be to go through these ‘dead’ accounts and inactivate them as reviewers, so we don’t keep fruitlessly messaging them [4].

The end result though – hopefully – a tranche of improved reviewer data, some elderly accounts pruned and a better working experience for everyone involved! I can see I might try and make this an annual event at the start of each summer! Check back in June/July 2023 to see if I do…

---

[1] I could be wildly wrong of course; this is only an assumption. But I know personally how rarely I update profile information about myself on any system unless something or someone prompts me to do so.

[2] The management information on OJS HAS improved in leaps and bounds, but it is still years behind where it should be. This isn’t a problem for us to resolve easily, as it relies on the open-source developer community to recognise that editors using OJS, NEED a whole lot more, better and more intuitive ways to query the data held by the system.

[3] I suspect I’ll also be dealing with a smattering of emails from academics asking me to delete/deactivate their accounts (it happened last time) too. Not sure this classes as a benefit though.

[4] An additional frustration with OJS is it doesn’t inform me when I send system emails which accounts are bouncing. I believe the WUP chief and tech sees these messages, but I’ve never been able to. Hence, this is one of the few times when I can really discover which OJS profiles on Exchanges are now effectively defunct.

Gareth Johnson : 30 Jun 2022 14:21 |  Tags: Administration Cleanup Data Database ITS Review Reviewers |  Comments (1) |  Report a problem