All 4 entries tagged Benchmarking

View all 10 entries tagged Benchmarking on Warwick Blogs | View entries tagged Benchmarking at Technorati | There are no images tagged Benchmarking on this blog

July 27, 2011

WRAP reaches 5000th item!!

Writing about web page

Following the announcement in February that we had reached 4000 items WRAP’s growth continues to be impressive and is now supported by the development of the University of Warwick Publications service. Visitors are coming from more than a 160 different countries every month and in June 2011 WRAP items were downloaded more than 21,000 times.

Today we announce that WRAP’s 5000th item is:

Mercer, Justine (2009) Junior academic-manager in higher education : an untold story? International Journal of Educational Management, Vol.23 (No.4). pp. 348-359. ISSN 0951-354X

Authors are encouraged to submit their journal articles to WRAP online at:

Visit WRAP:

Find out more about WRAP:

February 02, 2011

WRAP Reaches 4000 Items!

Writing about web page

The Warwick Research Archive Portal (WRAP) aims to provide worldwide access to the outputs of Warwick researchers to raise the profile of the high quality research being undertaken at the University.  Our collection of journal articles and PhD theses has been growing rapidly over the past twelve months and we have just made available the 4000th item in the database.

WRAP has doubled in size in just over a year and follows the news in October that WRAP was starting to see more than a 1000 visitors each weekday in the autumn term.  Visitors are coming from more than a 150 different countries every month and mostly find content through Google.

WRAP’s 4000th item is:

Bruijnincx, P.C.A. and Sadler, P.J. (2009). Controlling platinum, ruthenium, and osmium reactivity for anticancer drug design. Advances in Inorganic Chemistry, 61, pp. 1-62. ISSN: 0898-8838

Authors are encouraged to submit their journal articles to WRAP online at:

Visit WRAP:

Find out more about WRAP:

November 08, 2010

WRAP Book Club: Entry 1

Writing about web page

This entry is the first in an (hopefully long) series of entries on interesting articles and books in and around the institutional repository sector.  I'm starting with an area that I am still struggleing with as a Repository Manager, that of benchmarking.  I was asked in a recent interview with our Knowledge Centre where WRAP was in terms of the 'repository scene' and I couldn't really give her an answer that I was happy with, added to the fact that benchmarking is something that I have looked at again and again in the past few months meant that the article I found in the recent issue of Liber Quarterly was particularly timely:

  • Cassella, Maria (2010). Institutional repositories : an internal and external perspective on the value of IRs for researchers' communities. Liber Quarterly, 20(2), p. 210-225.

A perennial problem for repository managers if that as all repositories are slightly different, it what they collect, how they are administered and what they do with their content once they have it.  This makes it very difficult to hold a repository up against another repository and work out exactly how to rank one above the other in terms of performance.  So far the best that can be done is locate a single repository on a sliding scale of similarity and to benchmark it against itself.  Maria Cassella's article introduces a set of fourteen internal indicators and a further three external indicators to try and formalise the process of repository benchmarking.  These indicators are based on and rooted in Kaplan and Nation's 'Balenced Scorecard' (BSC) methodology which is already used in various measures of Library performance.

The measures proposed by Cassella fall into four broad categories:

  • The Customer Perspective;
  • The Internal Perspective;
  • The Financial Perspective and
  • The Innovation and Learning Perspective.

These performance indicators are intended to allow repository manages to "align their repository strategies with the institutional mission and goals and to identify priorities in performance measurement" (Cassella, p.214).  Many of the PI's that are suggested are things that, I'm sure, many repository managers are already recording in terms of who is depositing into the repository at what rate and the levels of external use that these items are getting.  Some of the PI's suggested are very interesting, for example she suggests that one things that should be measured is the number of 'value-added' services offered is used as a measure, which fits in well with the prevailing trends in the repository world, where the 'value-added' is being increasingly seen as if not more important then at least as important as the core functions of visibility and preservation.  Two of the 'financial' indicators I found very interesting, recently the 'cost per deposit' has become a focus of much discussion but Cassella goes one step further and suggests that another measure that would be interesting to keep is the 'cost per download'.  This she states will allow repository managers to "evaluate the scholarly efficiency of repository collections" (Cassella, p. 219) but she does allow that recording accurate sets of statistics has long been a problem.

Some of the measures suggested, disappointingly, assume you are using a certain type of software.  For example a measure of the 'number of active collections' in a repository is going to be impossible to record at a level lower than department for many people using the standard EPrints set up, as we are doing with WRAP.  Also some of the external measures are not quite as developed as I would have liked.  For example Cassella suggests that one of the external measures should be "Interoperability", which I agree is an important measure, but Cassella never quite articulates fully exactly what she means by interoperability and whether or not it should be an active or a passive measure.

Overall I found the article very thought provoking on the issue of benchmarking, and some of the measures I plan to add into my regular statistics collection, but there is definitely more work waiting to be done in this area.

February 15, 2010

Ranking repositories

Writing about web page

Webometrics have published their rankings for repositories, and their methodology is described online. This is the first time they've actually listed WRAP and we're at no. 273. They are primarily focussed on repositories like WRAP that are all about research content. Their criteria for measurement are listed as:

"Size (S). Number of pages recovered from the four largest engines: Google, Yahoo, Live Search and Exalead.
Visibility (V). The total number of unique external links received (inlinks) by a site can be only confidently obtained from Yahoo Search and Exalead.
Rich Files (R). Only the number of text files in Acrobat format (.pdf) extracted from Google and Yahoo are considered.
Scholar (Sc). Using Google Scholar database we calculate the mean of the normalised total number of papers and those (recent papers) published between 2001 and 2008."

But if you decided that the Webometrics ranking were an important one (a whole other issue!) then you might want to work on influencing these...

50% of the ranking is given to Visibility, so you'd want to concentrate on getting people to link in to your content from other sites. This is not only good for Webometrics, but reputedly also for your "Google Juice" (ie how high your content appears in Google results lists). I've yet to investigate whether we can find any stats out for ourselves from Yahoo Search or Exalead. However, sending this message out to your authors that they should link in to your content and encourage others to do so could cloud the main issue, which is about getting them to send us content in the first place. I think that this kind of a message is one for a mature repository to focus on, where there is already a culture of high deposits. Because the main priority for a repository is surely to make lots of content available on OA, not to score well in a repository ranking!

20% is dependent upon size. So getting lots of content and focussing on this message with your authors is important too. It is my highest priority in any case...

15% is dedicated to "Rich files" which seems to be if there are pdf files... this isn't necessarily the best thing for a repository from a preservation angle, nor if you would like to allow data-mining on your content. It might not even be the best display format for all types of content. So it would seem to me to be the least important metric to focus on, if I understand it correctly.

The final 15% is dependent on Google Scholar... Google Scholar does not currently index all of WRAP's content. I have written to them about this, and I know that other repositories have the same issue but I still haven't go to the bottom of it. My theory is that, if you read their "about" pages, they are indexing our content but not presenting it in their results sets because they de-duplicate articles in favour of final published versions: they present these rather than repository results, so if I look for all content on the wrap domain through GScholar I won't get as many results as I have articles in the repository. If my theory is right then it could be significant to learn whether Webometrics is using their raw data before any such de-duplication.  I might be wrong, though!

Also note the dates of publication that are relevant to the GScholar data. We have said to authors that as far back in time as they feel is important/significant is fine with us (helps to win them over, useful for REF data and web-pages driven by RSS feeds from WRAP). But if you wanted to be more strategic in raising your ranking on Webometrics then you'd need to change the policy to focus on content published in the last 10 years...

I don't think we shall be playing any such games! But it is interesting to see what ranking providers consider to be important...

February 2023

Mo Tu We Th Fr Sa Su
Jan |  Today  |
      1 2 3 4 5
6 7 8 9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 26
27 28               

Visit the WRAP repository

Twitter Feed

Search this blog



Most recent comments

  • @Jackie, thanks! I'm very proud of the team and everything we have achived in the past year. Looking… by Yvonne Budden on this entry
  • That's an impressive amount of full text Yvonne. Congratulations to everyone at Warwick. by Jackie Wickham on this entry
  • In my opinion the DEA is a danger to digital liberties and should be thrown out, period Andy @ Lotto… by Andy on this entry
  • Has anyone tried an assessment using the suggested PIs– including the author of the paper? It seems … by Hannah Payne on this entry
  • Hi Yvonne I came across this article myself recently. And I was wondering how much of an issue this … by Jackie Wickham on this entry

Blog archive

RSS2.0 Atom
Not signed in
Sign in

Powered by BlogBuilder