Where to find research metrics

There are a range of different data sources used to generate research metrics, with different services having different subject coverage, types of documents included, timespans and metrics.  Using only one of these services gives an incomplete picture so it is good practice to use a range of metrics, understand something about how they are calculated, and only compare like with like. 

It is important that, when determining research quality and performance, bibliometrics are only used to supplement peer review and expert opinion, rather than replace it. Find out more about using metrics responsibly.

This page introduces a range of different sources of metrics accessible to University of Birmingham researchers.  

 

 Scopus

Scopus is a large multidisciplinary database whose content complements Web of Science. The University has invested in the SciVal service which utilises Scopus data to provide enhanced research metrics to researchers and managers.

Key features of Scopus:

Scopus
 Access  From FindIt@Bham, subscribed resource
Metrics available  

Compare journals on Scopus by using:

  • SCImago Journal Rank Indicator (SJR)–
  • Impact per Publication (IPP)  this is the same calculation as Thomson Reuters’ Journal Impact Factor, but working from a different set of data and using a 3 year citation window -considered to be the optimal time period to accurately measure citations in most subject fields.
  • Source Normalised Impact per Paper (SNIP)

Scopus Author search gives an author summary including:

  • H-index
  • Total citations
  • List of publications

Article metrics are provided including:

  • Citation-based metrics
  • Alternative metrics (altmetrics) indicating community engagement
  • Citation benchmarking
 Strengths
  • Better coverage of non-science subjects
  • Gives the number of self-citations
  • Gives international collaboration (number of authors from more than one country)
  • Scopus and SciVal metrics are used by several external organisations including some research councils, World University Rankings, REF etc. 
 Limitations
  •  Citation information only available for 1996 onwards.
  • Scopus are investing in expanding their content coverage, to include more books, conference papers (particularly in the engineering fields) and an expansion of citations back to 1970 (currently goes back to 1996) to enhance the accuracy of their h-index calculation.
 Further information

Training material available via http://trainingdesk.elsevier.com/products/scopus

Find out more about SciVal at Birmingham

 Web of Science and Journal Citation Reports

 

This was the original source for bibliometrics, with Thomson Reuters having 50 years of expertise in content selection.  Web of Science is composed of several citation indexes, built specifically to enable users to track linkages between publications via their cited references. 

Key features of Thomson Reuters Web of Science and Journal Citation Reports: 

WoS
 Access  From FindIt@Bham, subscribed resource
 Metrics available
  • Journal Impact Factor (JIF) from Journal Citation Reports, based on a 2 year citation window.
  • Citation search available within Web of Science
  • H Index from Web of Science
 Strengths
  • Longest timespan for citation information
  • Good coverage of sciences
  • Covers conference proceedings
 Limitations
  •  No coverage of books
 Further information  Training material available at http://wokinfo.com/training_support/training/

 Altmetric.com

 

Altmetrics (alternative metrics) look beyond traditional academic publishing mechanisms to discover the impact of research or a researcher through social media.  Results for each social media channel will differ as each appeal to different audiences, and different platforms have different discipline biases, for example highly tweeted stories tend to be focussed on policy, gender, funding and contentious science issues. 

Altmetric.com, a service which is free of charge for researchers, generates a colour-coded “donut” and article score, indicating the level of sharing activity via sources such as blogs, news outlets, tweets, facebook, Wikipedia, google+, CiteULike, Mendeley readers and more.

Key features of Altmetric.com:

altmetric
 Access http://www.altmetric.com
 Metrics available The article score and colour-coded donut.  
 Strengths The article score includes a quality measure by giving higher scores to news items rather than facebook pages, and to attention from a researcher as opposed to automated tweets.
 Limitations The article score is a complex calculation, explained to some extent in the help pages, but underlines the importance of understanding the tool you are using.
 Further information

Altmetric.com offers

  • Altmetric bookmarklet – a free browser plug in that lets you instantly see the Altmetric data for any publication with a DOI. Click on the donut to view the full details page for each output.
  • Altmetric badges – these enable you to showcase the online attention surrounding your research, and it’s free to embed them in your individual profile or publications page in just a few simple steps.

 Google Scholar and Publish or Perish

Google Scholar harvests bibliographic information from major academic publishers and repositories worldwide, covering both free and subscription sources.  Google Scholar aims to be comprehensive, but they do not guarantee uninterrupted coverage of any particular source and updating frequencies can vary depending on the data source. 

Unlike formal bibliographic databases such as Web of Science and Scopus, data selected for inclusion is not subject to a clearly stated selection policy, so can cover a range of sources. 

It is worth remembering the slightly nebulous qualities of Google Scholar when using it for the generation of research metrics.

Search support for Microsoft Academic Search is still in its early stages and its coverage is more limited than that of Google Scholar.

Publish or Perish is a software program that retrieves and analyses academic citations from Google Scholar and (since release 4.1) Microsoft Academic Search to obtain the raw citations, then analyses these and presents a range of metrics.

Key features of Publish or Perish:

altmetric
 Access Harzing’s Publish or Perish (PoP) software is free to download from http://www.harzing.com/resources/publish-or-perish
 Metrics available
  • Total number of papers and total number of citations
  • Average citations per paper, citations per author, papers per author, and citations per year
  • Hirsch's h-index and related parameters
  • Egghe's g-index
  • The contemporary h-index
  • Three variations of individual h-indices
  • The average annual increase in the individual h-index
  • The age-weighted citation rate
  • An analysis of the number of authors per paper.
 Strengths
  • Best coverage of Open Access journals
  • Indexes books, so provides good coverage of humanities and social sciences
  • Best suited to disciplines where research is slower to be cited
Good coverage of conferences
 Limitations Cannot search journals by subject category, individual journal searches only.
 Further information

Training material available at http://www.harzing.com/resources/publish-or-perish

Eigenfactor Journal Ranking

 

Eigenfactor offers a journal ranking score, based on data from Thomson Reuters Journal Citation Reports. 

Key features of Eigenfactor:

altmetric
 Access Free of charge, available at http://www.eigenfactor.org/projects/journalRank/journalsearch.php
 Metrics available
  • Eigenfactor Journal Ranking Score
  • Article Influence Score (more comparable with JIF)
 Strengths

Attempts to normalise the score across disciplines and works with a 5 year citation window.

Takes account of disciplinary relationship between citing and cited journals
 Limitations

More weight given to highly ranked journals, which may be self-fulfilling.

Considered to be more robust, but is not generally used in the UK.
 Further information

Forms part of the Eigenfactor project and suite of products, see http://www.eigenfactor.org

 CiteSeerX

CiteSeerx is a scientific literature digital library and search engine focussing on the literature in computer and information science.  It crawls publicly available scholarly documents primarily from author webpages and other open resources, and does not have access to publisher metadata, consequently citation counts in CiteSeerX are usually less than those in Publish or Perish.  CiteSeerX computes citation statistics and related documents for all articles cited in the database, not just the indexed articles.

Key features of CiteSeerX:

altmetric
 Access Freely available, at http://citeseerx.ist.psu.edu/index
 Metrics available
  • Citation counts
 Strengths
  • A supplementary information source for subject areas perhaps not covered by the main bibliometrics data sources
 Limitations Limited metrics
 Further information

An evolving service that aims to provide resources such as algorithms, data, metadata, services, techniques, and software that can be used to promote other digital libraries.