Can citation indicators be used to assess research quality?

Citations are increasingly used as performance indicators in research policy and within the research system. In a new article it is argued that citations reflect aspects related to scientific impact but not to other key dimensions of research quality.

Citations indicators are based on data on how often publications are referred to in the subsequent scientific literature. Usually, citations are assumed to reflect the impact of the research or its quality. What is the justification for these assumptions and how do citations relate to research quality? These are questions that are discussed in a new article by NIFU researchers Dag W. Aksnes, Liv Langfeldt and Paul Wouters (CWTS, Leiden University) from R-QUEST.

The article, published in the journal SAGE Open, provides an overview of some of the main issues at stake, including theories of citation and the interpretation and validity of citations as performance measures. Research quality is a multidimensional concept, where plausibility/soundness, originality, scientific value, and societal value commonly are perceived as key characteristics. The article investigates how citations may relate to these various research quality dimensions. It is argued that citations reflect aspects related to scientific impact and relevance, although with important limitations. On the contrary, there is no evidence that citations reflect other key dimensions of research quality. Hence, an increased use of citation indicators in research evaluation and funding may imply less attention to these other research quality dimensions, such as solidity/plausibility, originality, and societal value.

The article is available here: Citations, citation indicators, and research quality: An overview of basic concepts and theories.

Dag W. Aksnes
Research Professor, NIFU
dag.w.aksnes@nifu.no