Best Practices

Changes in scientific assessment systems

An ever-increasing number of voices are advocating changes in order to improve systems for assessing scientific activity. They defend the development and application of criteria that attach value to different qualities and contributions in research, not solely to the visibility of studies or the bibliometric data of scientific publications.

 

Factsheet 
 

  • Geographical scope: global.
     
  • Sources: 

CURRY, S., et al. (2020): The changing role of funders in responsible research assessment: progress, obstacles and the way ahead, Research on Research Institute. 

DELGADO-LÓPEZ-CÓZAR, E. et al. (2021): «Letter: A call for a radical change in research evaluation in Spain», Profesional de la Información, 30(3). 

1. Context

This last decade has seen an intensification of debate, across the international scientific community, around the ways in which science is funded and practiced. Exchanges consider how scientific activity can attain greater openness and inclusion and achieve a higher impact. These debates are accompanied by critical reflections regarding research assessment processes and criteria.

2. Debate

The use of bibliometric indicators based on journal impact factors has spread to every branch of knowledge and practically all areas of scientific activity. Assessment systems have progressed from being based on evaluations by experts to depending almost exclusively on these metrics.

The consequences of this abusive use of bibliometric indicators are pernicious on an individual and a collective scale. Scientific activity is transformed into a spiral, geared towards accumulating more and more publications and increasing the number of citations. Researchers concentrate on publication as their main goal and neglect other activities that are not as profitable for their career. The pressure to publish increases the number of irrelevant studies and encourages the proliferation of poor practices and unethical behaviours. The governance and the strategic lines of universities and research centres are disrupted. The diversity of research agendas is undermined, while lines that are slow to produce results, uncertain, complex, or costly, are abandoned in favour of issues that are more fashionable, with sure and immediate returns. 

In response to this situation, numerous initiatives and declarations have emerged on a worldwide scale – such as the San Francisco Declaration on Research Assessment (DORA), in 2013, or the Leiden Manifesto, in 2015 – which advocate changes in assessment systems and have received the adhesion of numerous institutions from around the world. In recent years, countless organisations in different academic fields and regions have kicked off a process of reform in their assessment processes and are trialling new practices, such as the use of narrative formats for curriculums, the evaluation of actions linked to the advancing of open science, references to multidisciplinarity, peer assessment, and attention to relevance and the local and social impact of contributions.

3. Conclusions

The endless stream of initiatives that have been developed in recent years suggests that research assessment has entered a new era. Practices based purely on quantitative valuations and on the exclusive use of journal impact factors are being dropped, and progress is being made towards new assessment models that strive to be more plural and to take into consideration other factors too, such as the value, quality and impact – scientific but also social and local – of contributions and the accessibility of results.

Classification

Tags

Subject areas

Related content

El aburrimiento, por Josefa Ros

Josefa Ros, filósofa e investigadora en la Universidad Complutense de Madrid, nos desvela conocimientos relevantes sobre una experiencia que todo el mundo ha vivido: el aburrimiento.

How do gender, experience and caseload affect judicial decisions on intimate-partner violence cases?

Are male and female judges equally likely to grant restraining orders in cases of gender-based violence? According to this study, gender alone is not a determining factor but it is a key factor, together with experience and caseload.

Social Research Call

Flash Calls

El café del observatorio

You may also find interesting

Percentage of people who view themselves as above their parents or grandparents on the social ladder

Percentage of people who view themselves as above their parents or grandparents on the social ladder


Some 29% of Spaniards have a social position above that of their parents, and over 40% believe they have risen above their grandparents on the social ladder.

Number of generations necessary so that those born in families with low incomes can reach the average level of income in their society

Number of generations necessary so that those born in families with low incomes can reach the average level of income in their society


In Spain, according to this indicator, it takes four generations to improve the incomes of families with precarious finances.

Fertility rate

Fertility rate


The fertility rate in Spain is 1.23 children per woman, the second lowest rate among European Union countries.