Best Practices

Changes in scientific assessment systems

An ever-increasing number of voices are advocating changes in order to improve systems for assessing scientific activity. They defend the development and application of criteria that attach value to different qualities and contributions in research, not solely to the visibility of studies or the bibliometric data of scientific publications.

 

Factsheet 
 

  • Geographical scope: global.
     
  • Sources: 

CURRY, S., et al. (2020): The changing role of funders in responsible research assessment: progress, obstacles and the way ahead, Research on Research Institute. 

DELGADO-LÓPEZ-CÓZAR, E. et al. (2021): «Letter: A call for a radical change in research evaluation in Spain», Profesional de la Información, 30(3). 

1. Context

This last decade has seen an intensification of debate, across the international scientific community, around the ways in which science is funded and practiced. Exchanges consider how scientific activity can attain greater openness and inclusion and achieve a higher impact. These debates are accompanied by critical reflections regarding research assessment processes and criteria.

2. Debate

The use of bibliometric indicators based on journal impact factors has spread to every branch of knowledge and practically all areas of scientific activity. Assessment systems have progressed from being based on evaluations by experts to depending almost exclusively on these metrics.

The consequences of this abusive use of bibliometric indicators are pernicious on an individual and a collective scale. Scientific activity is transformed into a spiral, geared towards accumulating more and more publications and increasing the number of citations. Researchers concentrate on publication as their main goal and neglect other activities that are not as profitable for their career. The pressure to publish increases the number of irrelevant studies and encourages the proliferation of poor practices and unethical behaviours. The governance and the strategic lines of universities and research centres are disrupted. The diversity of research agendas is undermined, while lines that are slow to produce results, uncertain, complex, or costly, are abandoned in favour of issues that are more fashionable, with sure and immediate returns. 

In response to this situation, numerous initiatives and declarations have emerged on a worldwide scale – such as the San Francisco Declaration on Research Assessment (DORA), in 2013, or the Leiden Manifesto, in 2015 – which advocate changes in assessment systems and have received the adhesion of numerous institutions from around the world. In recent years, countless organisations in different academic fields and regions have kicked off a process of reform in their assessment processes and are trialling new practices, such as the use of narrative formats for curriculums, the evaluation of actions linked to the advancing of open science, references to multidisciplinarity, peer assessment, and attention to relevance and the local and social impact of contributions.

3. Conclusions

The endless stream of initiatives that have been developed in recent years suggests that research assessment has entered a new era. Practices based purely on quantitative valuations and on the exclusive use of journal impact factors are being dropped, and progress is being made towards new assessment models that strive to be more plural and to take into consideration other factors too, such as the value, quality and impact – scientific but also social and local – of contributions and the accessibility of results.

Classification

Tags

Subject areas

Related content

Article

Looking beyond mobile phones to understand the well-being of Portuguese young adults

In the current context of the COVID-19 pandemic, technologies such as the mobile phone are a double-edged sword. The results of this study show that, on the one hand, problematic use of mobile technologies is associated with lower levels of well-being among young adults. On the other hand, Portuguese young adults believe that mobile technologies have been essential to their well-being through the lockdowns and social distancing rules triggered by the pandemic.

Article

Health safety knowledge in Portugal and Spain

This article examines how being motivated by disease prevention (“safety”) or pleasure promotion (“pleasure”) shapes the way people construe sexual health and pursue their sexual goals.

Public presentation of the Dossier on "Research and Innovation in Portugal and Spain"

What are the decisive factors in a country's research and innovation systems? What about the links between science and business? We present the new Dossier of the Social Observatory of the "la Caixa" Foundation.

Interview

“Research can’t be simply turned on and off like a tap; it requires investment in the long term”

Andrew W. Wyckoff, the OECD’s current director of Science, Technology and Innovation, reviews the major challenges posed by the digital transformation in the economic, social and educational sphere.

Infodata

Scientists and engineers by sex

Portugal, together with Denmark, is an exception within the framework of the EU-27 and has percentages of women scientists and engineers that slightly exceed the European average.

You may also find interesting

Evolution of science and technology in Portugal and Spain

Article

Evolution of science and technology in Portugal and Spain


Science

Research and innovation in Portugal and Spain have developed along parallel paths, albeit with some distinctive features.

Science-business links in Portugal and Spain: untapped potential for innovation?

Article

Science-business links in Portugal and Spain: untapped potential for innovation?


Science

In Portugal and Spain, only 6% of PhD holders work in the business sector. There is a broad margin for improvement in the relationship between science and business.

Looking beyond mobile phones to understand the well-being of Portuguese young adults

Article

Looking beyond mobile phones to understand the well-being of Portuguese young adults


Science

In the current context of the COVID-19 pandemic, technologies such as the mobile phone are a double-edged sword. The results of this study show that, on the one hand, problematic use of mobile technologies is associated with lower levels of well-being among young adults. On the other hand, Portuguese young adults believe that mobile technologies have been essential to their well-being through the lockdowns and social distancing rules triggered by the pandemic.