About COMPARE
In the last decade there have been many national and international calls to reform the scientific evaluation system and the design of academic careers. The San Francisco Declaration (DORA), the Leiden Manifesto or the Hong Kong Principles are just some examples of these initiatives which call for a responsible use of research metrics. Furthermore, external pressures derived from the global health and socioeconomic crisis we live are pushing institutions to experiment with new evaluation models valuing transparency, interdisciplinarity and a knowledge transfer and outreach.
COMPARE aims at systematically profiling of scientists and team dynamics in order to better inform academic policymakers in their decision making processes. The project is structured in four components: exploration of new data sources, profiling of scientists, study of team dynamics and case studies. Following we detail the expectations to be fulfilled in each of these components.
Data sources
The COMPARE project aims to develop technical solutions and methodologies that make use of open and alternative scientific information sources and transcend traditional bibliometric approaches to the quantitative study of scientific activity. We aim to explore and build new datasets from open or proprietary sources which can later be shared openly and re-used by the community.
COMPARE also seeks to experiment with new formats of delivering and reporting research findings. As the project advances, results will be shared through our dashboard toolbox so that users can interactively explore the data and findings produced. We will be experimenting both internally and externally with new collaborative tools such as the Open Science Framework, which we will be sharing openly once the project has concluded.
Profiling of scientists
COMPARE builds upon the findings of the ECOSCI project, which aimed at studying the diversity of forms by which academics build and disseminate knowledge. This project found consistent archetypes of researchers based on their task distribution and proposed a logic model by which diversity can be systematically described and analyzed.
COMPARE takes a step further and expands its analyses to other forms of research. Specifically it aims at:
- Identification of archetypes in different fields of research, including the Arts and Humanities.
- Relation between international mobility, task distribution and the development of scientific fields.
- Profiling of researchers and scientific topics based on the societal orientation of their output.
Team dynamics
Evaluative processes struggle with the notion of diversity and the introduction of team-level approaches. This is because metrics rely solely on authorship as the means of attributing and distributing credit. The notion of a behaviourist attitude towards the reward system of science is contradicted by evidence suggesting that scientists behave in ways that can potentially be detrimental to their own career prospects.
COMPARE aims at studying how scholars performing different roles and showcasing different profiles collaborate and combine their expertise to produce new scientific knowledge. To do so we pursue the following objectives:
- What is a team in science? Is co-authorship or project collaboration sufficient to define and identify teams in science? Do we observe different collaboration attitudes by fields?
- How can we operationalize the different notions of team collaboration observed? Can we develop scientometric methods that can adjust to the different scenarios?