The University Medical Center (UMC) Utrecht has developed a new method to assess research and researchers. In the new assessment, creating impact is key. Researchers are asked how they arrived at their research questions and what they do to advance the results of their work.

“Read, don't count,” said Professor Frank Miedema, dean and vice-chairman of the Board of the UMC Utrecht, this week in a comment in Nature. “That is the essence of our new assessment method. The method is partly descriptive and partly qualitative. We want to use it to identify how researchers and research programs are equipped to create impact (and hence social relevance) with their results. It is also meant as a stimulus to enter into partnerships with partners outside the UMC, such as knowledge institutes and companies.”

Link to DUTCH Health Council REPORT

With its new policy, the UMC Utrecht closely follows the recommendations that the Dutch Health Council issued two weeks ago: “The research that is done in UMCs can contribute far more to quality and the affordability of care and prevention. That requires ongoing collaboration with other care providers, patients and municipalities, so that questions are researched that are important in day-to-day practice.”

Portfolio for candidate professors

For the appointment of professors, the new method means that candidates are asked to draw up a portfolio. In this portfolio, they should list not only their scientific output but also their performance in the domains of education; clinical work; innovation and impact; leadership, development and collaboration. This will allow the appointment committee to get a broader picture of the candidate than through the customary résumé and publication list. The first professors have since been appointed using this approach. The UMC Utrecht intends to make the portfolio the standard method for all professor appointments – some twenty every year.

Don't just look at findings

In the assessment of research programs, the new approach means that in addition to the findings of their research, the researchers must also describe the structures and processes. How was leadership organized? How did they arrive at their research questions? With whom did they collaborate? Did they think about the next step if they got positive results? What was the methodological quality of the research? The six research programs of the UMC Utrecht are currently using the method for self-assessment, which should be complete by the end of the year. The approach will be further rolled out in the UMC Utrecht over the next few years. Incidentally, not all researchers will as yet be acquainted with the new method; that takes time in a huge organization such as the UMC Utrecht.

International debate: “increasing value, reducing waste”

The new assessment method for scientific research is a response to the international debate on publication pressure, perverse incentives and reproducibility in science. At present, researchers are often judged on how many scientific publications they write and the journals in which these articles appear. That has led to a huge increase in the number of scientific publications, but also to great concern about their quality and relevance. In 2014, a large group of scientists published an article in The Lancet, calling for “increasing value, reducing waste” in biomedical research. Earlier this month in Nature Biotechnology, three researchers of the UMC Utrecht called for a new reward system for scientists, in order to forge a strong link between research and practice. As co-founder of Science in Transition in 2013, Frank Miedema strongly contributed to the debate on publication pressure in science. Miedema and staff adviser Rinze Benedictus of the UMC Utrecht jointly describe the new reward policy in a comment in the latest issue of Nature, which is published on 27 October.

Publication

Benedictus R, Miedema F. Redefining excellence – Fix incentives to fix science. Nature 2016;538:453-455 (27 october 2016) doi:10.1038/538453a