sofs.org.ua › About us › Archive › 2023 › №2(120) › 1.4

1.4

EVALUATION OF THE RESULTS OF RESEARCHERS’ ACTIVITIES: NEW TRENDS

I.Yu. YEGOROV 1
https://orcid.org/0000-0002-3829-6383
State Institution «Institute of Economics and Forecasting of the NAS of Ukraine»
І.A. ZHUKOVYCH 1
https://orcid.org/0000-0001-5027-5991
1 State Institution «Center for Evaluation of Activity of Research Institutions and Scientific Support of Regional Development of Ukraine NАS of Ukraine»

Nauka naukozn. 2023, 2(120): 42—58
https://doi.org/10.15407/sofs2023.02.042

Section: Problems of the development of science and technology potential
Language: Ukrainian
Abstract: The article is devoted to the analysis of new trends in evaluating the results of researchers’ activities using the example of the evaluation procedure used by the League of European Research Universities (LERU). The author’s vision of the expediency and possibilities of its implementation in Ukraine is also expressed. The sources of information for research are the publications of domestic and foreign scientists, the regulatory and legislative framework of Ukraine and the recommendations of international structures regarding the evaluation. New trends in the evaluation of the results of researchers’ activities are becoming relevant due to the fact that a number of international documents adopted in 2022 define the need to update the approach to evaluation, which should be based on multidimensional evaluations and ensure a balance between qualitative and quantitative criteria in order to obtain objective information. It was determined that the general structure of evaluating the results of researchers’ activity according to the LERU methodology includes three types of standard evaluation (effectiveness of research, educational activity and service to society), which are characterized by a wide range of criteria and allow assessing the results of activities over the past period. To improve the LERU assessment procedure, recommendations were developed to supplement the standard assessment with three characteristics: a multidimensional perspective, a development perspective, and a contextual perspective. Their essence is explored in the article. On the basis of the conducted research, it was concluded that the evaluation of the results of researchers’ activities should not be limited to identifying shortcomings and providing comments, but should include a critical analysis followed by the determination of areas for improvement of activities and the provision of appropriate recommendations. The main goal of the evaluation is not to fi x the results, but to push for a new paradigm of development – to improve the results of the work of both individual researchers and the scientific institution as a whole. The opinion is expressed that the adapted methodology of LERU can be used for the attestation (evaluation) of researchers in Ukraine.

Keywords: evaluation of the results of researchers’ activities, peer review, evaluation criteria, scientometric indicators, The League of European Research Universities, National Academy of Sciences of Ukraine.

References

  1. Ker, D., & Galindo-Rueda, F. (2017). Frascati Manual R&D and the System of National Accounts. OECD Science, Technology and Industry Working Papers, 2017/06. OECD Publishing, Paris. http://doi.org/10.1787/edb6e020-en
  2. Reaction to the Council Conclusions on Research Assessment and the Implementation of Open Science Policies. https://doi.org/10.5281/zenodo.704309
  3. Hicks, D., Wouters, P., Waltman, L., de Rijcke S., & Rafols, I. (2015). Bibliometrics: The Leiden Manifesto for research metrics. Nature, 520, 429—431. https://doi.org/10.1038/520429a
  4. Moher, D., Bouter, L., Kleinert, S., Glasziou, P., Sham, M.H., Barbour, V., et al. (2020). The Hong Kong Principles for assessing researchers: Fostering research integrity. PLoS Biol, 18 (7), e3000737. https://doi.org/10.1371/journal.pbio.3000737
  5. Yegorov, I.Yu., & Zhukovych, I.A. (2022). Development of approaches to the evaluation of the results of the activity of the scientific and recent establishments. Science and Science of Science, 3, 36—50. https://doi.org/10.15407/sofs2022.03.036 [in Ukrainian].
  6. Rybachuk, V.P. (2013). Methodological problems of evaluating the productivity of scientific activity. Science and Science of Science, 2, 46—52 [in Ukrainian].
  7. Laktionova, T.M., Parasochka, I.F., & Havrylova, N.A. (2021). Bibliometric indicators of publication activities in research units of the institute. Science and Science of Science, 4(114), 74—93. https://doi.org/10.15407/sofs2021.04.074 [in Ukrainian].
  8. Mryglod, O.I., & Nazarovets, S.A. (2019). Scientometrics and management of scientific activities once again about the global and Ukrainian. Visn. Nac. Akad. Nauk Ukr., 9, 81—94. https://doi.org/10.15407/visn2019.09.081 [in Ukrainian].
  9. Bakumenko, V., Chala, N., & Tsedik, M. (2021). To the question of evaluating the scientific results of Ukrainian scientists. Journal of research papers of the National Academy for Public Administration under the President of Ukraine, 1, 15—26 [in Ukrainian].
  10. Kostenko, L., & Symonenko, T. (2020) Scientometrics 2.0: conceptual positions and approbation. Libraries of national academies of sciences: functioning problems, development trends, 18, 52—63. https://doi.org/10.15407/maan2020.18.052 [in Russian].
  11. Yegorov, I. (2018). Indicators of public activity and quality criteria of scientific edition: approaches to definition Science, technologies, innovations, 3, 3—10 [in Ukrainian].
  12. Yegorov, I. (2019). Evaluation of research institutes of the National Academy of Sciences of Ukraine: problems with implementation of the best practices. Fteval Journal, 47, 89—93. https://doi.org/10.22163/fteval.2019.336
  13. Zhukovych, I.A., & Yegorov, I.Yu. (2020). The Evolution of Methodical Approaches for Evaluation the Efficiency of the Activities of Scientific Research Organizations in the Czech Republic. Statistics of Ukraine, 2—3, 117—126. https://doi.org/10.31767/su.2-3(89-90)2020.02-03.13 [in Ukrainian].
  14. Zhukovych, I.A., & Yegorov, I.Yu. (2021). Evolution of Methodological Approaches to Evaluating the Effi ciency of the Activities of Research Organizations in Ukraine. Statistics of Ukraine, 2, 4—15. http://doi.org/10.31767/su.2(93)2021.02.01 [in Ukrainian].
  15. Zhukovych, І.A. (2022). Peculiarities of Evaluating the Activity of Scientific Organizations in Some Countries of the Former USSR. Statistics of Ukraine, 2, 41—50. https://doi.org/10.31767/su.2(97)2022.02.05 [in Ukrainian].
  16. Yegorov, I.Yu., Zhukovych, I.A., Stupina, L.V., & Artiushenko, V.V. (2022). Analysis of the evaluation the efficiency the results of evaluation the efficiency of the research organizations of the NAS of Ukraine in the first round of the period 2016-2022. Visn. Nac. Akad. Nauk Ukr., 11, 79—89. https://doi.org/10.15407/visn2022.11.079 [in Ukrainian].
  17. Overlaet, В. (2022). A Pathway towards Multidimensional Academic Careers. A LERU Framework for the Assessment of Researchers. LERU position paper. January, 2022. URL: https://www.leru.org/files/Publications/LERU_PositionPaper_ Framework-for-the-Assessment-of-Researchers.pdf (last accessed: 04.01.2023).
  18. Merton, R. (1968). The Matthew Effect in Science: The reward and communication systems of science are considered. Science, 159 (3810), 56—63. https://doi.org/10.1126/science.159.3810.56
  19. Dobrov, G., Savelyev, A., & Tonkal, V. (1987). Studying research utilization in the Ukraine. Science and Public Policy, 14 (4), 231—235. https://doi.org/10.1093/spp/14.4.231
  20. Cooper, M. (2021). Research: Women Leaders Took on Even More Invisible Work During the Pandemic, Harvard Business Review, October 27. URL: https://hbr.org/2021/10/research-women-took-on-even-more-invisible-work-during-thepandemic (last accessed: 04.01.2023).

Full Text (PDF)