{"id":7264,"date":"2024-06-21T19:44:15","date_gmt":"2024-06-21T17:44:15","guid":{"rendered":"https:\/\/sofs.org.ua\/?page_id=7264"},"modified":"2024-07-22T10:03:59","modified_gmt":"2024-07-22T08:03:59","slug":"1-4","status":"publish","type":"page","link":"https:\/\/sofs.org.ua\/en\/sample-page\/arhiv\/2024-2\/2-124\/1-4\/","title":{"rendered":"1.4"},"content":{"rendered":"

VITAL ISSUES OF EVALUATING THE SOCIAL IMPACT OF SCIENTIFIC RESEARCH<\/strong><\/p>\n

T.\u041e<\/strong>. YAROSHENKO<\/strong> 1
\n<\/sup>https:\/\/orcid.org\/0000-0002-2985-2333<\/a>
\nO.I. YAROSHENKO <\/strong>1
\n<\/sup>https:\/\/orcid.org\/0000-0002-4716-5705<\/a>
\n1 <\/sup>National University of Kyiv-Mohyla Academy, Ukraine<\/p>\n

Nauka naukozn. 2024, 2(124): 52\u201482
\nhttps:\/\/doi.org\/10.15407\/sofs2024.02.052<\/a><\/p>\n

Section<\/strong>: Scientometrics
\nLanguage:<\/strong> Ukrainian
\nAbstract:<\/strong> The sound assessment of scientific research has gained the increased topicality in the past decade due to the challenges of open science, enhancing the research productivity, transparency and impartiality of researchers\u2019 scores. However, the measurement of quality and social impact of research is complicated because of its specifics across research disciplines and countries. Numerous initiatives call for new approaches to the measurement of the social impact of research, to keeping the balance between quantitative and qualitative indicators, to introducing industry-normalized or alternative metrics, especially in view of the challenges of open science. The article contains a review of advanced methods for measuring the social impact of scientific research in selected countries (U.K., Spain, U.S., Canada, Australia), and alternative metrics and tools for measuring this impact ((Dimensions, The Lens, SciVal), with focus on research in mass media and communications. The authors\u2019 research builds on general (abstraction, analysis and synthesis, study of sources, etc.) and specialized methods (content analysis, media discourse analysis, etc.), added by selected altmetric indicators. Top-10 articles with the highest Altmetric Attention Score (AAS) and the highest citation were determined by data from Dimensions. The highest AAS were revealed for articles devoted to COVID-19, giving evidence of the increased public attention to the topic of global pandemic. The topic of the Russian-Ukrainian war and respective research (publications in 2022\u20142023) in mass media and communications (based on data from Dimensions) was chosen to measure the public interest in alternative metrics of impact. Ten articles with the highest AAS were determined in the sample of 514 articles, with showing citation figures for them. The impact of research in mass media and communications even for a sensitive topic like a war in the 21st century in the center of Europe was found to be much weaker than the one for medical research, and it had a substantially lower citation figure. It is argued that drawbacks of the alternative metrics are rooted in nontransparent methodology and hidden estimation algorithm: the alternative metrics is an indicator of online attention to an article (not always positive) rather than the quality of respective research, and it cannot be fi t for cross-discipline comparisons of articles.<\/p>\n

Keywords<\/strong>: public impact of scientific research, research engagement, research assessment, alternative metrics, Altmetric, Altmetric Attention Score, Dimensions, SciVal, Lens, research in mass media and communications.<\/strong><\/p>\n

References<\/strong><\/p>\n

    \n
  1. Yaroshenko, T.O., & Zharinova, A.H. (2023). Scientific citation: historical and theoretical landscape. Science and Science of Science, 3\u00a0(121), 41\u201467. https:\/\/doi.org\/10.15407\/sofs2023.03.041<\/a> [in Ukrainian].<\/li>\n
  2. Yaroshenko, T., Serbin, O., & Yaroshenko, O. (2022). Open science: the role of universities and libraries in modern changes in scientific communication. Digital platform: information technologies in the socio-cultural sphere, 5\u00a0(2), 277\u2013292. https:\/\/doi.org\/10.31866\/2617-796X.5.2.2022.270132<\/a> [in Ukrainian].<\/li>\n
  3. Garfield, E. (1979). Citation Indexing: Its Theory and Application in Science, Technology, and Humanities. New York, NY: Wiley.<\/li>\n
  4. Elkana, Y. (1978). Toward a Metric of Science: The Advent of Science Indicators. New York, NY: Wiley.<\/li>\n
  5. Curry, S., Gadd, E. & Wilsdon, J. (2022). Harnessing the Metric Tide: indicators, infrastructures\u00a0& priorities for UK responsible research assessment. Report of The Metric Tide Revisited panel, December 2022. https:\/\/doi.org\/10.6084\/m9.fi gshare.21701624<\/a><\/li>\n
  6. Dobrov, G. (1989). Science about science. Introduction to General Science of Science. Kyiv: Nauk. dumka [in Russian].<\/li>\n
  7. Malitsky, B. (2017). Who and how should assess research results of a scientist. Science and Science of Science, 3\u00a0(97), 34\u201453 https:\/\/doi.org\/10.15407\/sofs2017.03.034<\/a> [in Russian].<\/li>\n
  8. Malitsky, B. (2007). Applied science studies. Kyiv: Phenix [in Ukrainian].<\/li>\n
  9. Malitsky, B., Popovych O., Soloviov V., Yegorov, V.,\u00a0& Bulkin, I. (2004). The rational R&D financing: A premise for building up the knowledge-based society in Ukraine. Kyiv: Phenix [in Ukrainian].<\/li>\n
  10. Popovych O., & Ryzhko L. (2022). Modern challenges to the research system and ways to overcome them. Science and Science of Science, 1\u00a0(115), 88\u2014104. https:\/\/doi.org\/10.15407\/sofs2022.01.088<\/a> [in Ukrainian].<\/li>\n
  11. Rybachuk, V. (2013). Methodological problems in the productivity assessment of research activities. Science and Science of Science, 2, 46\u201452 [in Ukrainian].<\/li>\n
  12. Yehorov, I.Yu. (Ed.) (2016). Indicators of R&D and innovation: international standards and the practice of their use in Ukraine. Kyiv: Institute of Economics and Forecasting of the National Academy of Sciences of Ukraine [in Ukrainian].<\/li>\n
  13. Gasanov, S., Kotlyarevsky, Y., Melnikov, O., Kniaziev, S., Shtangret, A., & Semenyuk,\u00a0E. (2019). Methodological approaches to labor norming in scholarly research and development. Science and Innovation, 15 (1), 5\u201424. https:\/\/doi.org\/10.15407\/scine15.01.005<\/a><\/li>\n
  14. Horovyi, V.M. (2015). Criteria for the quality of scientific research in the context of ensuring national interests. Visn. Nac. Akad. Nauk Ukr., 6, 74\u201480 [in Ukrainian].<\/li>\n
  15. Pavliuk, K.V., & Kaminska, O.S. (2019). Foreign experience in assessing the quality of scientific activity. RFI Scientific Papers, 3, 25\u201440 [in Ukrainian]. https:\/\/doi.org\/10.33763\/npndfi2019.03.025<\/a><\/li>\n
  16. Pylypenko, H.M., & Fedorova, N.Ye. (2020). Science as a factor of socio-economic development of society. Dnipro: NTU \u201cDP\u201d [in Ukrainian].<\/li>\n
  17. Yaroshenko, T., & Yaroshenko, O. (2020). Highly cited documents of Ukrainian scientists in citation databases: correlation of bibliometric indicators. Ukrainian Journal of Library Science and Information Sciences, 5, 108\u2014126. https:\/\/doi.org\/10.31866\/2616-7654.5.2020.205734<\/a> [in Ukrainian].<\/li>\n
  18. Reed, M. (2018). The research impact handbook. 2nd edn. Fast Track Impact, Huntly.<\/li>\n
  19. Reed, M., & Cairney, P. (2021). Using research to influence policy: a handbook for researchers. Leeds: University of Leeds.<\/li>\n
  20. Vutsova, A., Yalamov, T., & Arabadzhieva, M. (2023). In search of excellent research assessment. Baden-Baden: Nomos. https:\/\/doi.org\/10.5771\/9783748937203<\/a><\/li>\n
  21. Alstete, J.W., Beutell, N.J., & Meyer, J.P. (2018). Evaluating scholarship and research impact: history, practices, and policy development. Bingley, UK: Emerald Group Publishing Limited. https:\/\/doi.org\/10.1108\/9781787563872<\/a><\/li>\n
  22. Roemer, R.C. & Borchardt, R. (2015). Meaningful Metrics: A 21st Century Librarian\u2019s Guide to Bibliometrics, Altmetrics, and Research Impact. Chicago: Association of College and Research Libraries, A division of the American Library Association.<\/li>\n
  23. Bornmann, L. (2013). What is societal impact of research and how can it be assessed? A literature survey. Journal of the American Society for Information Science and Technology, 64\u00a0(2), 217\u2014233. https:\/\/doi.org\/10.1002\/asi.22803<\/a><\/li>\n
  24. Servaes, J. (2014). On impact factors and research assessment. Telematics and Informatics, 31\u00a0(1), 1\u20142. https:\/\/doi.org\/10.1016\/j.tele.2013.09.005<\/a><\/li>\n
  25. Samuel, G.N., & Derrick, G.E. (2015). Societal impact evaluation: Exploring evaluator perceptions of the characterization of impact under the REF2014. Research Evaluation, 24\u00a0(3), 229\u2014241. https:\/\/doi.org\/10.1093\/reseval\/rvv007<\/a><\/li>\n
  26. Peruginelli, J., & P\u00f6l\u00f6nen, J. (2023). The legal foundation of responsible research assessment: An overview on European Union and Italy. Research Evaluation, 32\u00a0(4), 670\u2014682. https:\/\/doi.org\/10.1093\/reseval\/rvad035<\/a><\/li>\n
  27. \u041cacFadden, B. (2019). Broader Impact of Science on Society. United Kingdom: Cambridge University Press, 2019. https:\/\/doi.org\/10.1017\/9781108377577<\/a><\/li>\n
  28. Molas-Gallart, J. (2012). Research Governance and the Role of Evaluation. A Comparative Study. American Journal of Evaluation, 33, 583\u2014598. https:\/\/doi.org\/10.1177\/1098214012450938<\/a><\/li>\n
  29. Hammarfelt, B., & Haddow, G. (2018). Conflicting Measures and Values: How Humanities Scholars in Australia and Sweden Use and React to Bibliometric Indicators. Journal of the Association for Information Science and Technology, 69, 924\u2014935. https:\/\/doi.org\/10.1002\/asi.24043<\/a><\/li>\n
  30. Wouters, P., Thelwall, M., Kousha, K., Waltman, L., de Rijcke, S., Rushforth, A.,\u00a0&\u00a0et\u00a0al. (2014). The Citation: From Culture to Infrastructure. B.\u00a0 Cronin, C.R.\u00a0Sugimoto (Eds). Beyond Bibliometrics: Harnessing Multidimensional Indicators of Scholarly Impact. Cambridge, MA: MIT Press, 47\u201466. https:\/\/doi.org\/10.7551\/mitpress\/9445.003.0006<\/a><\/li>\n
  31. Hinze, S., Butler, L., Donner, P., & McAllister, I. (2019). Different Processes, Similar Results? A Comparison of Performance Assessment in Three Countries. Springer Handbook of Science and Technology Indicators. Cham: Springer, 465\u201484. https:\/\/doi.org\/10.1007\/978-3-030-02511-3_18<\/a><\/li>\n
  32. Kostoff, R.N. (1997). Handbook of research impact assessment. 7th ed. Arlington: NTIS.<\/li>\n
  33. Pontika, N., Klebel, T., Correia, A., Metzler, H., Knoth, P., & Ross-Hellauer, T. (2022). Indicators of Research Quality, Quantity, Openness, and Responsibility in Institutional Review, Promotion, and Tenure Policies across Seven Countries. Quantitative Science Studies, 3, 888\u2014911. https:\/\/doi.org\/10.1162\/qss_a_00224<\/a><\/li>\n
  34. Loktiev, V.M. (2021). Does basic science have a proper status in Ukraine? Visn. Nac. Akad. Nauk Ukr., 6, 52\u201458. https:\/\/doi.org\/10.15407\/visn2021.06.052<\/a> [in Ukrainian].<\/li>\n
  35. Samuel, G.N., & Derrick, G.E. (2015). Societal impact evaluation: Exploring evaluator perceptions of the characterization of impact under the REF2014. Research Evaluation, 24\u00a0(3), 229\u2014241. https:\/\/doi.org\/10.1093\/reseval\/rvv007<\/a><\/li>\n
  36. Jim\u00e9nez-Contreras, E., de Moya Aneg\u00f3n, F., & Delgado-L\u00f3pez-C\u00f3zar, E. (2003). The Evolution of Research Activity in Spain: The Impact of the National Commission for the Evaluation of Research Activity (CNEAI). Research Policy, 32, 123\u2014142. https:\/\/doi.org\/10.1016\/S0048-7333(02)00008-2<\/a><\/li>\n
  37. Feenstra, R., & L\u00f3pez-C\u00f3zar, E. (2023). The footprint of a metrics-based research evaluation system on Spain\u2019s philosophical scholarship: An analysis of researchers\u2019 perceptions. Research Evaluation, 32\u00a0(1), 32\u201446. https:\/\/doi.org\/10.1093\/reseval\/rvac020<\/a><\/li>\n
  38. Orfila, M. (2023). Spain wants to change how it evaluates scientists \u2014 and end the \u201cdictatorship of papers\u201d: Officials aim to use wider range of research outputs to assess researchers at public universities. Science, 29 November. https:\/\/doi.org\/10.1126\/science.zehwqlq<\/a><\/li>\n<\/ol>\n

    Full Text (PDF)<\/strong><\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

    VITAL ISSUES OF EVALUATING THE SOCIAL IMPACT OF SCIENTIFIC RESEARCH T.\u041e. YAROSHENKO 1 https:\/\/orcid.org\/0000-0002-2985-2333 O.I. YAROSHENKO 1 https:\/\/orcid.org\/0000-0002-4716-5705 1 National University of Kyiv-Mohyla Academy, Ukraine Nauka naukozn. 2024, 2(124): 52\u201482 https:\/\/doi.org\/10.15407\/sofs2024.02.052 Section: Scientometrics Language: Ukrainian Abstract: The sound assessment of scientific research has gained the increased topicality in the past decade due to the challenges […]<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":7239,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"translation":{"provider":"WPGlobus","version":"2.12.2","language":"en","enabled_languages":["uk","en","ru"],"languages":{"uk":{"title":true,"content":true,"excerpt":false},"en":{"title":true,"content":true,"excerpt":false},"ru":{"title":true,"content":true,"excerpt":false}}},"yoast_head":"\n1.4 - Science and Science of Science<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/sofs.org.ua\/sample-page\/arhiv\/2024-2\/2-124\/1-4\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"1.4 - Science and Science of Science\" \/>\n<meta property=\"og:url\" content=\"https:\/\/sofs.org.ua\/sample-page\/arhiv\/2024-2\/2-124\/1-4\/\" \/>\n<meta property=\"og:site_name\" content=\"Science and Science of Science\" \/>\n<meta property=\"article:modified_time\" content=\"2024-07-22T08:03:59+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"8 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/sofs.org.ua\/sample-page\/arhiv\/2024-2\/2-124\/1-4\/\",\"url\":\"https:\/\/sofs.org.ua\/sample-page\/arhiv\/2024-2\/2-124\/1-4\/\",\"name\":\"1.4 - Science and Science of Science\",\"isPartOf\":{\"@id\":\"https:\/\/sofs.org.ua\/#website\"},\"datePublished\":\"2024-06-21T17:44:15+00:00\",\"dateModified\":\"2024-07-22T08:03:59+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/sofs.org.ua\/sample-page\/arhiv\/2024-2\/2-124\/1-4\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/sofs.org.ua\/sample-page\/arhiv\/2024-2\/2-124\/1-4\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/sofs.org.ua\/sample-page\/arhiv\/2024-2\/2-124\/1-4\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/sofs.org.ua\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"\u041f\u0440\u043e \u0436\u0443\u0440\u043d\u0430\u043b\",\"item\":\"https:\/\/sofs.org.ua\/sample-page\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"\u0410\u0440\u0445\u0456\u0432\",\"item\":\"https:\/\/sofs.org.ua\/sample-page\/arhiv\/\"},{\"@type\":\"ListItem\",\"position\":4,\"name\":\"2024\",\"item\":\"https:\/\/sofs.org.ua\/sample-page\/arhiv\/2024-2\/\"},{\"@type\":\"ListItem\",\"position\":5,\"name\":\"\u21162 (124)\",\"item\":\"https:\/\/sofs.org.ua\/sample-page\/arhiv\/2024-2\/2-124\/\"},{\"@type\":\"ListItem\",\"position\":6,\"name\":\"1.4\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/sofs.org.ua\/#website\",\"url\":\"https:\/\/sofs.org.ua\/\",\"name\":\"Science and Science of Science\",\"description\":\"international scientific journal\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/sofs.org.ua\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"1.4 - Science and Science of Science","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/sofs.org.ua\/sample-page\/arhiv\/2024-2\/2-124\/1-4\/","og_locale":"en_US","og_type":"article","og_title":"1.4 - Science and Science of Science","og_url":"https:\/\/sofs.org.ua\/sample-page\/arhiv\/2024-2\/2-124\/1-4\/","og_site_name":"Science and Science of Science","article_modified_time":"2024-07-22T08:03:59+00:00","twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"8 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/sofs.org.ua\/sample-page\/arhiv\/2024-2\/2-124\/1-4\/","url":"https:\/\/sofs.org.ua\/sample-page\/arhiv\/2024-2\/2-124\/1-4\/","name":"1.4 - Science and Science of Science","isPartOf":{"@id":"https:\/\/sofs.org.ua\/#website"},"datePublished":"2024-06-21T17:44:15+00:00","dateModified":"2024-07-22T08:03:59+00:00","breadcrumb":{"@id":"https:\/\/sofs.org.ua\/sample-page\/arhiv\/2024-2\/2-124\/1-4\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/sofs.org.ua\/sample-page\/arhiv\/2024-2\/2-124\/1-4\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/sofs.org.ua\/sample-page\/arhiv\/2024-2\/2-124\/1-4\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/sofs.org.ua\/"},{"@type":"ListItem","position":2,"name":"\u041f\u0440\u043e \u0436\u0443\u0440\u043d\u0430\u043b","item":"https:\/\/sofs.org.ua\/sample-page\/"},{"@type":"ListItem","position":3,"name":"\u0410\u0440\u0445\u0456\u0432","item":"https:\/\/sofs.org.ua\/sample-page\/arhiv\/"},{"@type":"ListItem","position":4,"name":"2024","item":"https:\/\/sofs.org.ua\/sample-page\/arhiv\/2024-2\/"},{"@type":"ListItem","position":5,"name":"\u21162 (124)","item":"https:\/\/sofs.org.ua\/sample-page\/arhiv\/2024-2\/2-124\/"},{"@type":"ListItem","position":6,"name":"1.4"}]},{"@type":"WebSite","@id":"https:\/\/sofs.org.ua\/#website","url":"https:\/\/sofs.org.ua\/","name":"Science and Science of Science","description":"international scientific journal","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/sofs.org.ua\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/sofs.org.ua\/en\/wp-json\/wp\/v2\/pages\/7264"}],"collection":[{"href":"https:\/\/sofs.org.ua\/en\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/sofs.org.ua\/en\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/sofs.org.ua\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/sofs.org.ua\/en\/wp-json\/wp\/v2\/comments?post=7264"}],"version-history":[{"count":6,"href":"https:\/\/sofs.org.ua\/en\/wp-json\/wp\/v2\/pages\/7264\/revisions"}],"predecessor-version":[{"id":7370,"href":"https:\/\/sofs.org.ua\/en\/wp-json\/wp\/v2\/pages\/7264\/revisions\/7370"}],"up":[{"embeddable":true,"href":"https:\/\/sofs.org.ua\/en\/wp-json\/wp\/v2\/pages\/7239"}],"wp:attachment":[{"href":"https:\/\/sofs.org.ua\/en\/wp-json\/wp\/v2\/media?parent=7264"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}