{"id":5690,"date":"2022-06-30T07:06:15","date_gmt":"2022-06-30T05:06:15","guid":{"rendered":"https:\/\/sofs.org.ua\/?page_id=5690"},"modified":"2022-07-18T10:27:21","modified_gmt":"2022-07-18T08:27:21","slug":"1-6","status":"publish","type":"page","link":"https:\/\/sofs.org.ua\/en\/sample-page\/arhiv\/2022-2\/2-116\/1-6\/","title":{"rendered":"1.6"},"content":{"rendered":"

SCIENTOMETRIC<\/strong> AND EXPERT ASSESSMENT: DISCUSSION ISSUES<\/strong><\/p>\n

T.M. KARMADONOVA
\n<\/strong>https:\/\/orcid.org\/0000-0002-3384-6067
\n<\/a>Dobrov Institute for Scientific and Technological Potential and Science History Studies of the NAS of Ukraine<\/p>\n

Nauka naukozn. 2022, 2(116): 65\u201384
\nhttps:\/\/doi.org\/10.15407\/sofs2022.02.065<\/a><\/p>\n

Section<\/strong>: Scientometrics
\nLanguage:<\/strong> Ukrainian
\nAbstract:<\/strong> The article is devoted to topical issues of the research performance evaluation. Applications of scientometric and expert assessment methods in the research system of our times and related problems are analyzed. The texts of San Francisco Declaration on the Evaluation of Research and the Leiden Manifesto for Bibliometrics were used as documents that are in the focus of scientists\u2019 and public attention. It is emphasized that one of their main conclusions concerns with the inappropriateness of overestimating the significance of quantitative indicators and the need to consider for the multidimensionality of research objectives.<\/p>\n

Advantages and drawbacks of scientometric evaluation of the research performance are analyzed. Potentials and limitations of citation indices, Hirsch index, impact factor and other indices in performance assessment of researchers or research institutions and ranking of scientific journals are shown.<\/p>\n

The interpretation of \u201cexpert opinion\u201d the is given, with discussing main methods of expert assessment: Delphi method, brainstorming, synectics. Possible errors and shortcomings involved in their applications are highlighted. A review of expert assessment models is made: single blind peer review, double blind peer review and open peer review. It is shown that Internet environment techniques have become determinants of the transition from the model of closed pre-publication review to the model of open post-publication online review.<\/p>\n

Based on the by S. Funtowicz and J. Ravetz concept of post-normal science, the concepts of \u201cextended expert community\u201d and \u201cextended expert assessment\u201d are elaborated on, with emphasizing on that that they reflect public participation in science and technology assessment. It is argued that the extended expert assessment has to be arraged in a way to avoid ideological, political or religious bias. The rise of a new phenomenon of \u201cbeta reader\u201d in evaluating research texts is analyzed, with discussing several online platforms of \u201cbeta readers\u201d.<\/p>\n

It is concluded that qualitative and quantitative methods of assessment should be used as complementary analytical tools.<\/p>\n

Keywords<\/strong>: expert assessment, citation index, Hirsh index, impact factor, expert opinion, peer review, expert review, extended expert assessment, \u201cbeta reader\u201d.<\/strong><\/p>\n

References<\/strong><\/p>\n

    \n
  1. Horovy, V. (2015) Criteria for the quality of scientific achievements in the context of the security of national interests. Visn. Nac. Akad. Nauk Ukr., 6, 74\u201380 [in Ukrainian].<\/li>\n
  2. Kirichenko, I.V., & Shelyubskaya, N.V. (2019). The system for assessing the quality of scientific research in European countries. University Management: Practice and Analysis, 23(4), 9\u201320. https:\/\/doi.org\/10.15826\/umpa.2019.04.025 [in Russian].<\/li>\n
  3. Kostenko, L., & Simonenko, T. (2016). Scientometrics: from numerology to the Leiden Manifesto. Scientific Periodicles of Vernadsky National Library of Ukraine, 43, 285\u2013295 [in Ukrainian]. https:\/\/doi.org\/10.15407\/np.43.285<\/a><\/li>\n
  4. Malitsky, B., Ribachuk, V., Koretsky, A., & Popovich A. (2013). Scientometrics: new functions and problems of adequacy. Nauka innov., 1 (119), 11\u201317 [in Russian].<\/li>\n
  5. Didenko, Yu.V., & Radchenko, A.I. (2017). Publication activity as a way of scientific communication and pursuit of ratings. Visn. Nac. Akad. Nauk Ukr., 9, 82\u201398. https:\/\/doi.org\/10.15407\/visn2017.09.082<\/a> [in Ukrainian].<\/li>\n
  6. Rybachuk, V.P. (2013). Bibliometric portrait of academician Vladimir Ivanovich Vernadsky: fame in the world. Libraries of National Academies of Sciences: Problems of Functioning, Development trends, 11, 22\u201333 [in Russian].<\/li>\n
  7. Kavunenko, L.P., Khorevin, V.I., Kostrytsya, O.P., & Levchenko, O.G. (2010). Scientometric monitoring of scientific periodicals in the socio-humanitarian sphere of Ukraine. Science of Ukraine in the Global Information Space, 3, 71\u201379 [in Ukrainian].<\/li>\n
  8. Malitsky, B.A. (2017). Who and how should evaluate the scientific results of a scientist. Science and Science of Science, 3, 34\u201353. https:\/\/doi.org\/10.15407\/sofs2017.03.034<\/a> [in Russian].<\/li>\n
  9. Moskaleva, O.V. (2013). The use of scientometric indicators for the evaluation of scientific activity. Science Policy Research, 13, 85\u2013109 [in Russian].<\/li>\n
  10. Vyalkov, A.I., & Glukhova E.A. (2013). Evaluation of the quality of scientific research activities of a medical organization using scientometric indicators. Healthcare of the Russian Federation, 3, 3\u20136 [in Russian].<\/li>\n
  11. Kostenko, L.I., Simonenko, T.V., Grachev, O.A., & Rybachuk, V.P. (2017). Bibliometrics of domestic science: opportunities and limitations of the application of the Google Scholar web system. Science and Science of Science, 3, 87\u201396 [in Russian]. https:\/\/doi.org\/10.15407\/sofs2017.03.087<\/a><\/li>\n
  12. Aleksinska, M. (2016). Beware the index! Why international ratings cannot properly assess the Ukrainian labor market. Economic policy. URL: https:\/\/voxukraine.org\/osterigaitesreityngiv-ua\/ (last accessed: 01.21.2022) [in Ukrainian].<\/li>\n
  13. Balatsky, E.V., Ekimova, N.A., & Tretyakova O.V. (2021). Methods for assessing the quality of scientific economic journals. Journal of Institutional Studies, 2, 27\u201352 [in Russian]. https:\/\/doi.org\/10.17835\/2076-6297.2021.13.2.027-052<\/a><\/li>\n
  14. Obolkina, S.V., & Popova, N.G. (2019). Expertise vs peer review: conceptual foundations of competencies. Sociology of Science and Technology, 4, 38\u201350 https:\/\/doi.org\/10.24411\/2079-0910-2019-14003<\/a> [in Ukrainian].<\/li>\n
  15. Maslennikov, E.V. (2017). The possibility of using expert knowledge as a source of concepts for the development of organizations. Bulletin of the Moscow University, 2, 229\u2013249 [in Russian]. https:\/\/doi.org\/10.24290\/1029-3736-2017-23-2-229-249<\/a><\/li>\n
  16. Zhuravleva, V.A. (2012) Expert assessment method: historical explication and modern model. Bulletin of Peoples\u2019 Friendship University of Russia. Series: Sociology, 2, 28\u201338 [in Russian].<\/li>\n
  17. Pavliuk, K.V. (2019). Problems of evaluation of scientific activity. RFI Scientific Papers, 4, 5\u201319. https:\/\/doi.org\/10.33763\/npndfi2019.04.005<\/a> [in Ukrainian].<\/li>\n
  18. Zhenchenko, M.I. (2016). New model of reviewing scientific publications in the digital environment. State and Regions. Series: Social Communications, 1, 169\u2013172 [in Ukrainian].<\/li>\n
  19. Gerasimenko, A.G., Mazaraki, N.A., & Duginets, G.V. (2019). Reviewing as a tool for promoting social and economic achievements. Economic Space, 141, 25\u201335. https:\/\/doi.org\/10.30838\/P.ES.2224.100119.25.343<\/a> [in Ukrainian].<\/li>\n
  20. Funtowicz, S.O., & Ravetz, J.R. (1991). A new scientific methodology for global environmental issues. Ecological economics: The science and management of sustainability. New York: Columbia Univ. Press.<\/li>\n
  21. Bucchi, M., & Trench, B. (2018). Handbook of Public Communication of Science and Technology. Trans. from English. Moscow: Alpina non-fiction [in Russian].<\/li>\n
  22. Jenkins, H. (1958). Convergence Culture: Where Old and New Media Collide. NYU Press.<\/li>\n
  23. Hellekson, K., & Busse, K. (2006). Fan Fiction and Fan Communities in the Age of the Internet: New Essays. McFarland.<\/li>\n
  24. Grigoriev, V.E. (2018). Sociology of science. Moscow: Prospekt [in Russian].<\/li>\n
  25. Bornmann, L. & Daniel, H.-D. (2005). Does the h-index for ranking of scientists really work? Scientometrics, 65(3), 391\u2013392. https:\/\/doi.org\/10.1007\/s11192-005-0281-4<\/a><\/li>\n
  26. Shostak, A.V., Lukach, V.S., Boris, M.M., & Kupchuk, I.M. (2012). Hirsch index and impact factor as a tool for scientometrics at the previous university. Collection of Scientific Works of the Vinnitsa National Agrarian University, 11(65), 375\u2013380 [in Ukrainian]<\/li>\n
  27. Chaikovskiy, Yu., Silkina, Yu., & Pototska, O. (2013). Scientometric bases and their quantitative indicators (Part I. Comparative characteristics of scientometric bases). Visn. Nac. Akad. Nauk Ukr., 8, 90\u201394 [in Ukrainian].<\/li>\n
  28. Popovich, O. (2020). Scientometric ignorance (the mania of bureaucracies in implementing digital assessment of R&D). Granite of Science. URL: https:\/\/un-sci.com\/ru\/2020\/02\/06\/ naukometrichne-neviglastvo-shhodo-mani\u0457-byurokratij-zaprovaditi-czifrove-oczinyuvannya-nauki\/ (last accessed: 21.01.2022) [in Ukrainian].<\/li>\n
  29. Currie, R.R., & Pandher, G.S. (2020). Finance journal rankings: Active scholar assessment revisited. Journal of Banking & Finance, 111, 1\u201314. https:\/\/doi.org\/10.1016\/j.jbankfin.2019.105717<\/a><\/li>\n
  30. Jacalyn Kelly, Tara Sadeghieh, & Khosrow Adeli (2014). Review in Scientific Publications: Benefits, Critiques, & A Survival Guide. The Journal of Internaitonal Federation of Clinical Chemistry and Laboratory Medicine, 25(3), 227\u2013243. URL: https:\/\/www.ncbi.nlm.nih. gov\/pmc\/articles\/PMC4975196\/ (last accessed: 01\/21\/2022).<\/li>\n
  31. Funtowicz, S., & Ravetz, J. (1993). Science for the post-normal age. Futures, 25(7), 739\u2013755. https:\/\/doi.org\/10.1016\/0016-3287(93)90022-L<\/a><\/li>\n<\/ol>\n

    Full Text (PDF)<\/strong><\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

    SCIENTOMETRIC AND EXPERT ASSESSMENT: DISCUSSION ISSUES T.M. KARMADONOVA https:\/\/orcid.org\/0000-0002-3384-6067 Dobrov Institute for Scientific and Technological Potential and Science History Studies of the NAS of Ukraine Nauka naukozn. 2022, 2(116): 65\u201384 https:\/\/doi.org\/10.15407\/sofs2022.02.065 Section: Scientometrics Language: Ukrainian Abstract: The article is devoted to topical issues of the research performance evaluation. Applications of scientometric and expert assessment methods […]<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":5672,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"translation":{"provider":"WPGlobus","version":"2.12.2","language":"en","enabled_languages":["uk","en","ru"],"languages":{"uk":{"title":true,"content":true,"excerpt":false},"en":{"title":true,"content":true,"excerpt":false},"ru":{"title":true,"content":true,"excerpt":false}}},"yoast_head":"\n1.6 - Science and Science of Science<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/sofs.org.ua\/sample-page\/arhiv\/2022-2\/2-116\/1-6\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"1.6 - Science and Science of Science\" \/>\n<meta property=\"og:url\" content=\"https:\/\/sofs.org.ua\/sample-page\/arhiv\/2022-2\/2-116\/1-6\/\" \/>\n<meta property=\"og:site_name\" content=\"Science and Science of Science\" \/>\n<meta property=\"article:modified_time\" content=\"2022-07-18T08:27:21+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"7 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/sofs.org.ua\/sample-page\/arhiv\/2022-2\/2-116\/1-6\/\",\"url\":\"https:\/\/sofs.org.ua\/sample-page\/arhiv\/2022-2\/2-116\/1-6\/\",\"name\":\"1.6 - Science and Science of Science\",\"isPartOf\":{\"@id\":\"https:\/\/sofs.org.ua\/#website\"},\"datePublished\":\"2022-06-30T05:06:15+00:00\",\"dateModified\":\"2022-07-18T08:27:21+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/sofs.org.ua\/sample-page\/arhiv\/2022-2\/2-116\/1-6\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/sofs.org.ua\/sample-page\/arhiv\/2022-2\/2-116\/1-6\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/sofs.org.ua\/sample-page\/arhiv\/2022-2\/2-116\/1-6\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/sofs.org.ua\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"\u041f\u0440\u043e \u0436\u0443\u0440\u043d\u0430\u043b\",\"item\":\"https:\/\/sofs.org.ua\/sample-page\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"\u0410\u0440\u0445\u0456\u0432\",\"item\":\"https:\/\/sofs.org.ua\/sample-page\/arhiv\/\"},{\"@type\":\"ListItem\",\"position\":4,\"name\":\"2022\",\"item\":\"https:\/\/sofs.org.ua\/sample-page\/arhiv\/2022-2\/\"},{\"@type\":\"ListItem\",\"position\":5,\"name\":\"\u21162(116)\",\"item\":\"https:\/\/sofs.org.ua\/sample-page\/arhiv\/2022-2\/2-116\/\"},{\"@type\":\"ListItem\",\"position\":6,\"name\":\"1.6\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/sofs.org.ua\/#website\",\"url\":\"https:\/\/sofs.org.ua\/\",\"name\":\"Science and Science of Science\",\"description\":\"international scientific journal\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/sofs.org.ua\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"1.6 - Science and Science of Science","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/sofs.org.ua\/sample-page\/arhiv\/2022-2\/2-116\/1-6\/","og_locale":"en_US","og_type":"article","og_title":"1.6 - Science and Science of Science","og_url":"https:\/\/sofs.org.ua\/sample-page\/arhiv\/2022-2\/2-116\/1-6\/","og_site_name":"Science and Science of Science","article_modified_time":"2022-07-18T08:27:21+00:00","twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"7 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/sofs.org.ua\/sample-page\/arhiv\/2022-2\/2-116\/1-6\/","url":"https:\/\/sofs.org.ua\/sample-page\/arhiv\/2022-2\/2-116\/1-6\/","name":"1.6 - Science and Science of Science","isPartOf":{"@id":"https:\/\/sofs.org.ua\/#website"},"datePublished":"2022-06-30T05:06:15+00:00","dateModified":"2022-07-18T08:27:21+00:00","breadcrumb":{"@id":"https:\/\/sofs.org.ua\/sample-page\/arhiv\/2022-2\/2-116\/1-6\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/sofs.org.ua\/sample-page\/arhiv\/2022-2\/2-116\/1-6\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/sofs.org.ua\/sample-page\/arhiv\/2022-2\/2-116\/1-6\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/sofs.org.ua\/"},{"@type":"ListItem","position":2,"name":"\u041f\u0440\u043e \u0436\u0443\u0440\u043d\u0430\u043b","item":"https:\/\/sofs.org.ua\/sample-page\/"},{"@type":"ListItem","position":3,"name":"\u0410\u0440\u0445\u0456\u0432","item":"https:\/\/sofs.org.ua\/sample-page\/arhiv\/"},{"@type":"ListItem","position":4,"name":"2022","item":"https:\/\/sofs.org.ua\/sample-page\/arhiv\/2022-2\/"},{"@type":"ListItem","position":5,"name":"\u21162(116)","item":"https:\/\/sofs.org.ua\/sample-page\/arhiv\/2022-2\/2-116\/"},{"@type":"ListItem","position":6,"name":"1.6"}]},{"@type":"WebSite","@id":"https:\/\/sofs.org.ua\/#website","url":"https:\/\/sofs.org.ua\/","name":"Science and Science of Science","description":"international scientific journal","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/sofs.org.ua\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/sofs.org.ua\/en\/wp-json\/wp\/v2\/pages\/5690"}],"collection":[{"href":"https:\/\/sofs.org.ua\/en\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/sofs.org.ua\/en\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/sofs.org.ua\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/sofs.org.ua\/en\/wp-json\/wp\/v2\/comments?post=5690"}],"version-history":[{"count":5,"href":"https:\/\/sofs.org.ua\/en\/wp-json\/wp\/v2\/pages\/5690\/revisions"}],"predecessor-version":[{"id":5823,"href":"https:\/\/sofs.org.ua\/en\/wp-json\/wp\/v2\/pages\/5690\/revisions\/5823"}],"up":[{"embeddable":true,"href":"https:\/\/sofs.org.ua\/en\/wp-json\/wp\/v2\/pages\/5672"}],"wp:attachment":[{"href":"https:\/\/sofs.org.ua\/en\/wp-json\/wp\/v2\/media?parent=5690"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}