UDC 001.81:004.65

VLASHCHENKO L. H.

Science Library, Kharkiv National University of Radio Electronics (Kharkiv, Ukraine),

e-mail liudmyla.vlashchenko@nure.ua, ORCID 0000-0003-2329-9503

NIKITENKO O. M.

Metrology and Technical Expertise Department, Kharkiv National University of Radio Electronics (Kharkiv, Ukraine), e-mail nikonxipe@gmail.com, ORCID 0000-0002-1082-5247

MODELS TO EVALUATE OF SCIENTISTS' RATING

Objective. This publication purpose is to review the models by which we can evaluate the scientists' ranking. Methods. Two models to estimate scientist rating was described. One model proposes to use data from international scientometric databases to evaluate the scientists rating. Three approaches using to estimate rating was shown. Formula calculations have been developed for two of these approaches. The example of scientist rating calculation according to data from the international scientometric database Scopus is given. Another model proposes to use the TRUST National Higher Education Quality Assurance Portal to estimate the higher school's rating. Results. This allows you to store information about publications and other achievements of the scientist, including full texts, to estimate scientists rating using more factors and to design necessary output forms for later use. Conclusions. The first model can be used to evaluate the scientists rating in a higher education institution using the Web of Science and Scopus science databases at the current time. The second model allows to store information about publications and other achievements of the scientist, including full texts.

Keywords: scientometric database; researcher's rating; scientometric indicator; TRUST

Introduction

Almost every university employee participates in different scientific researches.

At the present development stage on the one hand the leadership of Ukrainian universities seeks to best serve the achievements of their institutions, and on the other hand, every scientist also tries to submit his achievements both at the national and international levels.

The leaders of domestic higher education institutions are guided by western models and are trying to bring science back to Ukrainian universities.

Such efforts are confronted with a number of national problems, the most striking among them: limited funding, weak scientists' integration from Ukrainian universities into the world scientific space, localities of the vast majority of Ukrainian scientific publications and the complexity of conducting an appropriate expert assessment of the scientific work results of university employees. Actually, at first, for the addition of the latter, and then for other important decisions in the process of managing research activities in institutions, the leaders of many Ukrainian universities began to use scientometric indicators.

Accounting the scientometric factors by all authoritative world rankings of university such as Academic Ranking of World Universities, Times Higher Education World University Rankings and QS World University Ranking, has become an additional factor in the popularity growth of the practical application of scientometric in the domestic higher education. Leading international university rankings measure the publishing activity of university employees on the basis of indicators of scientometric computer platforms Web of Science Core Collection from the corporation Thompson Reuters and Scopus owned by the publishing corporation Elsevier. Therefore, the leading Ukrainian universities are guided by the presentation of the scientific results of activity precisely at the bases of these two platforms (Hryshchenko & Nikitenko, 2017; Nikitenko & Plechenko, 2017), although this is not enough.

This publication purpose is to review the models by which we can evaluate the scientists' ranking.

Methods

In order to stimulate the growth of scientific productivity and display its results in the mentioned scientometric resources, the leadership of many domestic universities introduced a ranking calculation for publications' authors. It is clear that in the Web of Science and Scopus databases different types of documents are indexed that have different scientific values, so it is not expedient to determine the ratings only for the mentioned indicators. Therefore, in many universities, additional scientometric factors are introduced for the rating calculation. As a rule, the impact factor of the journal where the article was published, is taken into account.

It should be noted separately that impact-factors count exclusively for magazines. However, in some scientific disciplines, not only magazines play the most important channel of scientific communication (for humanities – these are scientific monographs, and for the natural sciences and IT sciences – the conference materials which occupy a prominent place) (Nazarovets, 2016).

Another significant drawback is the use of impact factors to calculate the ratings of scientists – not taking into account the number of co-authors of publications. Today, most of the scientific works in the world are written in co-authorship.

To solve this problem, there are several approaches that typically use a variety of normalized science metrics (Nazarovets, 2016).

In this regard Science library of Kharkiv national university of radio electronics proposed such approaches for determining the ranking of scientists with the help of scientometric indicators:

• total number of publications;

• calculation of marks for publications, taking into account the type of publication: journal article or proceeding one;

• the calculation of marks for publication, taking into account the type of publication and part of the contribution to the publication of each co-author.

The first approach is the easiest. But this approach does not take into account the type of publication and the number of co-authors of the publication. This, on the one hand, leads to a leveling of the time spent on creating a particular publication. But it can shift the direction of publications to the creation of abstracts due to less time spent on their preparation and less stringent requirements for registration. On the other hand, in general, each publication will be taken into account as many times as it co-sponsors.

The second approach requires the introduction and justification of weighting factors for each type of publication. Based on the order № 200 on 15.7.2015 "On the time limits for planning and accounting of educational, methodological, scientific, organizational and educational work of scientific and pedagogical workers of Kharkiv National University of Radio Electronics" the scientific library proposed as weight coefficients to choose the meaning of normative hours for the implementation of scientific research: 70 points per article in the journal and 20 points for the thesis of reports at the conference.

In this approach of rating the scientists we proposed the calculation of the rating to perform as follows:

where n – the number of articles;

m – number of report theses.

But in this case, as in the previous approach, each publication will be taken into account as many times as it co-authors.

The third approach, besides taking into account the type of publication, also requires determining the portion of the contribution of each co-author to create a publication. Because those who rate the ranking of scholars are not related to the publication, they can not evaluate the contribution of each of the co-authors. In this regard, we are suggested to consider that each of the co-authors of the publication makes identical efforts to create it. Consequently, taking into account the above, one can propose the following formula for calculating the ranking of scholars:

where a_i – the number of co-authors of a particular article;

р_j – number of co-authors of a particular report.

As an example, we present the results of computing the scientific metrics for one employee who entered the top10 of the university according to the Scopus science-based database.

• Total number of publications: 45

• Of these articles - 7, reports - 38

• Separate articles and reports based on weighting factors:

7×70=490

38×20=760

Total value: 1250

• Separate articles and reports with co-authors: 43/12 1111/60

Separate articles and reports taking into account co-authors and weighting coefficients: 1505/6 1111/3

• Total value: 3727/6 ≈ 621

In order to compare the scores obtained from different approaches to evaluation, four authors with ten publications were selected from Scopus's science-research database.

Results and Discussion

Figure shows the calculation results of scientometric factors according to the described approaches. The total publication number in the picture is blue. The calculation of the second approach (indicated by red) showed that only taking into account articles and reports with different weight coefficients can differentiate the overall scientific achievement of the author. The application of the third approach provided an opportunity to more precisely determine the contribution of a separate scientist.

Figure. Publication activity indicators taking into account types of publications and co-authors

By the value of aggregate indicators, you can determine the rankings of the authors of the publications.

Thus, the calculation based on the above formulas allows to take into account not only the number of articles, but also the number of co-authors, as well as to separate articles from the abstracts.

Unfortunately, the proposed model takes into account only those publications that are reflected in the abovementioned international bases. University scholars have publications that are not reflected in these science databases. In addition to scientific publications, the University staff is involved in many activities that are not directly related to scientific publications.

For this purpose, it is desirable to be able to obtain an up-to-date objective assessment of their quality and achievements. This is the purpose of the TRUST National Higher Education Quality Assurance Portal. A comprehensive description of the portal concept is available at http://www.cs.jyu.fi/ai/Quality.

The TRUST portal is a publicly available personal portfolio manager for educators and educational institutions. Its users are both education sector representatives and end their work users.

The portal is a tool for public control and influence on the higher education quality. It aims to help educators openly share their own teaching and scientific achievements, and society to evaluate them on an up-to-date values scale (O. L Shevchenko, Horobets, O. Yu. Shevchenko, & Sokol, 2014). The TRUST National Higher Education Quality Portal is the only point of access to the cloud repository of entities providing educational services and their achievements and supporting documents. It is also a tool for generating and storing personalized ratings of registered resources, as well as storing, reviewing and comparing the value systems on which such ratings were or can be constructed. A convenient portal feature is also the ability to dynamically generate CVs and personal web pages of educational entities based on their achievements recorded in the portal.

Conclusions

So, the portal gives the average user (academic employee) the following opportunities:

• from any point of the globe to have unlimited 7/24/365 on-line access to information both about themselves and about other educational resources registered in the portal;

• maintain, manage, and dynamically generate and update self-generated information about your CVs, reports and personal web pages;

• search for registered educational resources, not even having the full resource name, but only it fragments, thanks to the Semantic Web technology used;

• find the right resources by category using the advanced filter;

• to find experts in the relevant field through registered achievements (articles, grants, authoring courses, etc.). For all articles registered in the portal, the impact factor is calculated based on the publication level where the article is published. The publication level is taken from an external resource (source) recommended by European experts. It was a international publications’ list, carefully compiled by the Finnish Academy of Sciences and recommended for publications. European experts regularly update this list, leading to its automatic updating in the portal. The list includes about nineteen-and-a-half thousand editions (scientific journals and conference reports) summarized in the Scopus and Web of Science databases with relevant impact factors. According to the recommendations of the Finnish Academy of Sciences, all gains have their level (1 – 3). Level 4 added – lowest, recommended for publications not included in the list by European experts.

For educational institutions, the portal provides the following features:

• access to information about registered educational resources: about employees, their achievements, not collected in the form of statistics, but with the ability to view and evaluate any achievement, including supporting documents;

• the possibility of transparent processes management, in the result of which the best indicators should be determined by the selected criteria, that is, those aimed at ranking: selection of candidates for the vacant position, best employees’ motivation, units, material resources’ distribution;

• the ability to flexibly adjust the rating system based on current goals and objectives, as well as save value systems and compare the results of ratings made at different times by different indicators to track the change dynamics. This capability is the basis for the internal quality assurance system in defining the unit / university development strategy and the quality assurance system's compliance with the chosen strategy, its defined goals, objectives and success criteria;

• finding experts in the relevant field, determining the scientist's work level based on reported achievements, including articles and reports at conferences, the impact factor of which is calculated by the above principle.

And much more, based on the analytical processing of the information registered in the portal.

The portal is built on the social system principles: portal users are the main information providers and consumers, its reliability controllers and its content evaluators.

So, we describe two models that can be used to evaluate the scientists rating.

The first model can be used to evaluate the scientists rating in a higher education institution using the Web of Science and Scopus science databases at the current time.

The second model allows to store information about publications and other achievements of the scientist, including full texts. This allows you to evaluate the scientist's rating on a larger number of parameters and create the necessary source forms for their further use.

REFERENCES

Hryshchenko, T. B. & Nikitenko, O. M.  (2017). Veznachennia rejtyngu naukovtsiv universytetu: try pidkhody. Informatsionnye sistemy i tekhnologii: Matyerialy 6-y Mezhdunarodnoy nauchno-tekhnicheskoy konferentsii, Koblevo, 11-16 sentyabrya 2017. Kharkiv, 243-244. Retriеved from http://openarchive.nure.ua/bitstream/document/4364/1/Konf_IST_2017.pdf (in Ukranian)

Nazarovets, S. (2016). Kvartylnyi pidkhid do problemy stymuliuvannia rostu naukovoi produktyvnosti v universytetakh. Forum. Bibliotechnyj forum: istoriya, teoriya i praktyka, 2(4), 10-12. Retriеved from http://ekmair.ukma.edu.ua/bitstream/handle/123456789/9168/nazarovets_quartile.pdf?sequence=1 (in Ukranian)

Nikitenko, O. M. & Plechenko, N. V. (2017). Otsiniuvannia rejtyngu naukovtsiv za vidjbrazhenniam u naukometrychnykh bazakh. Vymiriuvannia v bibliotekakh: otsinka efektyvnosti ta yakosti roboty Internet-konferentsiia, 6-10 chervnia 2017, Kharkiv. Retrieved from http://openarchive.nure.ua/bitstream/document/3656/4/NB_NURE_Nikitenko_Plechenko.pdf (in Ukranian)

Shevchenko, O. L., Horobets, Yu. M., Shevchenko, O. Yu. & Sokol V. V. (2014). Natsionalnyj portal zabezpechennia vyshchoji osvity: TRUST: tempus-proekt 516935-TEMPUS-1-2011-1-FITEMPUS-SMGR. Lviv: FOP Konkevych O. S. (in Ukranian)

ВЛАЩЕНКО Л. Г.

Наукова бібліотека, Харківський національний університет радіоелектроніки (Харків, Україна), e-mail liudmyla.vlashchenko@nure.ua, ORCID 0000-0003-2329-9503

НІКІТЕНКО О. М.

Каф. «Метрологія та технічна експертиза», Харківський національний університет радіоелектроніки (Харків, Україна), e-mail nikonxipe@gmail.com, ORCID 0000-0002-1082-5247

МОДЕЛІ Оцінювання рейтингу науковців

Мета. За основну мету цієї публікації ми ставимо огляд моделей, за якими можна здійснювати оцінювання рейтингу науковців. Методика. Розглянуто дві моделі оцінювання рейтингу науковців. За першою моделлю запропоновано використовувати дані з міжнародних наукометричних баз даних. Показано використання трьох підходів під час оцінювання рейтингу. Для двох із підходів розроблено формули обчислення рейтингу. Наведено приклад обчислення рейтингу науковця за даними з міжнародної наукометричної бази даних Scopus. За іншою моделлю для оцінювання рейтингу науковців запропоновано використовувати національний портал забезпечення якості вищої освіти TRUST. Результати. Розглянуті моделі дозволяють зберігати інформацію про публікації та інші досягнення науковця, включаючи повні тексти, оцінювати рейтинг науковця за більшою кількістю параметрів і створювати необхідні вихідні форми для подальшого їх використання. Висновки. За першою моделлю можна оцінювати рейтинг науковців певного закладу вищої освіти за наукометричними базами даних Web of Science та Scopus на поточний момент. Друга модель дозволяє зберігати інформацію про публікації та інші досягнення науковця, включаючи повні тексти.

Ключові слова: наукометрична база даних; рейтинг науковця; наукометричний показник