New methods to quantify an individual's scientific research output based on h-index and various factors

Transcription

1 Qualitative and Quantitative Methods in Libraries (QQML) 5: , 2016 New methods to quantiy an individual's scientiic research output based on h-index and various actors Weian Zhao, Wenjun Liu, and Ruiyu Zhang College o mathematics and statistics, Nanjing University o Inormation Science and Technology, Nanjing , China Abstract: Hirsch proposed h-index in To overcome some limitations o h-index in practical application, many scholars have proposed a series o derivative indexes. So ar, derivative indexes designed by the scholars are mostly based on the citations and total number o papers. Although they are simple to use, they have certain oneness on the data source. To overcome this situation, two methods are used in this paper to analyze various actors. Firstly, we use the principal component analysis method to analyze original data such as the total number o papers, citation and so on, and then draw a comprehensive evaluation result o Z to quantiy scientiic research output, which avoid the oneness on the actor. Then, based on the "Gold priority" rule and h -index proposed by Xu, we introduce a new comprehensive scientiic research evaluation index, named as h - index, to enhance the degree o dierentiation and sensitivity o h-index. For two methods above, examples are given respectively. Keywords: h-index, principal component analysis, a h -index 1. Introduction Jorge Hirsch (2005), an American physicist at the University o Caliornia, San Diego, designed a new evaluation index (h-index) in 2005 to characterize the scientiic output o a researcher by measuring the impact o the scientist s publications in terms o the received citations.. As soon as h-index was proposed, h-index obtained the widespread attention o the international scientiic community and became an international hotspot issue in the ield o scientiic metrology and science evaluation. Moreover, it is widely used to quantiy the research personnel, scientiic research institution, science and technology periodical. As a combination o publications and paper quality, it not Received: Accepted: ISSN ISAST

2 222 Weian Zhao, Wenjun Liu, and Ruiyu Zhang only possesses some advantages that the single index incomparable, but also has its own diiculty to overcome in the practical application. So ar, many scholars proposed a series o derivative index to enhance the academic evaluation o accuracy or h-index rom dierent aspects. Braun (2006) applied h-index to measure academic periodicals. Egghe, the amous science metrologist, thought that h-index should have relected these highly cited papers (2006), and thus he designed g-index in At that same year, Kosmulski (2006) putted orward h2 -index, which ocused on highly cited papers. In 2007, in et al (2007)] introduced the concept o "h-core", in which the amount o citation requency is located in the top h. In addition, to overcome the lack o the degree o sensitivity, dierentiation and volatility o h-index, they also introduced A-index, R-index and AR-index. However, h-index, as a perormance evaluation method, its score values oten gathered in the low score. To overcome this limitation, Xuemei Zhang (2007) putted orward h -index. In 2008, Wu (2008) proposed W-index, which inherits the advantage o h-index such as simpliication and easy to understand characteristics. W-index paid more attention to the inluence o the highly cited papers while h-index only calculated the number o highly cited papers. h -index, proposed by Anderson (2008) in the same year, took total citations into account to overcome the lack o dierentiation o the h-index. In 2009, Ye (2009) introduced -index, which was a new academic ranking index inspired by h-index. The index was linked to the subject, it combined quantity and quality, and suitable or disciplines, countries, institutions, periodicals, and Multidimensional academic scholars. In the same year, Xu (2009) proposed a more extensive scope o h -index under the enlightenment o the "Gold priority rule, the index considered the size o the core o perormance, measured the strength o the core o perormance, improved the accuracy o index, and implemented the comparability between the scholars with the same h-index. In 2010, Alonso (2010) designed hg-index, used in the measure to the scientiic research output, which not only considered highly cited papers, but also signiicantly reduced the eects o the highly cited papers. So, it can achieve a balance between most high quality papers and highly cited papers o the author. So ar, the indexes proposed by the scholars are mostly based on citations and the total number o papers. Although they are simple and convenient to use, it is diicult to know which index is more reasonable to quantiy some scholars scientiic research output in many cases. This paper aims to adopt some mathematical methods to analyze a variety o actors. Firstly, principal component analysis will be adopted to analyze various actors. Then, considering that there is a little complexity and complicated operation to apply the irst method, a new index named as h -index was proposed under the enlightenment o "Gold priority" rule and ha -index. It s shown that the proposed new index could improve the degree o dierentiation and sensitivity o h-index in a certain extent, and can be simple to use. t a m

3 Qualitative and Quantitative Methods in Libraries (QQML) 5: , This work will not only overcome some deects o the existing index in a certain extent, but also avoid the oneness when use citation requency and the total number o papers as the main basis to quantiy an individual s scientiic research output, thereby making the evaluation more air and reasonable. 2. A new method based on Principal Component analysis with multi-actor 2.1. Principal component analysis Principal component analysis (1901) was introduced or non-random variables by Karl Pearson in And then in 1933, Hotelling (1933) extend it to the random vectors. Generally, a ew new variables which are less than the number o original variables can explain variation o most o the data was selected, called principal component and a comprehensive index to explain the data. Thus, the principal component analysis is actually a dimension reduction method. It has the ollowing advantages: First, it used a ew comprehensive variables to replace the original multiple variables by dimension reduction techniques, most o these comprehensive variables ocused on the most inormation about the original variables. Secondly, it scored by calculating the comprehensive principal component unction, scientiically evaluated objective economic phenomena. Again, it ocused on the comprehensive evaluation o inormation contributions inluence on the application Method based on Principal Component analysis with multi-actor Since h-index was proposed, h-index was widely used to evaluate the researchers, research institutions and scientiic journals. Given the many shortcomings o the h-index, scholars proposed a series o derivative index. Due to the big dierence in rankings between h-index and other indexes, characteristics o the paper should be taken into account or more concrete analysis. Then it could be more just and reasonable when we measure the study perormance o scientiic researchers. For most o the evaluation index, researchers proposed them mostly based on citations. These indexes had certain limitations when calculating. What s more, it was a single source o data, ignored the characteristics o the paper. In this section, we would analyze a variety o actors (Issued amount, total citations and download requency, etc.) and using some mathematical methods to get a comprehensive evaluation value. This approach can to some extent avoid the oneness o using citations as the main basis o evaluation. As the principal component analysis method can transorm the number o correlation highly variables into each independent or irrelevant variables, we will adopt principal component analysis method to analyze the various actors Examples o application The database The subsequent analysis is based on 20 well-known scholars papers cited situation (in descending order o h-index). The data comes rom the paper which

4 224 Weian Zhao, Wenjun Liu, and Ruiyu Zhang entitled h-index research review and the empirical statistical analysis (2009) by Wei and Song. It is collected rom 1998~2008 Chinese journal ulltext database (see Appendix) The data analysis Observing rom the data in the Appendix, we notice that h-index and other indicators (total citations, issued amount, etc.) is not a simple linear correlation. Thus the index cannot substitute each other. To test this idea, the Pearson correlation coeicients between six indicators were obtained, and the results are as ollows (see Table 1). Table 1. The Correlation Matrix Issued amount Issued amount h-index Total citations The average citation rate Downloa d requency Browse requency h- index Total citation s The average citation rate Downloa d requenc y Browse requenc y From Table 1, we can know the presence o some correlation between some indicators, such as the correlation between h-index and total requency is , the correlation is strong. I we don t deal with the data, that is using directly, there would be a certain cross-resistance on inormation aspect, thus aecting the objectivity o comprehensive evaluation value. Considering the advantages o principal component analysis method, to transer a plurality o indicators into less number, we use it to quantiy comprehensively in this case. Principal component analysis is carried out on the six evaluation indexes, the characteristic root and its contribution rate o the correlation coeicient matrix are in Table 2.

5 No Qualitative and Quantitative Methods in Libraries (QQML) 5: , Table 2. The Characteristic Root and its Contribution Eigenvalue Contribution rate (%) Cumulative contribution rate (%) According to the data in Table 2, the cumulative contribution rate o the irst three characteristic value is 93%, more than 90%, which shows that the eect o the principal component analysis is very good. So the irst three principal components are selected or scientiic research and comprehensive evaluation below. The coeicient in ront o the standardized variable is the corresponding eigenvector o standardization sample principal component, and then principal component unction o three standardization sample can be obtained. y x x x x x x 1 y x x x 3 y x x x x x x6 With three principal component contribution rate or weight respectively, we reconstruct the principal components Comprehensive Assessment Model: Z y y y3 Plugging each principal component value into the above ormula or computing, comprehensive evaluation value and ranking results o each scholar are shown in Table x x x Table scholars comprehensive evaluation value and ranking results Name Suning Canhua Hong Huo Ren Fuman Gong You Chen Xiao Z Rank Name Yiwen Daming Weiguo Lijun Fan Tao Shao Zhu Xu Yang Z Rank Name Qianghui Chengu Xingyong Zuoxin Yuan Yao Ran Wu Zhang Zhou 6

6 226 Weian Zhao, Wenjun Liu, and Ruiyu Zhang Z Rank Name Chucai Wenshen Meixiang Tiecheng Yulin Cai Xiong Xu Zhu Jin Z Rank Results analysis Through Pearson correlation analysis between Z and other actors, the correlation coeicient Matrix can be acquired (see Table 4). Table 4. The Correlation Coeicient between Z and other actor Issued Total The average Download Browse h-index amount citations citation rate requency requency According to the Table 4, the correlation coeicient between Z and above actors is not very large, it have not reach 90%. Observing the results in Table 1, the correlation coeicient between h-index and issued amount is , while the correlation coeicient between h-index and total citation is What s more, there is a relatively strong correlation between Z and h-index while the correlation coeicient is in Table 4. However, the correlation coeicient between Z and issued amount is , while the correlation coeicient between Z and total citation is The comparison shows that the comprehensive evaluation value Z to a certain extent weakened the contribution o citation, so as to emphasize the contribution o other actors. However, considering that there is a little complexity and complicated operation to apply this method, we will introduce a new index in the next section. 3. A new index to measure: h -index 3.1. Gold priority rule Olympic gold medal table originates in the 1924 Olympic Games in Paris, mainly to sort and compare the level o national and regional sports. This ranking method is popular all over the world. It is the highest level o medal - gold medal in the number as the key results sorted. At the time o sorting, the irst comparison is the number o gold medals, an equal number o gold medals leads to the comparison o the silver number. I the numbers o silver are the same as well, then the number o bronze can be compared, this is in turn. In other words, try to use the highest level o sports achievement to relect the standard o sports in this country. This ranking rule is named "Gold priority" rule. "Gold priority" rule is used to measure various national and regional contribution to the Olympic sports. Although this ranking method is not ormal, but the delegations and audiences all over the world give the gold medal table a

7 Qualitative and Quantitative Methods in Libraries (QQML) 5: , high degree o attention, gold medal table and thereore becomes the ocus o the audience during the competition. Ranking as this method, during the 2004 Athens Olympic Games, Chinese delegation got 32 gold medals, the medal total to 63, while Russia got 27 gold medals, the medal total to 92, rom the perspective on the total number o medals, China lags behind that o Russia, but as a result o China's gold medals ahead o Russia, China eventually was second in the medal table. Remember the 2008 Beijing Olympic Games, the Chinese delegation got 51 gold medals, a total o 100 medals, although 10 medals behind the United States in the total number. But by the leading number o gold medals, China won the gold medal table champion The rationality o Gold priority rule applied to the evaluation Gold priority rule can be applied to improve the current evaluation system which owns quite a lot o deects. First, it is in line with the meaning o evaluation which can contribute to the academic level o the papers which published by the scientiic researcher. Second, it determines the content o evaluation. The key is the true level o scholars in research contribution. And other actors can only in the same situation consider as the secondary actors, such as "silver" and "bronze" actor. Third, it can truly and objectively show the overall strength o the best athletes rom each delegation. Compared with those evaluation approaches that excessively pursuit "objective quantitative evaluation, this approach is more air and reasonable The application o Gold priority rule to the evaluation Wei Bu, Chinese academy o the social sciences, and others have presided over a research project, mainly about the value evaluation o social science achievement, evaluation methods that research inally established, showing the characteristics o the "Gold priority rule. In 2006, inspired by the Olympic gold medal table ranking rule, Qiu et al. mentioned in The thoughts or "gold priority" rule applies to research talent evaluation (2006) that the academic level o researchers should depend on the researchers optimal results. This method can guide the scientiic researchers pay more attention to the quality o academic papers. In 2009, Xu proposed a new modiied index: h -index, which was based on the Gold priority rule on the initiative. a a h -index, on the basis o the "gold priority" rule, expanded the scope o application o the h-index, was more sensitive than h-index, with higher precision, so that the scholars with same h-index could be a deeper comparison. In this section, as a way o implementation or the "Gold priority" rule, a new method is proposed based on the idea rom Qiu et al. From another perspective, h it can also be regard as a correction or the a -index h -index I quantiying an individual's scientiic research output in accordance with the "Gold priority" rule directly, the value o each actor must be separately

8 228 Weian Zhao, Wenjun Liu, and Ruiyu Zhang calculated, and then one by one to compare. This requires multiple times, and the process is relatively complicated. In view o this situation, this paper puts orward h -index, which retain all properties and advantages o the "Gold priority" rule and simpliied it into a single value, then scientiic research rank can be based on the size o the h -index Min-Max Normalization X m 1) Min-Max Normalization: X. M m Wherein, X represents the original data, m represents the maximum value among the original data, M represents the minimum value among the original data. 2) I not speciied, X reserves two decimal places in this article The meaning o h -index The deinition o h -index: according to "Gold priority" rule, regard h-index as the gold medal, the score o average citation rate as the silver medal, the score o average download requency as bronze medal, a new index is concluded. The computational ormula o h -index is: 1 1 h H AR 2 AD Wherein, h represents the h-index, round represents the data rounded, Ar represents an average citation rate, Ad represents the thesis average download requency. Capital letter H represents the h-index with Min-Max Normalization. Similarly, Capital letter AR, AD represents the Ar, Ad with Min-Max Normalization respectively. Comparative method: Unlike ha -index, h -index can directly compare size, the greater the h -index indicates a larger inluence o the scholar, and then the greater the order will be The applicability o h -index h -index presented in Section also has certain limitation. Since the index retains six decimal places, h -index is likely to be equal in the high number o scholars being evaluated. In this case, we can expand the number o decimal places rom six to eight or more decimal digits. Expanding method is: In view o the single actor except "gold medal", changing percentile scoring system with a thousand-point scale even higher median score, the higher the digits, the more accurate ranking the single actor. Computation ormula is as ollows. The computation ormula with 6-bit decimal is as ollows.

9 Qualitative and Quantitative Methods in Libraries (QQML) 5: , h H AR 2 AD The computation ormula with 3N -bit decimal is as ollows: 1 1 h h AR AD N 2N Promotion o the h -index Section and Section mainly tell the scientiic evaluation method with three actors, this method is to promote, and the ormula can take more actors into consideration ater promoting. It is important to point out the importance and the sequence o each actor in the process o promoting. The scientiic evaluation method with two actors is as ollows. 1 h2 H AR h 100 Wherein, h represents the h-index, Ar(h) represents the average citation rate o h-core papers, AR (h) indicates the Ar(h) with Min-Max Normalization among persons evaluated. In the process o comparison to ha -index, the irst comparison with descending order is the integer part o h -index, which is the h-index. And then compare the decimal part o to cause conusion, whereas a a h -index with ascending order. This ranking method is easy h 2 -index can be directly compared according to the size o the values, so this method can also be seen as a correction o index. The scientiic evaluation method with our actors is as ollows h H C 2 N 3 D h 2 - Wherein, h represents the h-index, c represents the citation, n expresses the total number o papers, and d represents download requency. O course, our actors selected here is not ixed, they can be changed according to the actual situation. The scientiic evaluation method with N actors is as ollows hn X ( 1) X (2) X ( i) X ( N) i 1 N Wherein, X (i) represents the i-th actor in the process o scientiic research evaluation, which is ranked i-th according to the importance o actors Practice o the h -index Data source We use the same data as in Section 2.3.1, which can be got in the Appendix.

10 230 Weian Zhao, Wenjun Liu, and Ruiyu Zhang Data analysis We work out the h -index o 20 scholars according to the data in the Appendix. In order to show h-index more intuitive and accurate, comprehensive evaluation value Z, h -index and rankings, the results are as ollows (see Table 5). Table 5. The h-index. Z. h -index and ranking results o 20 scholars Name h-index Z h-index Rank or h Suning You Huo Ren Canhua Chen Fuman Gong Hong Xiao Yiwen Shao Daming Zhu Weiguo Xu Fan Tao Lijun Yang Qianghui Ran Chengu Wu Yuan Yao Xingyong Zhang Zuoxin Zhou Chucai Xiong Yulin Cai Wenshen Xu Meixiang Zhu Tiecheng Jin Observe the h-index column in Table 5, in 20 scholars, in addition to Suning You who located in the irst place and with a h-index o 11, much higher than other scholars, the rest o the 19 scholars, each h-index o the scholar is not unique, and even seven scholars have h-index o 4 (more than a third o people have equal h-index), in this case, it is obviously unreasonable to use the h-index to evaluate, so an index with higher sensitivity and dierentiation is very necessary. And then observed the h -index column in Table 5, by deinition,

11 Qualitative and Quantitative Methods in Libraries (QQML) 5: , the integer part o h -index maintaining the h-index, so the decimal part has only carried on the urther comparison, the eect o this approach is obvious, each h -index is unique, ranking results are thereore rather unique, won't appear name repetition The relation between h-index, comprehensive evaluation value Z and h -index Respectively calculate the Pearson correlation coeicient between h-index, comprehensive evaluation value Z and h -index, and the results as ollows: The correlation coeicient between h-index and Z is , the correlation coeicient between Z and h -index is , and the correlation between the h-index and h -index is up to Thereore, there is a relatively signiicant correlation between the h-index and Z, while h-index and h -index is particularly signiicant Dierence analysis between comprehensive evaluation value Z and h -index The correlation coeicient between Z and the h -index reached , but compare with the correlation coeicient between h-index and h -index o , there is still a big dierence, and the reason is as ollows. There is a dierent thought between comprehensive evaluation values Z and h - index. h -index is an amendment to the h-index, enables the scholar with same h-index to more in-depth comparison, and improves the degree o dierentiation or the h-index. In addition, because the h-index, the thesis average citation and thesis average download requency are considered in the h -index, this will to a certain extent enhance the h-index linked to other actors. As or the comprehensive evaluation value Z, proposed to avoid the oneness on numerical actors, it comprehensively considers various actors, is more complicated in the calculation process, and there is a big dierence with the h -index. 4. Conclusions h-index, rom the date o birth, has attracted the attention o the international scientiic community. Many scholars study the discussion o the h-index with great interest, hoping to get some results. The current paper mainly concentrated on two aspects: irst, to explore scientiicity and practicability o the h-index rom all aspects, so as to expand the applicable scope o the h-index; Second, to modiy the h-index and class h-index and then put orward some new derivative index, thereby improving the evaluation system.

12 232 Weian Zhao, Wenjun Liu, and Ruiyu Zhang In the process o discussion and research, people need to recognize that the h- index is not omnipotent, it still exists many deects. When we apply h-index and class h-index to quantiy an individual s scientiic research output, the advantages and disadvantages as well as prerequisites o these indexes are worth noting. When evaluating scholars, comprehensive evaluation method with multiaceted, multi-actor and multi-index should be taken into consideration, so as to make evaluation more eicient, air and reasonable. To make comprehensive assessment on scientiic research achievements o scholars more multiaceted, multi-actor, multi-indexes, we has used two methods to quantiy an individual s research achievements in a comprehensive analysis. The irst method is to analyze a variety o actors in the original data, taking the various actor's contribution to the scientiic research evaluation into account. The second method is to propose a new comprehensive research evaluation h -index, which can avoid the complex calculations when evaluating with some mathematical thoughts, and enhance the degree o dierentiation and sensitivity o h-index, easy to use. h-index is an indicator in the ield o literature metrology, and many achievements in theoretical and empirical research have been launched. But it is still not perect in many places and need a urther research and argumentation. Reerences Hirsch, J. E., (2005). An index to quantiy an individual s scientiic research output. Proceedings o the National Academy o Sciences o the USA, 102 (46): Braun, T, Glaznzel, W, Schubert, A, (2006). A Hirsch-type index or journals. Scientometrics,69 (1): Egghe, L, (2006). Theory and practise o the g-index. Scientometrics, 69 (1): Marek, K, (2006). A new Hirsch-type index saves time and works equally well as the original h-index. ISSI Newsletter, 2 (3): 4-6. Jin, B.H., Liang, L.M., (2007), Ronald Rousseau and Leo Egghe. The R- and ARindices: Complementing the h-index. Chinese Science Bulletin, 52 (6): Zhang, X.M., (2007). h -index Correction o h index. Library and inormation m service, 51 (10): (In Chinese). Qiang, Wu, (2008). The w-index: A signiicant improvement o the h-index. Access date available at Anderson, T.R., Hankin, R.K.S., Killworth, P.D., (2008). Beyond the Duree square: Enhancing the h-index to score total publication output. Scientometrics, 76 (3): Ye, Y, (2009). A new index o academic sort -index analysis. Inormation learned journal, 28 (1): (In Chinese). Xu, X.J. (2009). h -index:correction o h-index the revelation o the "gold a priority" rule. Journal o inormation theory and practice, 11 (32): 8-12 (In Chinese). Alonso, S, Cabrerizo, F. J., Herrera-Viedma, E, Herrera, F, (2010). hg-index: a new index to characterize the scientiic output o researchers based on the h- and g-indices. Scientometrics, 82: Pearson, K, (1901). On lines and planes o closest it to systems o points in space. Philosophical Magazine, 1901, 2 (6):

13 Qualitative and Quantitative Methods in Libraries (QQML) 5: , Hotelling, H, (1933). Analysis o a complex o statistical variables into principal components. Journal o Educational Psychology, 24 (6): Wei, R.B., Song, G, (2009). h-index research review and the empirical statistical analysis. Chinese Journal o Scientiic and Technical Periodicals, 20 (2): (In Chinese). Qiu, J.P., Zhu, S.Q., Liu, Y, (2006). The thoughts or "gold priority" rule applies to research talent evaluation. Knowledge o library and inormation science, 4: (In Chinese). Appendix Suning You Huo Ren Canhu a Chen Fuman Gong Hong Xiao Yiwen Shao Damin g Zhu Weigu o Xu Fan Tao Lijun Yang Qiangh ui Ran Cheng u Wu Yuan Yao Xingyo ng Zhang Issue d amou nt 20 well-known scholars papers cited situation h- ind ex Total citatio ns The averag e citatio n rate Downlo ad requenc y Averag e downlo ad requen cy Browse requen cy

14 234 Weian Zhao, Wenjun Liu, and Ruiyu Zhang Zuoxin Zhou Chucai Xiong Yulin Cai Wensh en Xu Meixia ng Zhu Tieche ng Jin