A recursive field-normalized bibliometric performance indicator: An application to the field of library and information science

Transcription

1 A recursve feld-normalzed bblometrc performance ndcator: An applcaton to the feld of lbrary and nformaton scence Ludo Waltman, 1 Erja Yan, 2 and Nees Jan van Eck 1 1 Centre for Scence and Technology Studes, Leden Unversty, Leden, The Netherlands {waltmanlr, ecknjpvan}@cwts.ledenunv.nl 2 School of Lbrary and Informaton Scence, Indana Unversty, Bloomngton, Indana, Unted States eyan@ndana.edu Two commonly used deas n the development of ctaton-based research performance ndcators are the dea of normalzng ctaton counts based on a feld classfcaton scheme and the dea of recursve ctaton weghng (lke n PageRank-nspred ndcators). We combne these two deas n a sngle ndcator, referred to as the recursve mean normalzed ctaton score ndcator, and we study the valdty of ths ndcator. Our emprcal analyss shows that the proposed ndcator s hghly senstve to the feld classfcaton scheme that s used. The ndcator also has a strong tendency to renforce bases caused by the classfcaton scheme. Based on these observatons, we advse aganst the use of ndcators n whch the dea of normalzaton based on a feld classfcaton scheme and the dea of recursve ctaton weghng are combned. 1. Introducton In bblometrc and scentometrc research, there s a trend towards developng more and more sophstcated ctaton-based research performance ndcators. In ths paper, we are concerned wth two streams of research. One stream of research focuses on the development of ndcators that am to correct for the fact that the densty of ctatons (.e., the average number of ctatons per publcaton) dffers among felds. Two basc approaches can be dstngushed. One approach s to normalze ctaton counts for feld dfferences based on a classfcaton scheme that assgns publcatons to felds (e.g., Braun & Glänzel, 1990; Moed, De Brun, & Van Leeuwen, 1995; Waltman, Van Eck, Van Leeuwen, Vsser, & Van Raan, 2011). The other approach s to normalze ctaton counts based on the number of references n ctng publcatons or ctng journals (e.g., Moed, 2010; Ztt & Small, 2008). The latter approach, whch s sometmes referred to as source normalzaton (Moed, 2010), does not need a feld classfcaton scheme. A second stream of research focuses on the development of recursve ndcators, typcally nspred by the well-known PageRank algorthm (Brn & Page, 1998). In the case of recursve ndcators, ctatons are weghed dfferently dependng on the status of the ctng publcaton (e.g., Chen, Xe, Maslov, & Redner, 2007; Ma, Guan, & Zhao, 2008; Walker, Xe, Yan, & Maslov, 2007), the ctng journal (e.g., Bollen, Rodrguez, & Van de Sompel, 2006; Pnsk & Narn, 1976), or the ctng author (e.g., Radcch et al., 2009; Życzkowsk, 2010). The underlyng dea s that a ctaton from an nfluental publcaton, a prestgous journal, or a renowned author should be regarded as more valuable than a ctaton from an nsgnfcant publcaton, an obscure journal, or an unknown author. It s sometmes argued that non-recursve 1

2 ndcators measure popularty whle recursve ndcators measure prestge (e.g., Bollen et al., 2006; Yan & Dng, 2010). Based on the above dscusson, we have two bnary dmensons along whch we can dstngush ctaton-based research performance ndcators, namely the dmenson of normalzaton based on a feld classfcaton scheme versus source normalzaton and the dmenson of non-recursve mechansms versus recursve mechansms. These two dmensons yeld four types of ndcators. Ths s shown n Table 1, n whch we lst some examples of the dfferent types of ndcators. It s mportant to note that all currently exstng ndcators that use a classfcaton scheme for normalzng ctaton counts are of a non-recursve nature. Hence, there currently are no recursve ndcators that make use of a classfcaton scheme. 1 Instead, the currently exstng recursve ndcators can best be regarded as belongng to the famly of source-normalzed ndcators. Ths s because these ndcators, lke non-recursve source-normalzed ndcators, are based n one way or another on the dea that each unt (.e., each publcaton, journal, or author) has a certan weght whch t dstrbutes over the unts t ctes. We refer to Waltman and Van Eck (2010a) for a detaled analyss of the close relatonshp between a source-normalzed ndcator (.e., the audence factor) and two recursve ndcators (.e., the Egenfactor ndcator and the nfluence weght ndcator). Table 1. A classfcaton of some ctaton-based research performance ndcators based on ther normalzaton approach and the presence or absence of a recursve mechansm. Non-recursve mechansm Recursve mechansm Normalzaton based on classfcaton scheme Ctaton z-score (Lundberg, 2007) CPP/FCSm (Moed et al., 1995) (Waltman et al., 2011) NMCR (Braun & Glänzel, 1990) Source normalzaton Audence factor (Ztt, 2010; Ztt & Small, 2008) Fractonal countng (Glänzel et al., 2011; Leydesdorff & Bornmann, 2011) SNIP (Moed, 2010) Source-normalzed (Waltman & Van Eck, 2010b) CteRank (Walker et al., 2007) Egenfactor (Bergstrom, 2007; West et al., 2010) Influence weght (Pnsk & Narn, 1976) Scence author rank (Radcch et al., 2009) SCImago journal rank (González-Perera et al., 2010) Weghted PageRank (Bollen et al., 2006) In ths paper, we focus on the empty cell n the lower left of Table 1. Hence, we focus on recursve ndcators that use a feld classfcaton scheme for normalzng ctaton counts. We frst propose a recursve varant of the mean normalzed ctaton score () ndcator (Waltman et al., 2011). We then present an emprcal analyss of ths recursve ndcator. In the analyss, the recursve ndcator s used to study the ctaton mpact of journals and research nsttutes n the feld of lbrary and nformaton scence. Our am s to get nsght nto the valdty of 1 However, a frst step n the drecton of such ndcators was taken by Van Leeuwen, Vsser, Moed, Nederhof, and Van Raan (2003). They proposed an ndcator that weghs ctatons by the average feldnormalzed number of ctatons per publcaton of the ctng journal. 2

3 recursve ndcators that use a classfcaton scheme for normalzng ctaton counts. We pay specal attenton to the senstvty of such ndcators to the classfcaton scheme that s used. 2. Recursve mean normalzed ctaton score The ordnary non-recursve ndcator for a set of publcatons equals the average number of ctatons per publcaton, where for each publcaton the number of ctatons s normalzed for dfferences among felds (Waltman et al., 2011). The normalzaton s performed by dvdng the number of ctatons of a publcaton by the publcaton s expected number of ctatons. The expected number of ctatons of a publcaton s defned as the average number of ctatons per publcaton n the feld n whch the publcaton was publshed. An example of the calculaton of the nonrecursve ndcator s provded n Table 2. Table 2. Example of the calculaton of the ordnary non-recursve ndcator. There are three publcatons. For each publcaton, the table lsts the number of ctatons, the feld, the expected number of ctatons, and the normalzed ctaton score. The normalzed ctaton score of a publcaton s obtaned by dvdng the number of ctatons by the expected number of ctatons. The ndcator equals the average of the normalzed ctaton scores of the three publcatons. Publcaton No. ct. Feld Expected Normalzed no. ct. ct. score A 3 X B 8 X C 10 Y = ( ) / 3 = 1.12 The non-recursve ndcator can also be referred to as the frst-order ndcator. We defne the second-order ndcator n the same way as the frst-order ndcator except that ctatons are weghed dfferently. In the frstorder ndcator, all ctatons have the same weght. In the second-order ndcator, on the other hand, the weght of a ctaton s gven by the value of the frstorder ndcator for the ctng journal. Hence, ctatons from journals wth a hgh value for the frst-order ndcator are regarded as more valuable than ctatons from journals wth a low value for the frst-order ndcator. We have now defned the second-order ndcator n terms of the frst-order ndcator. In the same way, we defne the thrd-order ndcator n terms of the second-order ndcator, the fourth-order ndcator n terms of the thrdorder ndcator, and so on. Ths yelds the recursve ndcator that we study n ths paper. To make the above defnton of the recursve ndcator more precse, we formalze t mathematcally. We use, j, k, and l to denote, respectvely, a publcaton, a journal, a feld, and an nsttute. We defne 1 f publcaton ctes publcaton ' c ' = (1) 0 otherwse, 3

4 where 1 f publcaton s publshed n journal j p j = 0 otherwse, (2) 1 f publcaton belongs to feld k b k = 0 otherwse, (3) 1 f publcaton s authored by nsttute l a l = 0 otherwse. (4) For α = 1, 2,, the αth-order ctaton score of publcaton s defned as ' = ( ) ( ) CS α w α c, (5) ' ' w denotes the weght of a ctaton from publcaton '. For α = 1, w 1 for all '. For α = 2, 3,, w s gven by ' ' ' = w ' = p ' j j ( α 1) j. (6) It follows from (5) and (6) that the αth-order ctaton score of a publcaton equals a weghted sum of the ctatons receved by the publcaton. For α = 1, all ctatons have the same weght. For α = 2, 3,, the weght of a ctaton s gven by the (α 1)thorder of the ctng journal. We defne the αth-order mean ctaton score of a feld as the average αth-order ctaton score of all publcatons belongng to the feld, that s, MCS k = b k CS b k. (7) The αth-order expected ctaton score of a publcaton s defned as the αth-order mean ctaton score of the feld to whch the publcaton belongs, 2 and the αth-order normalzed ctaton score of a publcaton s defned as the rato of the publcaton s αth-order ctaton score and ts αth-order expected ctaton score. Ths yelds ECS = b k k MCS k, (8) CS NCS =. (9) ECS 2 For smplcty, we assume that felds are non-overlappng. A publcaton therefore always belongs to exactly one feld. 4

5 If the αth-order normalzed ctaton score of a publcaton s greater (less) than one, ths ndcates that the αth-order ctaton score of the publcaton s greater (less) than the average αth-order ctaton score of all publcatons n the feld. We defne the αth-order of a set of publcatons as the average αth-order normalzed ctaton score of the publcatons n the set. In the case of journals and nsttutes, we obtan respectvely j = p j NCS p j, (10) l = a l NCS a l, (11) where al a l =. (12) a l ' l ' It follows from (11) and (12) that n the case of nsttutes we take a fractonal countng approach. That s, a publcaton resultng from a collaboraton of, say, three nsttutes s counted for each nsttute as 1/3 of a full publcaton. Alternatvely, a full countng approach could have been taken. A collaboratve publcaton would then be counted as a full publcaton for each of the nsttutes nvolved. A full countng approach s obtaned by replacng a l by a l n (11). Untl now, we have only dscussed the ssue of normalzaton for the feld n whch a publcaton was publshed. We have not dscussed the ssue of normalzaton for the age of a publcaton. The latter type of normalzaton can be used to correct for the fact that older publcatons have had more tme to earn ctatons than younger publcatons. Normalzaton for the age of a publcaton can easly be ncorporated nto ndcators that make use of a feld classfcaton scheme, such as the ndcator. It s more dffcult to ncorporate nto (recursve or non-recursve) sourcenormalzed ndcators (see however Waltman & Van Eck, 2010b). In the emprcal analyss presented n Secton 4, the recursve ndcator performs a normalzaton both for the feld n whch a publcaton was publshed and for the age of a publcaton. Ths means that n the above mathematcal descrpton of the recursve ndcator k n fact represents not just a feld but a combnaton of a feld and a publcaton year. As a consequence, b k ndcates whether a publcaton was ( ) publshed n a certan feld and year, and MCS α k ndcates the average αth-order ctaton score of all publcatons publshed n a certan feld and year. 3. Data To test our recursve ndcator, we use the ndcator to study the ctaton mpact of journals and research nsttutes n the feld of lbrary and nformaton scence (LIS). We focus on the perod from 2000 to Our analyss s based on data from the Web of Scence database. 5

6 We frst needed to delneate the LIS feld. We used the Journal of the Amercan Socety for Informaton Scence and Technology (JASIST) as the seed journal for our delneaton. We decded to select the 47 journals that, based on co-ctaton data, are most strongly related wth JASIST. Only journals n the Web of Scence subject category Informaton Scence & Lbrary Scence were consdered. JASIST together wth the 47 selected journals consttuted our delneaton of the LIS feld. From the journals wthn our delneaton, we selected all 12,202 publcatons n the perod that are of the document type artcle or revew. It s mportant to emphasze that n our analyss we only take nto account ctatons wthn the set of 12,202 publcatons. Ctatons gven by publcatons outsde ths set are not consdered. Self ctatons are also excluded. Table LIS journals and ther assgnment to three clusters. Lbrary scence (27 journals) Afrcan Journal of Lbrary Archves and Informaton Scence Australan Lbrary Journal College & Research Lbrares Electronc Lbrary Informaton Technology and Lbrares Interlendng & Document Supply Internatonal Informaton & Lbrary Revew Journal of Academc Lbraranshp Journal of Lbraranshp and Informaton Scence Journal of Scholarly Publshng Law Lbrary Journal Learned Publshng Lbrary & Informaton Scence Research Lbrary and Informaton Scence Lbrary Collectons Acqustons & Techncal Servces Lbrary H Tech Lbrary Quarterly Lbrary Resources & Techncal Servces Lbrary Trends Lbr Malaysan Journal of Lbrary & Informaton Scence Portal-Lbrares and the Academy Program-Electronc Lbrary and Informaton Systems Reference & User Servces Quarterly Scence & Technology Lbrares Serals Lbraran Serals Revew Informaton scence (14 journals) Annual Revew of Informaton Scence and Technology Aslb Proceedngs Canadan Journal of Informaton and Lbrary Scence Informaton Processng & Management Informaton Research Journal of Documentaton Journal of Informaton Scence Journal of the Amercan Socety for Informaton Scence Journal of the Amercan Socety for Informaton Scence and Technology Knowledge Organzaton Onlne Informaton Revew Perspectvas Em Cenca Da Informacao Proceedngs of the ASIST Annual Meetng Profesonal De La Informacon Scentometrcs (7 journals) ASIST Monograph Seres Australan Academc & Research Lbrares Investgacon Bblotecologca Journal of Informetrcs Research Evaluaton Revsta Espanola De Documentacon Centfca Scentometrcs In our analyss, we also study the effect of splttng up the LIS feld n a number of subfelds. We used the followng procedure to splt up the LIS feld. We frst collected bblographc couplng data for the 48 journals n our analyss. 3 Based on the bblographc couplng data, we created a clusterng of the journals. We used the VOS clusterng technque (Waltman, Van Eck, & Noyons, 2010), avalable n the VOSvewer software (Van Eck & Waltman, 2010), to create the clusterng. After 3 We also tred out usng co-ctaton data, but ths gave less satsfactory results. 6

7 some expermentng, we decded that the most satsfactory clusterng has three clusters. These clusters can roughly be nterpreted as follows. The largest cluster (27 journals) deals wth lbrary scence, the smallest cluster (7 journals) deals wth scentometrcs, and the thrd cluster (14 journals) deals wth general nformaton scence topcs. The assgnment of the 48 journals to the three clusters s shown n Table 3. The clusterng of the journals s also shown n the journal map n Fgure 1. Ths map, produced usng the VOSvewer software, s based on bblographc couplng relatons between the journals. Fgure 1. Journal map of 48 LIS journals based on bblographc couplng data. The color of a journal ndcates the cluster to whch t belongs. The map was produced usng the VOSvewer software. 4. Results As dscussed n the prevous secton, we can treat LIS ether as a sngle ntegrated feld or as a feld consstng of three separate subfelds (.e., lbrary scence, nformaton scence, and scentometrcs). In the latter case, the recursve ndcator normalzes for dfferences among the three subfelds n the average number of ctatons per publcaton. Below, we frst present the results obtaned when LIS s treated as a sngle ntegrated feld. We then present the results obtaned when LIS s treated as a feld consstng of three separate subfelds. We also present a comparson of the results obtaned usng the two approaches. We emphasze once more that n our analyss ctatons gven by publcatons outsde our set of 12,202 publcatons are not taken nto account Sngle ntegrated LIS feld We frst consder the case of a sngle ntegrated LIS feld. The recursve ndcator s sad to have converged for a certan α f there s vrtually no dfference between values of the αth-order ndcator and values of the (α + 1)th-order ndcator. For our data, convergence of the recursve ndcator can be 7

8 observed for α = 20. In our analyss, our man focus therefore s on comparng the frst-order ndcator (.e., the ordnary non-recursve ndcator) wth the 20th-order ndcator. In Table 4, we lst the top 10 journals accordng to both the frst-order ndcator and the 20th-order ndcator. In the case of the frst-order ndcator, the top 10 conssts of journals from all three subfelds. However, journals from the nformaton scence and scentometrcs subfelds seem to slghtly domnate journals from the lbrary scence subfeld. There are three lbrary scence journals n the top 10, at ranks 4, 8, and 10. Gven that more than half of the journals n our analyss belong to the lbrary scence subfeld (27 of the 48 journals), the lbrary scence journals seem to be underrepresented n the top 10. Also, wthn the top 10, the three lbrary scence journals have relatvely low ranks. Table 4. Top 10 journals accordng to both the frst-order ndcator and the 20th-order ndcator. LIS s treated as a sngle ntegrated feld n the calculaton of the ndcators. Journal Journal (α = 1) (α = 20) Journal of Informetrcs 4.49 Journal of Informetrcs Annual Revew of Informaton Annual Revew of Informaton 2.97 Scence and Technology Scence and Technology 3.79 Journal of the Amercan Socety for Informaton Scence 2.35 Scentometrcs 3.17 Interlendng & Document Supply 1.94 Journal of the Amercan Socety for Informaton Scence 2.72 Journal of the Amercan Socety for Informaton Scence and 1.84 Journal of the Amercan Socety for Informaton Scence and 2.36 Technology Technology Scentometrcs 1.72 Journal of Documentaton 1.36 Journal of Documentaton 1.58 Informaton Processng & Management 1.21 College & Research Lbrares 1.33 Journal of Informaton Scence 0.96 Informaton Processng & Lbrary & Informaton Scence 1.21 Management Research 0.82 Lbrary & Informaton Scence Research 1.17 Research Evaluaton 0.81 Let s now turn to the top 10 journals accordng to the 20th-order ndcator. Ths top 10 provdes a much more extreme pcture. The top 10 s now almost completely domnated by nformaton scence and scentometrcs journals. There s only one lbrary scence journal left, at rank 9. Moreover, when lookng at the values of the ndcator, large dfferences can be observed wthn the top 10. Especally the extremely hgh value of the ndcator for Journal of Informetrcs, the hghest ranked journal, s strkng. The value of the ndcator for ths journal s more than three tmes as hgh as the value of the ndcator for Annual Revew of Informaton Scence and Technology, whch s the secondhghest ranked journal. In Fgure 2, the frst-order ndcator s compared wth the second-order ndcator (left panel) and the 20th-order ndcator (rght panel) for the 48 LIS journals n our data set. As can be seen n the fgure, the dfferences between the frst- and the second-order ndcator are relatvely small, although there s one journal (Journal of Informetrcs) that benefts qute a lot from gong from the 8

9 frst-order ndcator to the second-order ndcator. The dfferences between the frst- and the 20th-order ndcator are much larger. As was also seen n Table 4, there are a number of journals for whch the value of the 20th-order ndcator s hgher than the value of the frst-order ndcator. These are all nformaton scence and scentometrcs journals. The lbrary scence journals all turn out to have a lower value for the 20th-order ndcator than for the frstorder ndcator. Fgure 2. Comparson of the frst-order ndcator wth the second-order ndcator (left panel) and the 20th-order ndcator (rght panel) for 48 LIS journals. LIS s treated as a sngle ntegrated feld n the calculaton of the ndcators. Lbrary scence, nformaton scence, and scentometrcs journals are ndcated by, respectvely, red ponts, green crosses, and blue crcles. Table 5. Top 10 research nsttutes accordng to both the frst-order ndcator and the 20th-order ndcator. LIS s treated as a sngle ntegrated feld n the calculaton of the ndcators. Insttute Insttute (α = 1) (α = 20) Leden Unv 3.85 Unv Antwerp 8.64 Unv Antwerp 3.77 Hungaran Acad Sc 7.92 Hungaran Acad Sc 3.70 Unv Amsterdam 7.45 Unv Amsterdam 3.53 Leden Unv 7.43 Royal Sch Lb & Informat Sc 3.20 Lmburg Unv Ctr 5.94 Indana Unv 2.52 Kathol Unv Leuven 5.19 Hebrew Unv Jerusalem 2.51 Indana Unv 3.56 Kathol Unv Leuven 2.48 Unv Wolverhampton 3.13 Unv Tennessee - Knoxvlle 2.39 Royal Sch Lb & Informat Sc 3.07 Unv Bar Ilan 2.35 Hebrew Unv Jerusalem 2.72 In addton to journals, we also consder research nsttutes n LIS. We restrct our analyss to the 86 nsttutes that have at least 25 publcatons n our data set. (Recall from Secton 2 that publcatons are counted fractonally.) The top 10 nsttutes accordng to both the frst-order ndcator and the 20th-order ndcator are lsted n Table 5. Comparng the results of the two ndcators, t s clear that nsttutes whch are manly actve n the scentometrcs subfeld beneft a lot from the use of a hgher-order ndcator. For these nsttutes, the value of the 20thorder ndcator tends to be much hgher than the value of the frst-order ndcator. Ths s consstent wth our above analyss for LIS journals, where we found that the two most mportant scentometrcs journals (Journal of Informetrcs 9

10 and Scentometrcs) beneft qute sgnfcantly from the use of a hgher-order ndcator Three separate LIS subfelds We now consder the case n whch LIS s dvded nto three separate subfelds (.e., lbrary scence, nformaton scence, and scentometrcs). In the calculaton of the recursve ndcator, a normalzaton s performed to correct for dfferences among the three subfelds n the average number of ctatons per publcaton. Lke n the above analyss, we focus manly on comparng the frst-order ndcator wth the 20th-order ndcator. In Table 6, the top 10 journals accordng to both the frst-order ndcator and the 20th-order ndcator s shown. Ths table s smlar to Table 4 above, except that n the calculaton of the ndcators LIS s treated as a feld consstng of three separate subfelds rather than as a sngle ntegrated feld. Comparng Table 6 wth Table 4, t can be seen that lbrary scence journals now play a much more promnent role, both n the case of the frst-order ndcator and n the case of the 20th-order ndcator. As a consequence, the top 10 journals now looks much more balanced for both ndcators. Also, unlke n Table 4, there are no journals n Table 6 wth an extremely hgh value for the 20th-order ndcator. Table 6. Top 10 journals accordng to both the frst-order ndcator and the 20th-order ndcator. LIS s treated as a feld consstng of three separate subfelds n the calculaton of the ndcators. Journal Journal (α = 1) (α = 20) Journal of Informetrcs 3.07 Interlendng & Document Supply 5.25 Annual Revew of Informaton Scence and Technology 2.62 Journal of Informetrcs 4.20 Annual Revew of Informaton Interlendng & Document Supply 2.46 Scence and Technology 3.40 College & Research Lbrares 1.90 College & Research Lbrares 1.83 Journal of the Amercan Socety for Lbrary & Informaton Scence Informaton Scence and Research 1.67 Technology 1.81 Journal of the Amercan Socety for Informaton Scence and Technology 1.66 Journal of the Amercan Socety for Informaton Scence 1.44 Serals Revew Lbrary & Informaton Scence Research Serals Revew Journal of the Amercan Socety for Informaton Scence Portal-Lbrares and the Academy 1.36 Learned Publshng 1.37 Journal of Documentaton 1.35 Journal of Documentaton 1.26 As can be seen n Table 6, the journal wth the hghest value for the 20th-order ndcator s Interlendng & Document Supply. We nvestgated ths journal n more detal and found that each ssue of the journal contans a revew artcle enttled Interlendng and document supply: A revew of the recent lterature. These revew artcles refer to other artcles publshed n the same ssue of the journal. Clearly, the practce of publshng these revew artcles s an mportant contrbutng factor to the journal s top rankng n Table 6. 10

11 In Fgure 3, a comparson s presented of the frst-order ndcator wth the second-order ndcator (left panel) and the 20th-order ndcator (rght panel) for the 48 LIS journals n our data set. The fgure shows that most of the dfferences between the frst-order ndcator and the hgher-order ndcators are not very large, especally when compared wth the dfferences shown n Fgure 2. The journal that s most senstve to the use of a hgher-order ndcator s Interlendng & Document Supply. As ponted out above, ths journal has qute specal ctaton characterstcs, whch explans ts senstvty to the use of a hgher-order ndcator. Fgure 3. Comparson of the frst-order ndcator wth the second-order ndcator (left panel) and the 20th-order ndcator (rght panel) for 48 LIS journals. LIS s treated as a feld consstng of three separate subfelds n the calculaton of the ndcators. Lbrary scence, nformaton scence, and scentometrcs journals are ndcated by, respectvely, red ponts, green crosses, and blue crcles. Table 7. Top 10 research nsttutes accordng to both the frst-order ndcator and the 20th-order ndcator. LIS s treated as a feld consstng of three separate subfelds n the calculaton of the ndcators. Insttute Insttute (α = 1) (α = 20) Unv Amsterdam 3.04 Unv Amsterdam 3.64 Unv Antwerp 2.79 Unv Antwerp 3.63 Royal Sch Lb & Informat Sc 2.67 Lmburg Unv Ctr 2.84 Leden Unv 2.60 Leden Unv 2.74 Hungaran Acad Sc 2.44 Hungaran Acad Sc 2.58 Cornell Unv 2.34 Indana Unv 2.54 Indana Unv 2.26 Unv Tennessee - Knoxvlle 2.46 Unv Tennessee - Knoxvlle 2.23 Unv Coll Dubln 2.40 Hebrew Unv Jerusalem 2.02 Cornell Unv 2.34 Unv Wolverhampton 1.97 Royal Sch Lb & Informat Sc 2.32 Results for research nsttutes n LIS are reported n Table 7. In the table, the top 10 nsttutes accordng to both the frst-order ndcator and the 20th-order ndcator are shown. Lke n Table 5, the top of the rankng s domnated by nsttutes wth a strong focus on scentometrcs research. Ths s the case both for the frst-order ndcator and for the 20th-order ndcator. However, comparng Table 7 wth Table 5, t can be seen that the values of the scentometrcs nsttutes have decreased qute consderably, especally when lookng at the 20th-order ndcator. Hence, although the scentometrcs nsttutes stll 11

12 occupy the top postons n the rankng, the dfferences wth the other nsttutes have become smaller Comparson In Fgure 4, we present a drect comparson of on the one hand the results obtaned when LIS s treated as a sngle ntegrated feld and on the other hand the results obtaned when LIS s treated as a feld consstng of three separate subfelds. The comparson s made for the 48 LIS journals n our data set. Fgure 4 clearly shows how treatng LIS as a sngle ntegrated feld benefts the scentometrcs journals and harms the lbrary scence journals. In addton, the fgure shows that ths effect s strongly renforced when nstead of the frst-order ndcator the 20th-order ndcator s used. Fgure 4. Comparson of the ndcator when LIS s treated as a sngle ntegrated feld wth the ndcator when LIS s treated as a feld consstng of three separate subfelds. The comparson s made for 48 LIS journals, and ether the frst-order ndcator (left panel) or the 20th-order ndcator (rght panel) s used. Lbrary scence, nformaton scence, and scentometrcs journals are ndcated by, respectvely, red ponts, green crosses, and blue crcles. 5. Dscusson and concluson Recursve bblometrc ndcators are based on the dea that ctatons should be weghed dfferently dependng on the source from whch they orgnate. Ctatons from a prestgous journal, for nstance, should have more weght than ctatons from an obscure journal. It s sometmes argued that by weghng ctatons dfferently dependng on ther source t s possble to measure not just the popularty of publcatons but also ther prestge. In ths paper, we have combned the dea of recursve ctaton weghng wth the dea of usng a classfcaton scheme to normalze ctaton counts for dfferences among felds. The combnaton of these two deas has not been explored before. Although when used separately from each other the two deas can be qute useful, our emprcal analyss for the feld of LIS ndcates that the combnaton of the two deas does not yeld satsfactory results. The man observatons from our analyss are twofold. Frst, our proposed recursve ndcator s hghly senstve to the way n whch felds are defned n the classfcaton scheme that one uses. And second, f wthn a feld there are subfelds wth sgnfcantly dfferent ctaton characterstcs, the recursve ndcator wll be strongly based n favor of the subfelds wth the hghest densty of ctatons. 12

13 The senstvty of bblometrc ndcators to the feld classfcaton scheme that s used for normalzng ctaton counts has been nvestgated n varous studes (Adams, Gurney, & Jackson, 2008; Bornmann, Mutz, Neuhaus, & Danel, 2008; Neuhaus, & Danel, 2009; Van Leeuwen, & Calero Medna, 2009; Ztt, Ramanana-Rahary, & Bassecoulard, 2005). Our emprcal results for the frst-order ndcator (.e., the ordnary non-recursve ndcator) are n lne wth earler studes n whch a sgnfcant senstvty of bblometrc ndcators to the classfcaton scheme that s used has been reported. For the 20th-order ndcator, ths senstvty even turns out to be much hgher. Treatng LIS as a sngle ntegrated feld or as a feld consstng of three separate subfelds yelds very dfferent results for the 20th-order ndcator. If a feld as defned n the classfcaton scheme that one uses s heterogeneous n terms of ctaton characterstcs, bblometrc ndcators wll have a bas that favors subfelds wth a hgher ctaton densty over subfelds wth a lower ctaton densty. Ths s a general problem of bblometrc ndcators that use a classfcaton scheme to normalze ctaton counts. In the case of the recursve ndcator, our emprcal results show that the dea of recursve ctaton weghng strongly renforces bases caused by the classfcaton scheme. Wthn the feld of LIS, the scentometrcs subfeld has the hghest ctaton densty, followed by the nformaton scence subfeld. The lbrary scence subfeld has the lowest ctaton densty. When LIS s treated as a sngle ntegrated feld, the dfferences n ctaton densty among the three LIS subfelds cause both the frst-order ndcator and the 20th-order ndcator to be based, where the bas favors the scentometrcs subfeld and harms the lbrary scence subfeld. However, the bas s much stronger for the 20th-order ndcator than for the frst-order ndcator. For nstance, n the case of the 20th-order ndcator, lbrary scence journals are completely domnated by journals n scentometrcs and nformaton scence. Based on the above observatons, we advse aganst the ntroducton of recursveness nto bblometrc ndcators that use a feld classfcaton scheme for normalzng ctaton counts (such as the ndcator). Instead of provdng more sophstcated measurements of ctaton mpact (measurements of prestge rather than popularty ), the man effect of ntroducng recursveness s to renforce bases caused by the way n whch felds are defned n the classfcaton scheme that one uses. References Adams, J., Gurney, K., & Jackson, L. (2008). Calbratng the zoom A test of Ztt s hypothess. Scentometrcs, 75(1), Bergstrom, C.T. (2007). Egenfactor: Measurng the value and prestge of scholarly journals. College and Research Lbrares News, 68(5). Bollen, J., Rodrguez, M.A., & Van de Sompel, H. (2006). Journal status. Scentometrcs, 69(3), Bornmann, L., Mutz, R., Neuhaus, C., & Danel, H.-D. (2008). Ctaton counts for research evaluaton: Standards of good practce for analyzng bblometrc data and presentng and nterpretng results. Ethcs n Scence and Envronmental Poltcs, 8(1), Braun, T., & Glänzel, W. (1990). Unted Germany: The new scentfc superpower? Scentometrcs, 19(5 6), Brn, S., & Page, L. (1998). The anatomy of a large-scale hypertextual Web search engne. Computer Networks and ISDN Systems, 30(1 7),

14 Chen, P., Xe, H., Maslov, S., & Redner, S. (2007). Fndng scentfc gems wth Google s PageRank algorthm. Journal of Informetrcs, 1(1), Glänzel, W., Schubert, A., Thjs, B., & Debackere, K. (2011). A pror vs. a posteror normalsaton of ctaton ndcators. The case of journal rankng. Scentometrcs, 87(2), González-Perera, B., Guerrero-Bote, V.P., & Moya-Anegón, F. (2010). A new approach to the metrc of journals scentfc prestge: The SJR ndcator. Journal of Informetrcs, 4(3), Leydesdorff, L., & Bornmann, L. (2011). How fractonal countng of ctatons affects the mpact factor: Normalzaton n terms of dfferences n ctaton potentals among felds of scence. Journal of the Amercan Socety for Informaton Scence and Technology, 62(2), Lundberg, J. (2007). Lftng the crown ctaton z-score. Journal of Informetrcs, 1(2), Ma, N., Guan, J., & Zhao, Y. (2008). Brngng PageRank to the ctaton analyss. Informaton Processng and Management, 44(2), Moed, H.F. (2010). Measurng contextual ctaton mpact of scentfc journals. Journal of Informetrcs, 4(3), Moed, H.F., De Brun, R.E., & Van Leeuwen, T.N. (1995). New bblometrc tools for the assessment of natonal research performance: Database descrpton, overvew of ndcators and frst applcatons. Scentometrcs, 33(3), Neuhaus, C., & Danel, H.-D. (2009). A new reference standard for ctaton analyss n chemstry and related felds based on the sectons of Chemcal Abstracts. Scentometrcs, 78(2), Pnsk, G., & Narn, F. (1976). Ctaton nfluence for journal aggregates of scentfc publcatons: Theory, wth applcaton to the lterature of physcs. Informaton Processng and Management, 12(5), Radcch, F., Fortunato, S., Marknes, B., & Vespgnan, A. (2009). Dffuson of scentfc credts and the rankng of scentsts. Physcal Revew E, 80(5), Van Eck, N.J., & Waltman, L. (2010). Software survey: VOSvewer, a computer program for bblometrc mappng. Scentometrcs, 84(2), Van Leeuwen, T.N., & Calero Medna, C. (2009). Redefnng the feld of economcs: Improvng feld normalzaton for the applcaton of bblometrc technques n the feld of economcs. In B. Larsen, & J. Leta (Eds.), Proceedngs of the 12th Internatonal Conference on Scentometrcs and Informetrcs (pp ). Van Leeuwen, T.N., Vsser, M.S., Moed, H.F., Nederhof, T.J., & Van Raan, A.F.J. (2003). The Holy Gral of scence polcy: Explorng and combnng bblometrc tools n search of scentfc excellence. Scentometrcs, 57(2), Walker, D., Xe, H., Yan, K.-K., & Maslov, S. (2007). Rankng scentfc publcatons usng a model of network traffc. Journal of Statstcal Mechancs: Theory and Experment, P Waltman, L., & Van Eck, N.J. (2010a). The relaton between Egenfactor, audence factor, and nfluence weght. Journal of the Amercan Socety for Informaton Scence and Technology, 61(7), Waltman, L., & Van Eck, N.J. (2010b). A general source normalzed approach to bblometrc research performance assessment. In Book of Abstracts of the 11th Internatonal Conference on Scence and Technology Indcators (pp ). Waltman, L., Van Eck, N.J., & Noyons, E.C.M. (2010). A unfed approach to mappng and clusterng of bblometrc networks. Journal of Informetrcs, 4(4),

15 Waltman, L., Van Eck, N.J., Van Leeuwen, T.N., Vsser, M.S., & Van Raan, A.F.J. (2011). Towards a new crown ndcator: Some theoretcal consderatons. Journal of Informetrcs, 5(1), West, J.D., Bergstrom, T.C., & Bergstrom, C.T. (2010). The Egenfactor metrcs: A network approach to assessng scholarly journals. College and Research Lbrares, 71(3), Yan, E., & Dng, Y. (2010). Weghted ctaton: An ndcator of an artcle s prestge. Journal of the Amercan Socety for Informaton Scence and Technology, 61(8), Ztt, M. (2010). Ctng-sde normalzaton of journal mpact: A robust varant of the Audence Factor. Journal of Informetrcs, 4(3), Ztt, M., Ramanana-Rahary, S., & Bassecoulard, E. (2005). Relatvty of ctaton performance and excellence measures: From cross-feld to cross-scale effects of feld-normalsaton. Scentometrcs, 63(2), Ztt, M., & Small, H. (2008). Modfyng the journal mpact factor by fractonal ctaton weghtng: The audence factor. Journal of the Amercan Socety for Informaton Scence and Technology, 59(11), Życzkowsk, K. (2010). Ctaton graph, weghted mpact factors and performance ndces. Scentometrcs, 85(1),