Exploring new applications of scholarly usage data

Transcription

1 Exploring new applications of scholarly usage data Herbert Van de Sompel & Johan Bollen Research Library Los Alamos National Laboratory URL:

2 Johan Bollen and daughter Andrea

3 LANL Research Library: explorations of usage data Item level usage data Datamining of structural patterns in usage data based on the creation of item/agent networks Towards large-scale aggregation of usage data to achieve more globally representative dataminig results => Part of a quest for alternative metrics (usage and citation based) to assess the impact of scholarly items (and hence of their creators)

4 From descriptive statistics to structural properties Descriptive statistics 1) Usage data is recorded 2) Count events related to items 3) Generate descriptive statistics Structural properties 1) Usage data is recorded 2) Extract event sequences from usage data: co-usage data 3) Create usage network; item relationship is co-usage 4) Datamine the network s structural properties

5 Assessing resources: towards a user/structural approach authors Assessment of resources (quality, status, prestige) is required at all levels of our digital infrastructure. Who is generating the information? frequentist structural How is the information analyzed? users

6 Assessing resources: towards a user/structural approach scholarship evaluation now authors IF, citation counts Google s PageRank Trend: 1. author -> user 2. frequency -> structure frequentist structural itunes top songs Amazon.com recommendations novel methods of resource assessment users LANL experiments since 1999

7 Usage networks: a crucial intermediary Impact: social network status as opposed to hit rate popularity Recommendations: use network links to find related documents Collection management: clusters of related documents in network as indicators of community interests Mapping scholarly activity: explore agent/document relationships, trends,

8 Structure 1) Usage data analysis: usage and citation 2) Community dynamics 3) Recommender systems 4) Towards a global usage data aggregation 5) Challenges and new efforts 6) Conclusion

9 Structure 1) Usage data analysis: usage and citation 2) Community dynamics 3) Recommender systems 4) Towards a global usage data aggregation 5) Challenges and new efforts 6) Conclusion

10 usage and citation data analysis: methodology

11 Usage log data sets: LANL and CalState LANL: o o o CalState: o o o Collect reliable usage data from multiple services at LANL SFX linking server logs February 2004 to April 2005 usage data - 392,455 usage events : any indication of preferences/interest - 330,109 articles - 5,866 users Logs collected at 9 institutions SFX linking server logs November 2003 to August 2005 usage data - 3,507,484 usage events : any indication of preferences/interest - 2,133,556 documents - 167,204 users

12 (Descriptive stats are fun too: the academic rhythm at Calstate) Night work Sleep-in Spring break Summer Fall Semester

13 LANL usage and ISI citation measures rank Usage (PageRank) IF (2003) ISSN Title PHYS REV LETT J CHEM PHYS J NUCL MATER X PHYS REV E J APPL PHYS NATURE SCIENCE J AM CHEM SOC APPL PHYS LETT J GEOPHYS RES Structural Local Frequentist Global

14 CSU usage and ISI citation measures rank Usage (PageRank) IF (2003) ISSN Title JAMA-J AM MED ASSOC SCIENCE NATURE J AM ACAD CHILD PSY X AM J PSYCHIAT NEW ENGL J MED AM J PUBLIC HEALTH MED SCI SPORT EXER J ADV NURS AM J CLIN NUTR Structural Local Frequentist Global

15 LANL CS readers and authors agree? Somewhat.

16 CSU CS readers and authors agree? Somewhat.

17 Structure 1) Usage data analysis: usage and citation 2) Community dynamics 3) Recommender systems 4) Towards a global usage data aggregation 5) Challenges and new efforts 6) Conclusion

18 Propagation of news in the CSU community Item traced: "Growth Trends: Big Pickups.", NYT, volume 153, issue (December 2003) Legend: Nodes: users Edges: similar items used Colors: affiliation Numbers: order in which item was used

19 Structure 1) Usage data analysis: usage and citation 2) Community dynamics 3) Recommender systems 4) Towards a global usage data aggregation 5) Challenges and new efforts 6) Conclusion?

20 Usage-based recommender system Recommendations derived from usage network Starts from (set of) queried documents (articles or journals) and scans usage network links for related documents Results: o Efficient, fast due to sparse networks o Relevant results derived from accumulated, aggregated usage?

21 Recommendations Usage data from 15 institutions: 1,121,129 events 419,099 documents

22 Recommendations Calstate usage data: 3,507,484 events 2,133,556 documents

23 Recommendations Calstate usage data

24 Recommendations

25 Structure 1) Usage data analysis: usage and citation 2) Community dynamics 3) Recommender systems 4) Towards a global usage data aggregation 5) Challenges and new efforts 6) Conclusion

26 Usage data aggregation publishers P P P P data I I I institutions COUNTER SUSHI Institutional journal usage data is interesting. But how about aggregated usage data? Raw usage data! Usage data involving all scholarly items! Monitor science as it takes place! I P Variety of scholarly communication items Variety of scholarly repositories Variety of potential applications P Aggregator A Dataset repo I P Software repo

27 Local aggregation of usage data: linking servers Publisher sites ISI ingenta Linking servers record activities across multiple OpenURL-enabled information sources of a specific digital library environment Full text DBs Link Resolver Ehost EJS Linking server logs are representative of the activities of a particular user population Linking servers wellestablished in scholarly information environment British Library Google A&I service Allows recording of clickstream data: connect same user, different system streams

28 Aggregation of local usage data Log Repository 1 Link Resolver CO CO CO OpenURL ContextObjects Link Resolver Log Repository 2 Log Repository 3 CO CO CO CO CO CO Log harvester Aggregated Usage Data Aggregated Log DB logs Data mining Item relations Metrics Services Recommender Services Analysis services Collection management Trend analysis Link Resolver

29 OpenURL ContextObject to represent usage data Event information: * event datetime * globally unique event ID Referent * identifier * metadata Requester * User or user proxy: IP, session, ServiceType Resolver: * identifier of linking server <?xml version= 1.0 encoding= UTF-8?> <ctx:context-object timestamp= T10:22:33Z identifier= urn:uuid:58f202ac-22cf-11d1-b12d b29062 > <ctx:referent> <ctx:identifier>info:pmid/ </ctx:identifier> <ctx:metadata-by-val> <ctx:format>info:ofi/fmt:xml:xsd:journal</ctx:format> <ctx:metadata> <jou:journal xmlns:jou= info:ofi/fmt:xml:xsd:journal > <jou:atitle>isolation of common receptor for coxsackie B <jou:jtitle>science</jou:jtitle> </ctx:referent> <ctx:requester> <ctx:identifier>urn:ip: </ctx:identifier> </ctx:requester> <ctx:service-type> <full-text>yes</full-text> </ctx:service-type> Resolver Referrer. </ctx:context-object>

30 Structure 1) Usage data analysis: usage and citation 2) Community dynamics 3) Recommender systems 4) Towards a global usage data aggregation 5) Challenges and new efforts 6) Conclusion

31 Just a few challenges Privacy and other legal issues: user and affiliation identification, legal implications of log storage, ownership, retention Data validity: usage definition, recording and representation, falsification issues Data analysis: community agreement on metrics, approaches to map trends, Aggregation: o o o o o different architectural frameworks: linking server-based, repositorybased scalability, anonymization trusted log repositories incentives for contribution

32 What s next? Emerging activities in the realm of applications of usage data : o o o o o COUNTER/SUSHI IRS: Interoperable Repository Statistics (U. Southampton) UKSG initiative regarding usage impact U. Stuttgart and U. College London: investigate linking server usage data aggregation for IRs in EU Mellon W. Foundation grants $881,000 to support the development of usage-based metrics at LANL: October 2007-October 2009

33 Graphical overview of Mellon project

34 Structure 1) Usage data analysis: usage and citation 2) Community dynamics 3) Recommender systems 4) Towards a global usage data aggregation 5) Challenges and new efforts 6) Conclusion

35 Conclusion Scholarship and scholarly communication is going through a revolution. Scholarly evaluation, the (global) mapping of scholarly activity, and discovery of scholarly materials will too. Usage data holds the promise of adding new tools to existing kit (that is largely concerned with bibliographic and citation data). But we are not there yet.

36 Some relevant papers: Johan Bollen and Herbert Van de Sompel. An architecture for the aggregation and analysis of scholarly usage data. In Joint Conference on Digital Libraries (JCDL2006), pages , June Johan Bollen and Herbert Van de Sompel. Mapping the structure of science through usage. Scientometrics, 69(2), Johan Bollen, Marko A. Rodriguez, and Herbert Van de Sompel. Journal status. Scientometrics, 69(3), December 2006 (arxiv.org:cs.dl/ ) J. Bollen, H. V. de Sompel, J. Smith, and R. Luce. Toward alternative metrics of journal impact: a comparison of download and citation data. Information Processing and Management, 41(6): , J. Bollen, R. Luce, S. Vemulapalli, and W. Xu. Detecting research trends in digital library readership. In Proceedings of the Seventh European Conference on Digital Libraries (LNCS 2769), pages 24-28, Trondheim, Norway, August Springer-Verlag. J. Bollen, R. Luce, S. Vemulapalli, and W. Xu. Usage analysis for the identification of research trends in digital libraries. D-Lib Magazine, 9(5), Find copies and other references at