Getting Started with Article-Level Metrics

Transcription

1 Getting Started with Article-Level Metrics Jean Liu and Euan Adie

2 Every researcher is a communicator Within academia Presentations and seminars Funding and ethics applications Journal articles and posters Term papers and essays Meetings and conferences Correspondence Within society Speaking at public events Press Social media Blogs How can we measure both academic and societal research impact?

3 New perspectives of impact ACADEMIC IMPACT SOCIETAL IMPACT Journal Impact Factor Citation counts + Download counts Page views Mentions in news reports Mentions in social media Mentions in blogs Reference manager readers etc. Traditional metrics Alternative metrics altmetrics

4 Each day, scholarly articles receive ~12,000 new mentions across social media, news, and blogs. Each week, ~20,000 unique articles are shared. That s 1 mention every 7 seconds! Mentions range in complexity, from quick shares to comprehensive reviews. Article-level metrics are worth paying attention to.

5 Altmetrics are o alternative metrics o new ways of measuring different, non-traditional forms of impact. o alternative to only using citations, not alternative to citations. o complementary to traditional citation-based analysis. Article-level metrics have come to refer to the altmetrics surrounding a scholarly article.

6 What does Altmetric do? We measure the online attention surrounding journal articles and datasets. News Social media You We collect and deliver article-level metrics to publishers, libraries, repositories, researchers, and more. Online reference managers

7 Altmetric Data: Details Pages See all the conversations and mentions. Monitor news mentions with the News Tracker. Visualise the impact with the Altmetric score. See article-level metrics and a score of attention below

8 The Altmetric Explorer The Altmetric Explorer web app can be used to filter and browse through attention for over 1,000,000 scholarly articles. Browse through altmetrics data for all mentioned papers. Use built-in analytic tools to compare journals.

9 The Altmetric Bookmarklet The free Altmetric Bookmarklet lets you instantly retrieve Altmetric details for any article. Altmetric it! Altmetrics data will appear on the righthand side of your page. Click the Altmetric it button brings up a paper s altmetrics. Installation instructions:

10 What can I do with article-level metrics?

11 You can use article-level metrics to 1 Track an article s uptake within a community. 2 Assess the overall impact of a body of scholarly work. 3 Characterise reader communities. 4 Discover new articles to read.

12 1 Track an article s uptake within a community Did a specific article reach the community it was intended for? EXAMPLE: Position paper by the American College of Physicians (ACP) and the Federation of State Medical Boards (FSMB), on physicians use of social media Target audience: American physicians who use social media TWITTER 48% of tweets mentioning the paper were sent from the US. 21% of tweets were sent by practitioners. Tweeted by several professional organisations with many followers. NEWS Mentioned in 2 articles on Forbes. BLOGS Discussed in-depth on 4 medical/physician-led blogs. Example from the Altmetric blog post The Doctor is Online : Altmetric details page:

13 2 Assess the impact of a body of scholarly work Use the powerful filters of the Altmetric Explorer. Filter papers by: Journal DOI prefix Publisher Pick what you want to look at and filter the rest out.

14 2 Assess the impact of a body of scholarly work Use the powerful filters of the Altmetric Explorer. Find specific papers by entering their unique identifiers, then look at their Altmetric scores. EXAMPLE: Altmetric conducted an analysis of the social impact of recent papers authored by Médecins Sans Frontières (MSF) staff. To retrieve the altmetrics data, we searched for a known set of DOIs and PubMed IDs. Publications by MSF staff. A graph from our altmetrics analysis of MSF papers.

15 3 Characterise reader communities Many distinct online communities come together to discuss papers. EXAMPLE: Nurses on Reddit, speaking about a 2011 paper published in Critical Care.

16 3 Characterise reader communities Identify details about reader communities using Altmetric data. Where are the readers? Altmetric article details pages show: Demographics of the readers A paper s online reach Who are the readers? What is the online reach of the community? View Twitter follower numbers.

17 3 Characterise reader communities Attention surrounding a single paper can reveal the existence of a large online community J Cell Sci 121, EXAMPLE: The bench scientists blogosphere was revealed by an essay on the importance of stupidity in scientific research The article was extremely popular with bench scientists on social media (Twitter, Reddit) and blogs. Lots of scientists responded by sharing their own experiences. Follows the trend of the bench scientists blogosphere, which often talks about: BLOGS New scientific articles Their own publications Lab/career experiences Example from the Altmetric blog post It s OK, Scientific Stupidity is Normal : Altmetric details page:

18 3 Characterise reader communities Attention surrounding a single paper can reveal the existence of a large online community. EXAMPLE: The concern of members of the Japanese public was revealed by the attention surrounding an article on the biological impacts of the Fukushima nuclear accident. 2012, Scientific Reports 2, 570 TWITTER: 68% of tweets were sent from Japan. 77% of tweeters were members of the public. 18% of tweeters were scientists. Twitter had been used to share news alerts during the Fukushima disaster now it was being used as a scientific news engine. Example from the Altmetric blog post Conversations About Disaster : Altmetric details page:

19 4 Discover new articles to read Use the Altmetric Explorer to rank articles by attention and find interesting papers. EXAMPLE: Filter by MEDLINE subject neurology Note: The MEDLINE subject filter only applies to journal classifications. We re working on developing a way to determine the subjects of individual articles.

20 4 Discover new articles to read Use the Altmetric Explorer to find interesting pre-prints. EXAMPLE: Filter by recently-mentioned articles on arxiv

21 ARTICLE LEVEL METRICS Track an article s uptake Identify communities ALTMETRIC EXPLORER Assess overall impact of a body of work Discover new papers

22 Interested? Let s talk. Jean: jean@altmetric.com Visit us on the web: Read our blog: Follow us on