Advances in altmetrics – BoF

Advances in Altmetrics BOF session Research Support Community Day 2017

Thanks to Toni Le Roux for note taking during the session

From Altmetrion Conference

The Altmetric Roadmap is a great source of information

 

Some stats

  • 16 types of attentions are scored including 41 million individual mentions
  • Only 30% of altmetric attention is for articles published in the last month. The rest are older so there is definite value in telling your researchers to tweet from their ‘back catalogue’ – especially when relevant (ie if a story is getting media exposure and your academic has relevant research)
  • 10% of publications get 50% of the altmetic attention
  • Twitter receives the biggest number of mentions
  • 59% of tweets are to open access versions of publications
  • Average time from being published to being cited in policy is 2 years – sooner if publication is open access.
  • Only 2% of publications are ever cited in policy

 

New Features from Altmetric.com

  • Now tracking clinical trials and datasets with an ISBN or DOI
  • Shortly will begin tracking Google Books, Amazon books, book reviews
  • Open Syllabus is being tracked: http://opensyllabusproject.org/faq/ (no centralised place to track Aust syllabus at present.
  • In Explorer Scopus citations are included and Web of Science about to be included if the institution has appropriate subscriptions. This will enable traditional and alternative metrics to be visualised to determine correlation.
  • Adding a journal comparison facility shortly
  • Adding conferences with an identifier soon

 

Forthcoming/wishlist Features

  • Hoping to add journal, people and group metrics and linking related outputs. Altmetric want to create their own.
  • Hoping to add Non traditional research outputs if they can find a way to identify them

 

Suggestions/tips:

  • Remind your media units to include the DOI or ISBN in the press releases
  • The story is in the narrative, not the raw numbers – interrogate the data to discover the narrative
  • Improvements to the policy tracker

Over the last few months, the Altmetric development team have been busy re-writing our policy tracking system from the ground up. The new policy tracker now picks up many more matches in each document – via text mining and extracting DOIs, ISBNs and URLs – with improved reference matching accuracy.

 

Discussion on the Day

  • Jackie W advised Altmetric will add sources if they have an RSS feed.
  • Aparenty advise Amy Rees at Altmetric in London if you have content to be indexed
  • PlumX in Ebsco provides good metrics but PlumX have just been purchased by Elsevier so may not be free for much longer. Elsevier had purchased Plum Analytics from EBSCO. PlumX metrics will now be included in Mendeley, Scopus, ScienceDirect, SciVal and Pure, as well as Elsevier’s partner sites.
  • Bookmetrix http://www.bookmetrix.com/ (Springer) They do track articles too, even if the article fulltext is not available.
  • Snowball metrics in Scopus offer an alternative to using just raw metrics
  • ImpactStory – some unis are recommending their researchers create accounts. Useful tool but some say just one more account to create and keep updated.
  • DOI’s are needed to track NTROs. Contact ANDS or Figshare etc to get these minted and promote the importance of including them
  • Some libraries experienced considerable drops in their Altmetric scores due to bad implementation, so keep an eye on this if you add an API.
  • Several libraries have just acquired Explorer, and are about to start implementation.
  • CSIRO is currently implementing Explorer and reported problems getting timely support as the team is UK based and the time difference has been an issue. They have also been advised that they can only use one API.
  • Please contact Altmetric if you know of any sources that would like their content to be tracked – example discussed where multiple local newspapers contacted Altmetric and they were more than happy to track this content – Newspapers must have an RSS feed to be trackable.
  • Two research institutes connected with WSU use API’s on their websites. Recently the Hawkesbury Institute had an article published in Nature. The API tracked each mention in real time and this was great publicity on the website as well as an incentive for the other researchers.

How to engage researchers with Altmetric data

  • General discussion around engagement strategies– consensus was that not many researchers seem to know about it, or understand what it does – although some, such as those who have submitted to The Conversation seem to be very aware. May well have a use in helping tell a researcher’s “Impact Story” especially as there is now a trend to move away from journal based metrics in grant applications. La Trobe’s Cochrane group recently did an analysis on authors and used Altmetric. Medicine/Science researchers are particularly interested in policy mentions – great for adding into Impact Story.

Will Altmetrics be used in the pilot engagement exercise?

  • General consensus is that this would be of benefit. The ARC/NISA – Engagement and Impact Assessment Consultation Paper The ARC/NISA – Engagement and Impact Assessment Consultation Paper  published in May 2016  states that “data providers that collect research metrics, for example, Altmetrics… could be used as measures of impact in disciplines where, patenting for example, is not widespread. Although the metrics used to assess impact (or pathways to impact) may be less developed than those measuring engagement, it may be possible through the development of the assessment to identify a suite of metric based indictors that can form a robust assessment of research impact.”

Altmetrics may be of benefit and the pilot exercise to assess potential metrics is currently being undertaken.

 

Some useful/interesting links

 

http://maestrelab.blogspot.com.au/2014/11/how-i-use-altmetrics-data-in-my.html

How I use altmetrics in my proposals

This study has also been widely discussed in the social media, as indicated by an Altmetric score of 50, which makes it scoring higher than 98% of its contemporaries and includes it into the top 5% of all the articles tracked by Altmetric (more than 1,660,000; see http://goo.gl/aNVUUk for details). In addition, this work has been featured by newspapers, magazines, web pages and blogs from around the world (see http://goo.gl/JrJ4EY for a selection of news).

 

This article has attracted lots of attention from scientists since its publication, as it was the object of a “News & Views” in Nature (Wardle, 2013, Nature 502: 628-629), and has been viewed more than 6300 times since its publication two months ago (see http://goo.gl/EuHYOv for details). This article has also been widely discussed in the social media, as indicated by an Altmetric score of 151, which makes it scoring higher than 99% of its contemporaries and includes it into the top 5% of all the articles tracked by Altmetric (more than 1,730,000; see http://goo.gl/f3fu3A for details). This study has also received substantial attention by newspapers, magazines, web pages and blogs from around the world (see http://goo.gl/CU2hSR for a selection of news).

 

http://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1002003

 

Alternative Perspectives on Impact: The Potential of ALMs and Altmetrics to Inform Funders about Research Impact

 

Shema et al. [10] define these new metrics as “web-based metrics for the impact of scholarly material, with an emphasis on social media outlets as sources of data”. In addition to academic citations, these metrics aggregate views, downloads, discussions, and recommendations of research outputs across the scholarly web [11], as well as citations in nonacademic communications such as policy documents [12], patent applications, and clinical guidelines.

  1. Shema H, Bar-Ilan J, Thelwall M (2014) Do blog citations correlate with a higher number of future citations? Research blogs as a potential source for alternative metrics. J Am Soc Inf Sci Technol 65: 1018–1027.
  2. Lin J, Fenner M (2013) Altmetrics in evolution: defining & redefining the ontology of article-level metrics. ISQ 25: 20–26.

Box 1. Behind the Metrics: Social Media Activity as a Proxy of Engagement in the Policy Sphere

Though the paper remained uncited for three months following its publication, it was tweeted at four times the average rate for PLOS Medicine articles published in 2013 and 70 times the average rate of all Wellcome Trust–associated articles published across the family of PLOS journals (see Figure 1). Many of the accounts tweeting about the paper belonged to key influencers, including members of the European Parliament, international nongovernmental organisations (NGOs), and a sector manager for Health, Nutrition, and Population at the World Bank, suggesting that a study with an apparently parochial focus had nonetheless had a rapid influence all over the world.

Thelwall et al. [15] found statistically significant associations between metric scores and citations for all metrics for which there was sufficient evidence, suggesting that altmetrics may have some potential as a means of predicting academic impact soon after publication.

  1. Thelwall M, Haustein S, Larivière V, Sugimoto CR (2013) Do Altmetrics Work? Twitter and Ten Other Social Web Services. PLoS ONE 8: e64841 doi:10.1371/journal.pone.0064841