Is the Altmetric Attention Score reproducible?
DOI:
https://doi.org/10.3145/thinkepi.2019.e13e08Keywords:
Bibliometrics, Altmetrics, Altmetric Attention Score, research indicators, Reproducibility.Abstract
The aim of this study is to know to what extent the Altmetric Attention Score (AAScore) can be reproduced. A sample of more than 28,000 research articles extracted from the public API of Altmetric.com were employed to reproduce the AAScore from the weights published in Altmetric.com. The indicator was correctly replicated in 13.2% of the cases, increasing the errors as the impact increases. The metrics that influence the most on the non-reproducibility are Tweeters (44%), news (16%) and blogs (12%). The main reason of this problem is the existence of non-published scoring criteria that impede the replication. This fact is an important limitation to the use of AAScore as research evaluation indicator.
References
Altmetric Support (2019). "How is the Altmetric Attention Score calculated?". Altmetric support, 29 Jul. https://help.altmetric.com/support/solutions/articles/6000060969-how-is-the-altmetric-score-calculated-
American Society for Cell Biology (2013). The San Francisco declaration on research assessment (DORA). Bethesda, MD: American Society for Cell Biology. https://sfdora.org/read
Hicks, Diana; Wouters, Paul; Waltman, Ludo; De-Rijcke, Sarah; Rafols, Ismael (2015). "Bibliometrics: the Leiden Manifesto for research metrics". Nature news, v. 520, n. 7548, pp. 429. https://www.nature.com/news/bibliometrics-the-leiden-manifesto-for-research-metrics-1.17351
Gumpenberger, Christian; Glí¤nzel, Wolfgang; Gorraiz, Juan (2016). "The ecstasy and the agony of the altmetric score". Scientometrics, v. 108, n. 2, pp. 977-982. https://doi.org/10.1007/s11192-016-1991-5
Mukherjee, Bhaskar; Subotić, Sinisa; Chaubey, Ajay-Kumar (2018). "And now for something completely different: the congruence of the Altmetric Attention Score´s structure between different article groups". Scientometrics, v. 114, n. 1, pp. 253-275. https://doi.org/10.1007/s11192-017-2559-8
NISO (2016). Outputs of the NISO Alternative Assessment Metrics Project. NISO RP-25-2016. ISBN: 978 1 937522 71 1 https://groups.niso.org/apps/group_public/download.php/17091/NISO%20RP-25-2016%20Outputs%20of%20the%20NISO%20Alternative%20Assessment%20Project.pdf
Ortega, José-Luis (2018). "Reliability and accuracy of altmetric providers: A comparison among Altmetric.com, PlumX and Crossref Event Data". Scientometrics, v. 116, n. 3, pp. 2123-2138 https://doi.org/10.1007/s11192-018-2838-z
