By Aaron Tay, Head, Data Services
In 2024, the Financial Times created a new ranking of business schools - FT research insights report that went beyond traditional citation counts, focusing instead on "rigour, resonance and relevance." To do this, they drew from others platforms like Overton, Open Syllabus, Scite, OpenAlex and more—alternatives to conventional sources like Web of Science or Scopus. More recently, they published the 2025 version of it.
What do these alternative platforms measure, and how do they work?
As such, SMU Libraries curated a series of talks exploring how platforms such as Overton, Open Syllabus, Scite.ai, and Open Alex track alternative indicators - from using policy documents to course syllabi to changing the nature of the way we track scholarly citations themselves.
The aim here is to help researchers discover what these platforms offer and consider how they might complement traditional approaches to understanding research influence.
As of writing, we have already finished our first talk in the series conducted by Euan Adie, CEO of Overton. He showed how the Overton web platform can be used to help monitor and improve the impact academic institutions have on public policy, both nationally and internationally by tracking citations from policy documents and not just scholarly works. We have an institutional subscription to Overton.
While policy citations in policy documents are often very similar to journal article citations, not all policy documents will have a standard reference section at the end of the paper or report, which is why the citation parser of Overton needs to be more flexible to pick up citations in the text, footnotes etc. On top of that, certain policy documents such as legislations often do not cite specific articles but mention the work of specific researchers in a certain area. Overton uses named entity extraction to try to detect such "mentions".
My main takeaway: the policy influence of academic work is often indirect—papers are frequently cited first by intermediary “data brokers” (e.g. working papers in think tanks, ngos) before appearing in legislation. This is also why Overton tracks cites to working papers, white papers etc.
One slide showed that only ~30% of references extracted from policy documents point to scholarly articles, while ~40% cite other policy documents and ~10% cite news articles!
Euan also highlighted China as an interesting case: while legislation and policy outputs exist, many intermediary data-broker sources aren’t discoverable online. He also shared with us the idea of doing Overton’s second-order citation tracking via Key Sources feature to surface these indirect influence pathways.
We discussed whether policy citations skew toward open access articles, top-tier journals, and “local” research. There appears to be a preference in large countries for citing domestic research. Singapore seems to be an outlier—likely because our national research base is smaller and less comprehensive compared to larger countries. Practical advice for increasing policy citations: active engagement with the policy community.
If you missed the talk, the recording is available here.
Upcoming Talks in the series
The other talks in the series are
Exploring research impact beyond traditional metrics: Open Syllabus on education
Date: Tuesday, 11 November 2025
Time: 10:00 AM – 11:00 AM
Venue: Online via Zoom
Summary: Dr. Karaganis will introduce Open Syllabus, a global non-profit archive that collects syllabi from thousands of colleges and universities, maps the curriculum, and builds tools to make that data useful to students and educators. He will discuss Open Syllabus's role in developing teaching-based altmetrics for faculty, tools for curricular design, and resources to support student mobility.
Speaker: Joe Karaganis, Director of Open Syllabus. He has published on information policy and knowledge management, including Shadow Libraries: Access to Knowledge in Global Higher Education (MIT, 2018).
Exploring research impact beyond traditional metrics: Scite on citation sentiment
Date: Tuesday, 18 November 2025
Time: 10:00 AM – 11:00 AM
Venue: Online via Zoom
Summary: Institutional evaluations have traditionally been limited in that their assessment of research impact via citations received has relied on a simple count of citations. In this talk, Sean C. Rife, PhD, co-founder of Scite, and a Professor of Psychology at Murray State University, will present a new approach to rankings that incorporates the content of citations using Scite's data, which classifies citations as supporting, mentioning, or contrasting.
Sean will outline the process of collecting citation data, classifying citations, tying published works to institutions, and how these data are used to compute a content-weighted impact score for institutions that incorporates both the volume of citations received as well as the number of supporting and contrasting citations received. Examples from Scite's rankings report for institutions and journals will be included, and limitations will be discussed.
Speaker: Sean C. Rife, PhD is a co-founder of Scite, and a Professor of Psychology at Murray State University in Murray, Kentucky, USA. His research and teaching focuses on moral and political psychology, as well as metascience and the role bibliometrics can play in the reform of scientific institutions. He is Head of Academic Relations at Research Solutions, Inc., Scite’s parent company, where he is heading up their research on entity rankings.
Exploring research impact beyond traditional metrics: OpenAlex on open research intelligence
Date: Monday, 24 November 2025
Time: 10:00 AM – 11:00 AM
Venue: Online via Zoom
[Registration opening soon]
