The future of AI in HEOR is on display at ISPOR 2025, and we're thrilled to announce that our latest research will be taking center stage: Our poster "Improve Analyst Accuracy in Systematic Literature Reviews Using Reliant Tabular and LLM-Based Relevance Scoring" showcases exciting advancements in the way artificial intelligence (AI) transforms literature research analysis. Here's everything you need to know about our participation and groundbreaking findings.
The ISPOR conference is the premier global event for health economics and outcomes research (HEOR) professionals. It’s where cutting-edge innovations meet practical applications, bringing researchers, medical communications professionals, pharmaceutical experts, and policymakers together in one dynamic space. This year, ISPOR continues its legacy of driving actionable discussions. For organizations like ours, it's a unique opportunity to contribute valuable research that can influence decision-making processes in healthcare and beyond.
What Our Research Covers
Our presentation focuses on enhancing the accuracy and efficiency of systematic literature reviews. Leveraging the power of fine-tuned large language models (LLMs) and proprietary engineering, we designed an advanced AI-based relevance-scoring agent, dramatically improving the process of identifying and analyzing relevant literature.
Here’s a brief overview of our findings - but be sure to visit us at the conference for the full story:
Objectives
This study evaluated how incorporating LLM fine-tuning could address inefficiencies in standard SLR workflows, not only by improving recall rates but by reducing false negatives - a critical pain point for analysts.
Methods
We developed and embedded a fine-tuned version of the open-source Llama 3.1 70B LLM within an advanced data agent to:
- Perform retrieval-augmented generation of document summaries.
- Employ relevance scoring algorithms based on chain-of-thought prompting.
- Replicate SLR workflows initially tested by Kerr et al., comparing AI-driven effectiveness against both GPT-4 and human analysts.
Results
The results were groundbreaking:
- Recall rate of 100% for relevant abstracts identified by the AI agent, outperforming both GPT-4 (87%) and a junior analyst (92%).
- The system’s false positive rate was comparable to human efforts (9.1%), with minimal redundant inputs.
- Collaborative processes between human analysts and AI demonstrated significant value by filtering false positives quickly while reducing missed key sources.
These outcomes confirm the utility of advanced AI solutions in high-stakes research contexts, paving the way for trustworthy and efficient AI tools that improve analyst workflows.
Why It Matters
False negatives in SLRs are costly, forcing analysts to revisit sources, delay timelines, and risk overlooking crucial information. Our research proves that fine-tuning LLMs with targeted datasets, combined with intelligent tabular methods, isn’t just about improving speed: it’s about bolstering accuracy and trust in every step of the review process.
By addressing core challenges in evidence synthesis, our data agent sets a new benchmark for what’s possible with modern AI integration.
Meet Us at ISPOR 2025
At ISPOR, we’re not just sharing facts and figures. We’re inviting you to explore the possibilities of AI-engineered research tools. Beyond presenting our findings, our team will be available to discuss how this technology applies to your specific challenges, even beyond HEOR.
Here’s How You Can Engage:
- Visit Our Poster Session
Learn about the details of our research firsthand. We’ll walk you through the methodology and demonstrate why fine-tuned LLMs are the future of literature research:
Poster Session: Poster Session 2
Poster Code: MSR37
Poster Session Date/Time: Wednesday, May 14, 4:00 - 7:00 PM
Discussion Period: 6:00 - 7:00 PM
- Schedule a Demo
Curious about how this technology can apply to your work? Book a demo with us through our website to explore our solutions tailored specifically to LS organizations. Request a Demo Here.
- One-on-One Meetings
Want a deeper conversation? Can’t wait until May? Our experts are available for individual meetings. Have your toughest questions ready—we’re excited to collaborate and exchange ideas. Request a Meeting Here.
Accelerate Your Research with Our AI-Powered Solutions
The ISPOR 2025 conference is just the beginning. Our research underscores our commitment to providing trustworthy, high-performance AI tools that integrate seamlessly into your operations: helping researchers, systematic reviewers, and commercial strategy professionals gain insights faster and with confidence.
Take the first step towards transforming your organization’s workflows with our AI-driven solutions. Whether you’re an SLR expert, a pharmaceutical executive, or part of a growing medical communications team, we’re here to help you navigate complexity with precision.
