AutoCriteria: a generalizable clinical trial eligibility criteria extraction system powered by large language models
Presenters
Surabhi Datta, PhD
Xiayang Wang, PhD, NLP
Statement of Purpose
Eligibility criteria in clinical trials play a pivotal role in patient recruitment and in safety and treatment evaluation, leading to advancements in medical knowledge and improved patient care. Traditional approaches to extract eligibility criteria from trial text are often laborious, time-consuming, and prone to human errors. To this end, the development of automated systems for eligibility criteria extraction using natural language processing (NLP) techniques can expedite the process and reduce errors, thereby facilitating comprehensive trial evaluation and improved recruitment processes. Moreover, computable representations that facilitate electronic health record (EHR) interoperable eligibility criteria enable trial coordinators/recruiters to automatically match them against the corresponding clinical information in EHRs and notify providers about potential patients eligible for a specific trial.
Recently, large language models have gained traction in the general as well as biomedical and clinical domains. These models encode general knowledge in their parameters and are designed to work across different NLP tasks. They also encode clinical knowledge and have been employed for various medical applications. Inspired by this, we propose to leverage such a powerful and general-purpose model, OpenAI’s GPT-4, to extract eligibility criteria from clinical trial text.
We developed AutoCriteria to extract granular eligibility criteria including identifying temporal information and criteria conditions from clinical trial documents (collected from ClinicalTrials.gov) belonging to a wide variety of diseases, ranging from oncology (e.g., breast cancer) to neurological progressive diseases (e.g., Alzheimer’s) to rare diseases (e.g., sickle cell disease). AutoCriteria prompts are flexible and generalizable across different disease domains and can be easily extended to new diseases without requiring manual annotation and training. Such a generalizable and scalable criteria extraction system could significantly streamline the patient recruitment process and expedite the construction of criteria knowledge base.
Learning Outcomes
- Explain the importance of creating a structured representation of free-text clinical trial eligibility criteria for patient recruitment to trials.
- Demonstrate the construction of generalizable prompts for eligibility criteria extraction from clinical trial documents across different disease domains.
Additional Information
Disclosures
Disclosures of relevant financial relationships of all planners and presenters of the Journal Club.
Planning Committee
The planning committee and reviewers reported that they have no relevant financial relationship(s) with ineligible companies to disclose.
- Joanna Abraham, PhD, FACMI, FAMIA
- Jifan Gao, MS
- Frances Hsu, BS, MS
- Sonish Sivarajkumar
- Song Wang, MS
- Faisal Yaseen
AMIA Staff
The following staff have no relevant financial relationship(s) with ineligible companies to disclose.
- Jennifer Wahl
- Melissa Kauffman