AutoCriteria: a generalizable clinical trial eligibility criteria extraction system powered by large language models
Read the abstract
Presenters
Statement of Purpose
Eligibility criteria in clinical trials play a pivotal role in patient recruitment and in safety and treatment evaluation, leading to advancements in medical knowledge and improved patient care. Traditional approaches to extract eligibility criteria from trial text are often laborious, time-consuming, and prone to human errors. To this end, the development of automated systems for eligibility criteria extraction using natural language processing (NLP) techniques can expedite the process and reduce errors, thereby facilitating comprehensive trial evaluation and improved recruitment processes. Moreover, computable representations that facilitate electronic health record (EHR) interoperable eligibility criteria enable trial coordinators/recruiters to automatically match them against the corresponding clinical information in EHRs and notify providers about potential patients eligible for a specific trial.
Recently, large language models have gained traction in the general as well as biomedical and clinical domains. These models encode general knowledge in their parameters and are designed to work across different NLP tasks. They also encode clinical knowledge and have been employed for various medical applications. Inspired by this, we propose to leverage such a powerful and general-purpose model, OpenAI’s GPT-4, to extract eligibility criteria from clinical trial text.
We developed AutoCriteria to extract granular eligibility criteria including identifying temporal information and criteria conditions from clinical trial documents (collected from ClinicalTrials.gov) belonging to a wide variety of diseases, ranging from oncology (e.g., breast cancer) to neurological progressive diseases (e.g., Alzheimer’s) to rare diseases (e.g., sickle cell disease). AutoCriteria prompts are flexible and generalizable across different disease domains and can be easily extended to new diseases without requiring manual annotation and training. Such a generalizable and scalable criteria extraction system could significantly streamline the patient recruitment process and expedite the construction of criteria knowledge base.
Learning Outcomes
- Explain the importance of creating a structured representation of free-text clinical trial eligibility criteria for patient recruitment to trials.
- Demonstrate the construction of generalizable prompts for eligibility criteria extraction from clinical trial documents across different disease domains.
Additional Information
The target audience for this activity includes physicians, nurses, other healthcare providers, and medical informaticians.
No commercial support (funding from a governmental agency, ineligible company or in-kind donation) was received for this activity.
Completion of this “Enduring Material” is demonstrated by participating in the live webinar or viewing the on-demand recording, engaging with presenters during the live session by submitting questions, and completing the evaluation survey at the conclusion of the course.
Learners may claim credit and download a certificate upon submission of the evaluation. Participation in additional resources and the course forum is encouraged but optional.
ACCME Accreditation Statement
The American Medical Informatics Association is accredited by the Accreditation Council for Continuing Medical Education (ACCME) to provide continuing medical education for physicians.
Designation Statement
The American Medical Informatics Association designates this Enduring activity for a maximum of 1 AMA PRA Category 1 Credit(s)™. Physicians should claim only the credit commensurate with the extent of their participation in the activity.
ANCC Accreditation Statement
The American Medical Informatics Association is accredited as a provider of nursing continuing professional development by the American Nurses Credentialing Center's Commission on Accreditation.
Nurse Planner (Content): Robin Austin, PhD, DNP, DC, RN, NI-BC, FAMIA, FAAN
Approved Contact Hours: 1 CME/CNE
*Learners may earn 1 contact hour for each monthly Journal Club session, for a maximum of 6 contact hours per year. To receive the full 6 contact hours, participants must either attend the live webinar or view the on-demand recording for each Regularly Scheduled Series (RSS) Journal Club presentation.
It is the policy of the American Medical Informatics Association (AMIA) to ensure that Continuing Medical Education (CME) activities are independent and free of commercial bias. To ensure educational content is objective, balanced, and guarantee content presented is in the best interest of its learners and the public, the AMIA requires that everyone in a position to control educational content disclose all financial relationships with ineligible companies within the prior 24 months. An ineligible company is one whose primary business is producing, marketing, selling, re-selling or distributing healthcare products used by or on patients. Examples can be found at accme.org.
In accordance with the ACCME Standards for Integrity and Independence in Accredited Continuing Education, AMIA has implemented mechanisms prior to the planning and implementation of this CME activity to identify and mitigate all relevant financial relationships for all individuals in a position to control the content of this CME activity.
In accordance with the ACCME Standards for Integrity and Independence in Accredited Continuing Education, AMIA has implemented mechanisms prior to planning and implementation of this CME activity to identify and mitigate all relevant financial relationships for all individuals in a position to control the content of this CME activity.
Faculty and planners who refuse to disclose any financial relationships with ineligible companies will be disqualified from participating in the educational activity.
For an individual with no relevant financial relationship(s), course participants must be informed that no conflicts of interest or financial relationship(s) exist.
Disclosures
Disclosures of relevant financial relationships of all planners and presenters of the Journal Club.
Planning Committee
The planning committee and reviewers reported that they have no relevant financial relationship(s) with ineligible companies to disclose.
- Joanna Abraham, PhD, FACMI, FAMIA
- Jifan Gao, MS
- Frances Hsu, BS, MS
- Sonish Sivarajkumar
- Song Wang, MS
- Faisal Yaseen
AMIA Staff
The following staff have no relevant financial relationship(s) with ineligible companies to disclose.
- Jennifer Wahl
- Melissa Kauffman