Temporal Rule Mining for Enhanced Risk Pattern Extraction: A Case Study with Acute Kidney Injury
Association rule mining is a widely used data mining technique for extracting knowledge from large datasets. Its application in healthcare involves uncovering meaningful patterns within electronic health records (EHR) to inform clinical decision-making and treatment strategies. However, most association rule mining studies overlook temporal information, potentially missing valuable patterns associated with specific time periods or events. In recent years, several methods have been developed to mine temporal association rules, offering improved predictive and descriptive capabilities. We propose a multi-step rule mining framework that utilize temporal pattern mining algorithm to extract actionable and temporal risk patterns for acute kidney injury (AKI) using EHR data. Our algorithm discovered around 26K rules, with low support and high confidence, centered at 40 actionable. The derived rules have a median support of 0.057 and confidence of 0.49. We highlight selected rules, their potential etiology, and provide a network view of more specific actionable insights.
Learning Objectives
- Deploy temporal association rule mining on Electronic Health Record (EHR) data to generate actionable insights.
Speaker
- Ho Yin Chan, PhD (University of Florida)
Studying Veteran food insecurity longitudinally using electronic health record data and natural language processing
As AI adoption in healthcare grows, there is an increasing need for continuous monitoring after implementation, known as algorithmovigilance. While existing tools provide some support, few systems enable comprehensive proactive oversight and governance of AI across a healthcare system. This study outlines the human-centered design process used to develop the Vanderbilt Algorithmovigilance Monitoring and Operations System (VAMOS). We describe key insights and design recommendations to guide the development of robust algorithmovigilance tools for healthcare institutions.
Learning Objectives
- Understand the role food insecurity plays in a patient's health and the role healthcare systems play in addressing it.
- Learn how food insecurity can be measured over time using electronic health record data.
Speaker
- Alec Chapman, MS (University of Utah)
Large Language Models in Biomedical Named Entity Recognition
Large language models (LLMs), like GPT-4, have revolutionized natural language processing (NLP), demonstrating exceptional performance across various tasks. However, their effectiveness in biomedical named entity recognition (BioNER) remains limited due to the need for domain-specific knowledge. This study focuses on fine-tuning general-domain LLMs, specifically Llama-2 models, for BioNER tasks. We convert five BioNER datasets from the BLURB benchmark into an instruction-following format to optimize fine-tuning. Our approach incorporates zero-shot prompting, Chain-of-Thought (CoT) reasoning, and a perplexity-based evaluation method. We evaluate the fine-tuned Llama-2 models on the AnatEM, BioNLP11EPI, and BioNLP13GE datasets, and our method consistently outperforms baseline models such as UniNER-7B, InstructUIE-11B, and BioLinkBERT. Furthermore, larger models like Llama2-13B demonstrate superior performance compared to smaller ones, highlighting the significance of model parameters. This study underscores the potential of instruction-tuned LLMs for BioNER tasks and opens avenues for their application in other biomedical NLP tasks.
Learning Objectives
- Understand how instruction-following fine-tuning enhances generative large language models (LLMs) for biomedical named entity recognition (BioNER) by simulating real-world scenarios through question-answering.
- Explore how advanced prompting strategies—such as zero-shot prompting and Chain-of-Thought reasoning—can improve model performance, and how perplexity-based evaluation can be used for model selection in BioNER tasks.
- Discuss the benefits of fine-tuning LLMs using instruction-following data as a promising approach to improving BioNER performance.
Speaker
- Cong Sun, Ph.D. (Weill Cornell Medicine)
Identifying Necrotizing Enterocolitis Diagnosis from Progress Notes Using Natural Language Processing and Classification Models
Necrotizing Enterocolitis (NEC) is a serious neonatal condition with high mortality and morbidity. This study utilized NLP to analyze progress notes, enhancing NEC patient identification accuracy and specificity. The method surpasses manual chart review and traditional cohort discovery approaches. Improving precision in patient classification significantly reduces the reliance on labor-intensive reviews, offering a scalable solution for NEC identification.
Learning Objectives
- Improve cohort discovery using clinical notes for the neonatal population.
Speaker
- Woo Yeon Park, MS (Johns Hopkins University)
A Multipronged Approach: Harnessing LLMs and NLP on Structured and Unstructured Data to Enhance Traditional Chart Review
Accurate and efficient chart review is crucial for extracting clinically relevant information. It is performed for several purposes from validation studies to care assessments. The manual review process is time consuming, costly, and prone to human error. Using AI by leveraging LLMs combined with practical NLP, we can enhance the chart review process in a meaningful way. At MGB, we developed a flexible “reasoning chain pipeline” using LLMs and NLP to improve specificity and sensitivity.
Learning Objectives
- Describe the limitations of traditional manual chart review and explain how AI technologies, including LLMs and NLP, can address these challenges.
- Demonstrate an understanding of the “reasoning chain pipeline” approach developed at MGB and evaluate its impact on the specificity and sensitivity of clinical data extraction.
Speaker
- Nich Wattanasin, MS (Mass General Brigham)
A Comparison of Rule-based, Machine Learning, and Large Language Model Methods for Extracting Adverse Events from Clinical Notes
Adverse event detection is a necessary component of clinical trial data collection and currently requires massive expenditure of effort in the form of manual chart review. NLP techniques can automate this effort, but their performance is uncertain within the context of clinical trial replicability. We developed a rule-based AE detection approach and evaluated it alongside an LLM and a previously piloted best-of-breed technique in notes for patients with mantle cell lymphoma.
Learning Objectives
- Understand different methods for evaluating and extracting adverse events from clinical trial notes.
Speaker
- Aashri Aggarwal, BA (Weill Cornell Medicine)
Continuing Education Credit
Physicians
The American Medical Informatics Association is accredited by the Accreditation Council for Continuing Medical Education (ACCME) to provide continuing medical education for physicians.
The American Medical Informatics Association designates this online enduring material for 1.0 AMA PRA Category 1™ credits. Physicians should claim only the credit commensurate with the extent of their participation in the activity.
Claim credit no later than March 10, 2028 or within two years of your purchase date, whichever is sooner. No credit will be issued after March 10, 2028.
ACHIPsTM
AMIA Health Informatics Certified ProfessionalsTM (ACHIPsTM) can earn 1 professional development unit (PDU) per contact hour.
ACHIPsTM may use CME/CNE certificates or the ACHIPsTM Recertification Log to report 2025 Summit sessions attended for ACHIPsTM Recertification.
Claim credit no later than within three years of the release date or within two years of your purchase date, whichever is sooner.