Utilization and Impact of Artificial Intelligence-Generated Draft Replies to Patient Messages in Pediatrics
AI-generated draft replies to patient messages have not been studied in pediatric clinical settings, raising concerns about the feature’s acceptability and applicability to this context. In this single-site cohort study, users from both pediatric- and adult-facing specialties were given access to AI-generated drafts. Pediatric providers reported significant reduction in task load associated with responding to patient messages and recommended the tool more highly than adult providers despite overall lower utilization of generated drafts.
Learning Objectives
- Evaluate the effectiveness and acceptability of AI-generated draft replies in reducing task load for pediatric providers, and compare provider responses across pediatric and adult clinical settings.
Speaker
- April Liang, MD (Stanford University)
Adoption and Utility: Evaluation of Usage Rate and Editing Overlap for Artificial Intelligence-Drafted Replies to Patient Messages
This session will provide insight into the implementation and evaluation of Generative Artificial Intelligence (AI) technology to respond to patient portal messages. Attendees will learn about the high variation in usage rates and editing efforts, prompting a discussion of best practices for implementation of these technologies in attendees’ own clinical settings. They will also learn data analysis and visualization strategies they can apply when implementing and evaluating the utility of AI tools in clinical practice.
Learning Objectives
- Gain insight into the implementation and evaluation of Generative Artificial Intelligence (AI) technology performed at another organization.
Speaker
- Aasf Hanish, MPH (Penn Medicine)
Enhancing Patient Communication: The Impact of LLMs on Care Team Messaging Workflows
The deployment of AI-generated draft replies in healthcare settings has introduced new dynamics in how medical assistants (MAs), nurses (RNs), and physicians (MDs) interact with patient messages. We performed inductive coding of messages sent with and without access to draft replies to investigate their influence on scope creep and error rates in message handling by MAs and found that scope creep was rare and did not appear to be adversely affected by the use of AI-generated draft messages. We also explored the utility of fine-tuned LLMs for the automated triage of incoming messages, which would have the potential to redirect incoming patient messages with clinical questions to the appropriate role with the appropriate urgency. We found that even small open-source LLMs such as Llama-3-8B was able to achieve high accuracy when compared to human labels. These results underscore the potential for increased automation through the use of LLMs in clinical settings.
Learning Objectives
- Understand how AI generated draft replies to patient messages can impact user behavior
Speaker
- Stephen Ma, MD, PhD (Stanford University School of Medicine)
Automation of Critical Lab Result Communication Improves Lab Efficiency
An academic health system designed and implemented an automated critical lab results notification system to enhance lab staff efficiency. Based on technologies from Epic, Twilio, UiPath, and Sunquest, a bot called the provider using interactive voice response and, after confirming identity, delivered the results with Epic secure chat. The automation achieved a 27.4% success rate and maintained the 30 minute turnaround time benchmark.
Learning Objectives
- Develop a workflow to automate critical lab results using Robotic Process Automation.
Speaker
- Jonathan Austrian, MD (NYU Langone Medical Center)
About CME/CNE Credit
The following information pertains to individual sessions included in the AMIA 2025 Clinical Informatics Conference On Demand product. A total of 16.75 CME/CNE credits may be earned if all sessions are completed.
Continuing Education Credit
Physicians
The American Medical Informatics Association is accredited by the Accreditation Council for Continuing Medical Education (ACCME) to provide continuing medical education for physicians.
The American Medical Informatics Association designates this online enduring material for 16.75 AMA PRA Category 1™ credits. Physicians should claim only the credit commensurate with the extent of their participation in the activity.
Claim credit no later than within two years of the release date or within one year of your purchase date, whichever is sooner.
ANNC Accreditation Statement
The American Medical Informatics Association is accredited as a provider of nursing continuing professional development by the American Nurses Credentialing Center's Commission on Accreditation.
- Nurse Planner (Content): Robin Austin, PhD, DNP, DC, RN, NI-BC, FAMIA, FAAN
- Approved Contact Hours: 16.75 participant maximum CME/CNE
ACHIPsTM
AMIA Health Informatics Certified ProfessionalsTM (ACHIPsTM) can earn 1 professional development unit (PDU) per contact hour.
ACHIPsTM may use CME/CNE certificates or the ACHIPsTM Recertification Log to report 2025 CIC sessions attended for ACHIPsTM Recertification.
Claim credit no later than within two years of the release date or within one year of your purchase date, whichever is sooner.