Skip to main content

Utilization and Impact of Artificial Intelligence-Generated Draft Replies to Patient Messages in Pediatrics

AI-generated draft replies to patient messages have not been studied in pediatric clinical settings, raising concerns about the feature’s acceptability and applicability to this context. In this single-site cohort study, users from both pediatric- and adult-facing specialties were given access to AI-generated drafts. Pediatric providers reported significant reduction in task load associated with responding to patient messages and recommended the tool more highly than adult providers despite overall lower utilization of generated drafts.

Learning Objectives

  • Evaluate the effectiveness and acceptability of AI-generated draft replies in reducing task load for pediatric providers, and compare provider responses across pediatric and adult clinical settings.

Speaker

  • April Liang, MD (Stanford University)

Adoption and Utility: Evaluation of Usage Rate and Editing Overlap for Artificial Intelligence-Drafted Replies to Patient Messages

This session will provide insight into the implementation and evaluation of Generative Artificial Intelligence (AI) technology to respond to patient portal messages. Attendees will learn about the high variation in usage rates and editing efforts, prompting a discussion of best practices for implementation of these technologies in attendees’ own clinical settings. They will also learn data analysis and visualization strategies they can apply when implementing and evaluating the utility of AI tools in clinical practice.

Learning Objectives

  • Gain insight into the implementation and evaluation of Generative Artificial Intelligence (AI) technology performed at another organization.

Speaker

  • Aasf Hanish, MPH (Penn Medicine)

Enhancing Patient Communication: The Impact of LLMs on Care Team Messaging Workflows

The deployment of AI-generated draft replies in healthcare settings has introduced new dynamics in how medical assistants (MAs), nurses (RNs), and physicians (MDs) interact with patient messages. We performed inductive coding of messages sent with and without access to draft replies to investigate their influence on scope creep and error rates in message handling by MAs and found that scope creep was rare and did not appear to be adversely affected by the use of AI-generated draft messages. We also explored the utility of fine-tuned LLMs for the automated triage of incoming messages, which would have the potential to redirect incoming patient messages with clinical questions to the appropriate role with the appropriate urgency. We found that even small open-source LLMs such as Llama-3-8B was able to achieve high accuracy when compared to human labels. These results underscore the potential for increased automation through the use of LLMs in clinical settings.

Learning Objectives

  • Understand how AI generated draft replies to patient messages can impact user behavior

Speaker

  • Stephen Ma, MD, PhD (Stanford University School of Medicine)

Automation of Critical Lab Result Communication Improves Lab Efficiency

An academic health system designed and implemented an automated critical lab results notification system to enhance lab staff efficiency. Based on technologies from Epic, Twilio, UiPath, and Sunquest, a bot called the provider using interactive voice response and, after confirming identity, delivered the results with Epic secure chat. The automation achieved a 27.4% success rate and maintained the 30 minute turnaround time benchmark.

Learning Objectives

  • Develop a workflow to automate critical lab results using Robotic Process Automation.

Speaker

  • Jonathan Austrian, MD (NYU Langone Medical Center)

 


About CME/CNE Credit

The following information pertains to individual sessions included in the AMIA 2025 Clinical Informatics Conference On Demand product. A total of 16.75 CME/CNE credits may be earned if all sessions are completed.

Continuing Education Credit

Physicians

The American Medical Informatics Association is accredited by the Accreditation Council for Continuing Medical Education (ACCME) to provide continuing medical education for physicians.

The American Medical Informatics Association designates this online enduring material for 16.75 AMA PRA Category 1™ credits. Physicians should claim only the credit commensurate with the extent of their participation in the activity.

Claim credit no later than within two years of the release date or within one year of your purchase date, whichever is sooner.

ANNC Accreditation Statement

The American Medical Informatics Association is accredited as a provider of nursing continuing professional development by the American Nurses Credentialing Center's Commission on Accreditation.

  • Nurse Planner (Content): Robin Austin, PhD, DNP, DC, RN, NI-BC, FAMIA, FAAN
  • Approved Contact Hours: 16.75 participant maximum CME/CNE

ACHIPsTM

AMIA Health Informatics Certified ProfessionalsTM (ACHIPsTM) can earn 1 professional development unit (PDU) per contact hour.

ACHIPsTM may use CME/CNE certificates or the ACHIPsTM  Recertification Log to report 2025 CIC sessions attended for ACHIPsTM Recertification.

Claim credit no later than within two years of the release date or within one year of your purchase date, whichever is sooner. 

FAQs

Content was recorded live at AMIA's Informatics Summit March 10-13, 2025 in Pittsburgh, PA and at AMIA’s Annual Symposium event November 9-13, 2024, in San Francisco, CA.

Plan now to join us for the next Annual Symposium or Informatics Summit!

CME or CNE credit must be claimed no later than two from the release date or within one year of your purchase date, whichever is sooner. No credit will be issued that time.

Yes! AMIA On Demand is available for anyone to purchase. Become an AMIA member before you purchase to receive exclusive member discounts. Join AMIA today.

We’re glad you asked! AMIA offers a variety of membership options, all with exclusive benefits and abundant networking opportunities. Choose the membership that’s right for you.

The Audio-only format of all sessions is available free of charge exclusively to AMIA members. 

Access the audio recordings now (login required):

Join us at the next AMIA event and engage with leaders from across the health informatics field. 

Yes! You can claim Self-Study credit when you complete AMIA On Demand sessions, in addition to claiming Live credit for attending the live event. View the full details on self-study accreditation for this product.

Yes, The AMIA 2024 Annual Symposium On Demand Bundle (Presenter, Slides, and Audio) may be purchased for 8 educational credits using your health system’s code at checkout. Individual sessions (Presenter, Slides, and Audio) may be purchased for 1 educational credit per session using your health system’s code at checkout.

Available On:
Dates and Times:
Available Until:
Type: AMIA On Demand
Course Format(s): On Demand
Credits:
0.75
CME
,
0.75
CNE
Price: Member: $60, Nonmember: $85
Register now
Share