In Vitro to in Vivo Translation of Artificial Intelligence for Clinical Use: Screening for Acute Coronary Syndrome to Identify ST-Elevation Myocardial Infarction
Read the abstract
Presenter
Gabrielle Bunney, MD, MBA, MS
Moderator
Statement of Purpose
Despite rapid growth in the development of clinical prediction models, methodological guidance for their technical translation from desktop, in vitro, development to clinical, in vivo, operation within electronic health record (EHR) systems remains limited. Current evaluation practices often emphasize retrospective model performance or post-deployment monitoring, leaving a critical gap in prospective, real-time assessment of the model’s performance in the clinical environment prior to affecting clinical care. While prior work has acknowledged the value of silent or pre-interventional testing, there is a lack of concrete, reproducible frameworks that operationalize this step and systematically distinguish between data-related, computational, and implementation-induced sources of error.
The purpose of this work is to introduce and demonstrate the Clinical Technical Implementation of a Prediction Model (CTIPM) framework, a structured, phased methodology for silently testing prediction models in live clinical environments prior to clinical use. The CTIPM framework formalizes an iterative “informatics run-in period” consisting of a Technical Component Analysis, Technical Fidelity Analysis, and Comparative Effectiveness Analysis, each designed to answer specific implementation-focused questions about model execution and performance relative to the original desktop model. Applied to an emergency department acute coronary syndrome screening model, this work shows how CTIPM enables prospective assessment of in vitro–to–in vivo agreement, isolates discrepancies arising from dynamic EHR data versus computational translation, and supports targeted remediation before effectiveness testing. By shifting methodological emphasis toward implementation fidelity and technical validity, the CTIPM framework advances the field of medical informatics by providing a practical, generalizable approach for safely operationalizing prediction models within clinical systems.
Learning Objectives
- Recall why silent, prospective technical validation is a critical step prior to implementing prediction models into clinical care.
- Recognize common sources of failure in the in vitro–to–in vivo translation of clinical prediction models.
- Explain how structured, iterative evaluation frameworks can be used to detect and address these issues before clinical deployment.
Additional Information
The target audience for this activity includes physicians, nurses, other healthcare providers, and medical informaticians.
No commercial support (funding from a governmental agency, ineligible company or in-kind donation) was received for this activity.
Completion of this “Enduring Material” is demonstrated by participating in the live webinar or viewing the on-demand recording, engaging with presenters during the live session by submitting questions, and completing the evaluation survey at the conclusion of the course.
Learners may claim credit and download a certificate upon submission of the evaluation. Participation in additional resources and the course forum is encouraged but optional.
ACCME Accreditation Statement
The American Medical Informatics Association is accredited by the Accreditation Council for Continuing Medical Education (ACCME) to provide continuing medical education for physicians.
Designation Statement
The American Medical Informatics Association designates this Enduring activity for a maximum of 1 AMA PRA Category 1 Credit(s)™. Physicians should claim only the credit commensurate with the extent of their participation in the activity.
ANCC Accreditation Statement
The American Medical Informatics Association is accredited as a provider of nursing continuing professional development by the American Nurses Credentialing Center's Commission on Accreditation.
Nurse Planner (Content): Robin Austin, PhD, DNP, DC, RN, NI-BC, FAMIA, FAAN
Approved Contact Hours: 1 CME/CNE
*Learners may earn 1 contact hour for each monthly Journal Club session, for a maximum of 6 contact hours per year. To receive the full 6 contact hours, participants must either attend the live webinar or view the on-demand recording for each Regularly Scheduled Series (RSS) Journal Club presentation.
It is the policy of the American Medical Informatics Association (AMIA) to ensure that Continuing Medical Education (CME) activities are independent and free of commercial bias. To ensure educational content is objective, balanced, and guarantee content presented is in the best interest of its learners and the public, the AMIA requires that everyone in a position to control educational content disclose all financial relationships with ineligible companies within the prior 24 months. An ineligible company is one whose primary business is producing, marketing, selling, re-selling or distributing healthcare products used by or on patients. Examples can be found at accme.org.
In accordance with the ACCME Standards for Integrity and Independence in Accredited Continuing Education, AMIA has implemented mechanisms prior to the planning and implementation of this CME activity to identify and mitigate all relevant financial relationships for all individuals in a position to control the content of this CME activity.
In accordance with the ACCME Standards for Integrity and Independence in Accredited Continuing Education, AMIA has implemented mechanisms prior to planning and implementation of this CME activity to identify and mitigate all relevant financial relationships for all individuals in a position to control the content of this CME activity.
Faculty and planners who refuse to disclose any financial relationships with ineligible companies will be disqualified from participating in the educational activity.
For an individual with no relevant financial relationship(s), course participants must be informed that no conflicts of interest or financial relationship(s) exist.
Disclosures
Disclosures of relevant financial relationships of all planners and presenters of the Journal Club.
Planning Committee
The planning committee and reviewers reported that they have no relevant financial relationship(s) with ineligible companies to disclose.
- Joanna Abraham, PhD, FACMI, FAMIA
- Ratie Akabari, MS
- Zo Co
- Ivan Gu
- Andrew Lu, MSc, RN
- Elanore "Nora" Rae Scheer, ME
The following planning committee members have relevant financial relationship(s) with ineligible companies to disclose.
- Amy Krefman, MS - AbbVie; Individual Stocks/Stock Options
Presenter
The following presenters have no relevant financial relationship(s) with ineligible companies to disclose.
- Gabrielle Bunney, MD, MBA, MS
AMIA Staff
The AMIA staff have no relevant financial relationship(s) with ineligible companies to disclose.
*All of the relevant financial relationships listed for these individuals have been mitigated.