2025 AMIA AI Evaluation Showcase
Regulators, researchers, and clinical practitioners have all been exploring how to best evaluate the different phases of the health AI deployment lifecycle.
The American Medical Informatics Association (AMIA) as the world's largest professional society for medical informaticists has been taking an active role in helping the community identify oversight mechanisms to ensure the safe, effective use of artificial intelligence (AI) applications in healthcare.
The AMIA AI Evaluation Showcase Series aims to highlight current use cases in the field and form a community to help build consensus around best practice. The community and the curated best practice examples are expected to strengthen the thought leadership position of the AMIA community in the industrial standard understanding and setting.
Stages
Stage I - Technical Performance Presentations
Presentations at the AMIA Informatics Summit highlighting a system description, results from a study of algorithm performance, and an outline of the methods of the full evaluation plan.
Stage II - Usability and Workflow Presentations
Presentations at the AMIA Clinical Informatics Conference highlighting usability and workflow aspects of the AI system, including prototype usability or satisfaction, algorithm explainability, implementation lessons, and/or system use in context.
Stage III - Health Impact Presentations
Presentations at the AMIA Annual Symposium summarizing the the comprehensive evaluation of the Health AI implementation such as impacts of the AI implementation based on the measures of their choice, e.g., clinical outcomes, patient-reported outcome measures (PROMs), patient-reported experience measures (PREMs), clinician experience and adoption measures, cost of care, healthcare quality, or other measures from health economic studies.
How to Participate
Contact the AMIA education team, and they will direct your inquiry to the Scientific Program Committee (SPC) for the appropriate conference. AMIA AI Evaluation Showcase presentations are invitation-based and selected by the SPC.