Nation’s health and bioinformatics experts establish policy agenda for ‘Adaptive CDS’
The American Medical Informatics Association (AMIA) released a Board-approved position paper in the Journal of the American Medical Informatics Association (JAMIA), calling for new and flexible oversight mechanisms to ensure the safe, effective use of artificial intelligence (AI) applications in healthcare. Specifically, AMIA focuses on AI-driven clinical decision support (CDS) systems, offering policy recommendations to the Food & Drug Administration (FDA) and articulating a new framework for the non-regulatory oversight of “Adaptive CDS.”
Stemming from a policy meeting held in December 2019, the AMIA position paper uses the term “Adaptive CDS” to describe CDS that can learn and change its performance over time, incorporating new clinical evidence; new data types and data sources; and new methods for interpreting data. Adaptive CDS enables personalized decision support in a way that has not been possible previously because it has the capacity to learn from data and modify recommendations based on those data. Adaptive CDS stands in contrast to “static” CDS, which are those tools that provide the same output (recommendation/guidance) each time the same input is provided without change through use.
“The informatics community invented CDS, and AMIA members have championed the advancement of CDS for decades,” said Patricia C. Dykes, PhD, RN, FAAN, FACMI, AMIA Board Chair and Program Director of Research at the Brigham and Women’s Center for Patient Safety, Research, and Practice. “An exponential growth in health data, combined with growing capacities to store and analyze such data through cloud computing and machine learning, obligates the informatics community to lead a discussion on ways to ensure safe, effective CDS in such a dynamic landscape.”
“The use of AI in healthcare presents clinicians and patients with opportunities to improve care in unparalleled ways,” said Carolyn Petersen, lead author and AMIA Public Policy Committee Member. “Equally unparalleled is the urgency to create safeguards and oversight mechanisms for the use of machine learning-driven applications for patient care.”
AMIA focuses on the use case of Adaptive CDS because it represents a wide range of potential tools and applications – some examples of which exist today, but many of which do not – and because Adaptive CDS represents a conceptual use case within a larger ecosystem of potential use cases of AI in healthcare. By framing discussion on Adaptive CDS, AMIA hopes to engender a practical discussion of policies needed to ensure safe and effective use of AI-driven CDS for patient care and facilitate a wider discussion of policies needed to build trust in the broader use of AI in healthcare.
The position paper identifies two categories of Adaptive CDS that warrant distinction for purposes of establishing public policy. First, Adaptive CDS that is sold to customers for use in a healthcare setting is referred to as Marketed ACDS. Second, Adaptive CDS that is developed in-house by healthcare systems and not marketed or sold to others is referred to as Self-Developed ACDS. Marketed ACDS is subject to FDA oversight per the 21st Century Cures Act and related FDA interpretation. Self-Developed ACDS is likely unregulated by any federal entity and is already used routinely without oversight by any authoritative body – public, private, or non-profit.
“The current policy and oversight landscape for Adaptive CDS is inadequate,” said Joseph Kannry, MD, AMIA Policy Committee Chair and paper author. “Gaps in federal jurisdiction of Adaptive CDS have left patients subject to algorithmic bias and potentially exposed to patient safety issues. In this paper we present a policy framework that spans the design and development, implementation, evaluation, and on-going maintenance of Adaptive CDS.”
First, transparency in how Adaptive CDS is trained is paramount. Without transparency, there can be no accountability. Specifically, the framework requires transparency standards for how algorithms are trained, including the semantics and provenance of training datasets are necessary for validation prior to deployment. Additionally, transparency into the data acquisition process, selection criteria of cohorts, and descriptions and prevalence of attributes likely to influence how a model may perform on new data are needed. “These choices mark the venues where bias can be introduced,” the paper notes.
Second, communications standards to convey specific attributes of how the model was trained, how it is designed, and how it should operate in situ are needed to objectively compare, evaluate, and guide ongoing maintenance of the algorithm. A range of questions regarding the intended use and expected users of Adaptive CDS must be addressed in a consistent manner, and product labeling regulations, such as those used by the FDA to explain prescription drugs, provide relevant correlates. For instance, FDA labeling requirements include concepts such as: indications and usage; contraindications; warnings & precautions; interactions; adverse reactions; and use in specific populations. Adaptive CDS may not be useful in specific clinical settings or for specific clinical purposes, so such standards for communicating how a clinician should apply Adaptive CDS are needed.
Logically stemming from these points is the practical need to establish agency and oversight – both regulatory and nonregulatory – to manage how these objectives are achieved through consistent systems and controls. With this need in mind, the AMIA paper calls for creation of new bodies, groups, or departments that govern implementation and use of AI within an institution, as well as a system of oversight across institutions. It also calls for Adaptive CDS Centers of Excellence to develop, test, evaluate, and advance the use of safe, effective ML-driven applications in practice.
“This AMIA Board-approved position paper establishes a policy agenda for the safe, effective use of Adaptive CDS in the U.S. healthcare system – and it positions AMIA as the organization to lead this agenda’s execution,” said Dykes.
###
AMIA, the leading professional association for informatics professionals, is the center of action for 5,500 informatics professionals from more than 65 countries. As the voice of the nation’s top biomedical and health informatics professionals, AMIA and its members play a leading role in assessing the effect of health innovations on health policy, and advancing the field of informatics. AMIA actively supports five domains in informatics: translational bioinformatics, clinical research informatics, clinical informatics, consumer health informatics, and public health informatics.