Informatics experts commend FDA for leading a much-needed conversation on the emerging and highly-important topic
BETHESDA, MD – In comments submitted this week to the U.S. Food and Drug Administration (FDA), the nation’s informatics professionals applauded the agency for developing a regulatory Modification Framework for Artificial Intelligence (AI)/Machine Learning (ML)-based software as a medical device (SaMD). The American Medical Informatics Association (AMIA) offered additional observations and recommendations for consideration in developing the framework further.
In early April, the FDA announced that it is developing a framework for regulating AI products that self-update based on new data. While FDA has authorized other AI-based devices, those products utilize “locked” algorithms that do not continually adapt or learn every time the algorithm is used. The agency was thus seeking comments on how to regulate self-updating algorithms, issuing a discussion paper and Request for Feedback that outlines its initial thoughts.
AMIA commended the FDA for publishing this draft framework and for leading the conversation on this emerging and important topic. Further, the group applauded the FDA for offering concepts such as SaMD Pre-Specifications (SPS), Algorithm Change Protocol (ACP) and Good Machine Learning Practices (GMLP) to drive the Modifications Framework. However, AMIA also offered FDA recommendations to improve the framework across four areas: continuously learning versus “locked” algorithms; new data inputs’ impact on algorithms’ outputs; cybersecurity in the context of AI/ML-based SaMD; and evolving knowledge about algorithm-driven bias
First, AMIA stressed the stark difference between continuously learning and “locked” algorithms, in that they do not necessarily provide the same opportunity for periodic, intentional updates. It thus recommended that the Modification Framework include requirements for periodic evaluation irrespective of planned updates or re-training. Further, AMIA recommended that FDA seeks additional feedback to understand a basis for determining when periodic evaluation should occur.
AMIA also expressed its concern that modern AI is both vulnerable to learning from poor or biased data and it may not be able to provide a cogent explanation for any decision it offers. This can thus result in a user of SaMD not knowing whether the device reasonably applies to his or her population. AMIA therefore encouraged FDA to consider a requirement for review when the SaMD’s learning comes from population(s) different from its training population.
Thirdly, AMIA encouraged FDA to consider how cybersecurity risks, such as hacking or data manipulation that may influence the algorithm’s output, may be addressed in a future version of the Framework. AMIA could envision, for example, a need for specific types of error detection geared towards preventing a system adaptation to an erroneous signal.
Finally, AMIA recommended that FDA tackle algorithm bias against persons of particular ethnicities, genders, ages, socioeconomic backgrounds, and physical and cognitive abilities by developing guidance about how and how often developers of SaMD-based products test their products and adjust their algorithms for such bias.
“Properly regulating AI and Machine Learning-based SaMD will require ongoing dialogue between FDA and stakeholders” said AMIA President and CEO Douglas B. Fridsma, MD, PhD, FACP, FACMI. “This draft Framework is only the beginning of a vital conversation to improve both patient safety and innovation. We certainly look forward to continuing it.”
Click here for AMIA’s full comments.
###
AMIA, the leading professional association for informatics professionals, is the center of action for 5,600 informatics professionals from more than 65 countries. As the voice of the nation’s top biomedical and health informatics professionals, AMIA and its members play a leading role in assessing the effect of health innovations on health policy, and advancing the field of informatics. AMIA actively supports five domains in informatics: translational bioinformatics, clinical research informatics, clinical informatics, consumer health informatics, and public health informatics.