Getting Smarter: FDA Publishes Draft Guidance on Predetermined Change Control Plans for Artificial Intelligence/Machine Learning (AI/ML) Devices

On April 3, 2023, the U.S. Food and Drug Administration (“FDA”) released a draft guidance document titled “Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence/Machine Learning (AI/ML)-Enabled Device Software Functions.”¹ This draft guidance follows a number of steps FDA has taken in recent years to address AI/ML-based software, including, among other things, the Agency’s 2019 discussion paper and request for feedback on a Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD)” (“Proposed Framework”) and its 2021 “Artificial Intelligence/Machine Learning (“AI/ML”)-Based Software as a Medical Device (“SaMD”) Action Plan” (“Action Plan”), discussed in a previous Ropes & Gray Alert. In the 2019 Proposed Framework, FDA introduced the concept of a predetermined change control plan (“PCCP”) that would enable FDA to authorize anticipated AI/ML-based modifications to device software products during the premarket review process. The 2021 Action Plan outlined FDA’s plans for further developing its regulatory framework for AI/ML-based SaMD, including issuing a draft guidance on PCCPs. In the Food and Drug Omnibus Reform Act of 2022, Congress added section 515C to the Federal Food, Drug, and Cosmetic Act, providing FDA with express authority for PCCPs.

FDA recognizes that the advantage of ML-based device software functions (“ML-DSFs”) is the ability to improve their performance through iterative modifications, which is in tension with FDA’s traditional framework for premarket review of changes to a medical device. With PCCPs, FDA seeks to allow manufacturers to obtain premarket authorization for pre-specified automatic and manual modifications that may be made to an ML-DSF, without re-submitting the device for FDA review. This would allow manufacturers to make improvements to ML-DSFs available to health care providers and patients more quickly than they are able to under the traditional marketing authorization process.² This includes ML-DSFs for which modifications to the ML model are implemented automatically (i.e., for which the modifications are implemented automatically by software), as well as modifications to the ML model that are implemented manually (i.e., involving steps that require human input, action, review, or decision-making prior to implementation). The draft guidance explains how FDA intends to use the PCCP to facilitate software development by reducing the scenarios in which additional marketing submissions are required and sets out FDA’s recommendations for the content to be included in a PCCP.

Scope and Components of a PCCP

The draft guidance defines a PCCP as “[t]he documentation describing what modifications will be made to the ML-DSF and how the modifications will be assessed” and clarifies that “modifications described in the PCCP include device changes that would otherwise require a PMA supplement, De Novo submission, or new 510(k) notification.” It further explains that a PCCP is meant to cover modifications intended to maintain or improve the safety or effectiveness of an ML-DSF and may include modifications related to quantitative measures of ML-DSF performance specifications, modifications related to device inputs to the ML-DSF, and limited modifications related to a device’s use and performance. Modifications that would change a device’s intended use or indications for use are not appropriate for inclusion in a PCCP. Additionally, a PCCP should not include minor modifications that would not require a new marketing submission.

FDA intends to review and authorize a PCCP as part of a device’s marketing submission. The PCCP should be discussed in the marketing submission as part of the device description, labeling, and relevant sections used for determining substantial equivalence or reasonable assurance of safety and effectiveness of the device. Labeling will generally need to explain that the device incorporates machine learning and has a PCCP, so that users are aware that the device may require the user to perform software updates that may modify the device’s performance, inputs, or use. Such information may be necessary for a user to understand changes in the device and to continue to use the device safely and effectively.

The draft guidance sets out three primary components of a PCCP: a Description of Modifications, a Modification Protocol, and an Impact Assessment.

The Description of Modifications details each specific planned modification to be made to an ML-DSF, and once authorized, defines the “range of FDA-authorized specifications.” In the Description of Modifications, manufacturers should list and provide the rationale for each proposed device modification.

A Modification Protocol documents the methods to be used for developing, validating, and implementing the modifications defined in the Description of Modifications. The draft guidance describes four primary components of a Modification Protocol:

• Data management practices consist of the methods for collecting, annotating, curating, storing, retaining, controlling, and using input and reference data for each modification.
• Re-training practices are the processing steps subject to change for a given modification and the methods for implementing each modification.
• Performance evaluation includes the verification and validation plans to be used to ensure a modified ML-DSF meets the specifications for all modifications implemented.
• Update procedures explain how devices will be updated to implement modifications, including confirmation that the same verification and validation plans are used for the device both prior to and after modification.

FDA recommends that manufacturers indicate in the PCCP which parts of the Modification Protocol are applicable to each of the modifications identified in the Description of Modifications. If the PCCP covers multiple modifications, the Agency suggests that the manufacturer provide a traceability table identifying which components of the Modification Protocol apply to each of the modifications in the Description of Modifications. In the Draft Guidance, FDA also provides a detailed appendix with suggested questions to consider for the various elements of the Modification Protocol.

The Impact Assessment provides the manufacturer’s assessment under its quality system of the benefits and risks of implementing a PCCP and how risks can be mitigated. Among other things, the Impact Assessment should explain the impact of proposed modifications on the ML-DSF as well as overall device functionality. Manufacturers should demonstrate that the aggregate proposed modifications are unlikely to present additional risks that cannot be adequately mitigated.

Making Changes with an Authorized PCCP

FDA considers an approved PCCP to be part of a device’s marketing authorization, which can be in the form of a 510(k), De Novo authorization, or PMA. As explained above, the purpose of a PCCP is to eliminate the need for additional marketing submissions for pre-specified modifications. For a manufacturer to avoid the requirement of an additional marketing submission for a modification, the modification must be consistent with the PCCP, meaning that it must be specified in the Description of Modifications and implemented according to the Modification Protocol. In such cases, the manufacturer’s only obligation would be to document the modification in accordance with its quality system. A modification may require a new marketing submission if it is not specified in the Description of Modifications or, even if specified, if it is not implemented in accordance with the Modification Protocol.

The draft guidance also briefly addresses scenarios in which manufacturers may desire to make modifications to an authorized PCCP itself. FDA’s current position is that because a modification to a PCCP would likely have a significant impact on the safety and effectiveness of a device, a modification to an authorized PCCP will generally require a new marketing submission for the device.

FDA Has Already Experimented with PCCPs

Some marketing authorizations in recent years have included PCCPs or plans similar to PCCPs. For example:

• FDA granted De Novo authorization in February 2020 to a software-only device that uses AI to emulate the expertise of a sonographer and is intended to provide real-time guidance to the user during acquisition of certain radiological images to assist in obtaining anatomically correct images.³ The De Novo request incorporated a PCCP under which future algorithm improvements could be made. The PCCP included a protocol for mitigating algorithm changes that may change the device’s technical specifications or negatively impact clinical functionality or performance specifications.
• FDA granted De Novo authorization in April 2018 to a device that includes software with an adaptive algorithm that evaluates ophthalmic images to detect diabetic retinopathy.⁴ The De Novo request included a protocol that would be used to mitigate the risk that future algorithm improvements may change the device’s technical specifications and result in incorrect results.

Takeaways

The new draft guidance is one of several steps FDA is taking to enhance its regulatory framework for AI/ML technologies. For example, since publishing the Action Plan, in October 2021, FDA released guiding principles for Good Machine Learning Practice for Medical Device Development in collaboration with Health Canada and the Medicines and Healthcare Products Regulatory Agency. FDA also held a public workshop in October 2021 regarding transparency of AI/ML-enabled medical devices, which included discussion on how manufacturers could best share information about AI/ML-enabled devices. Looking ahead, the Center for Devices and Radiological Health has prioritized issuing final guidances that will be relevant to ML-DSFs by the end of 2023, including Cybersecurity in Medical Devices: Quality System Considerations and Content of Premarket Submissions and Content of Premarket Submissions for Device Software Functions.⁵

The PCCP framework, if finalized, would provide medical device manufacturers with a mechanism for obtaining authorization for planned modifications to ML-DSFs without additional marketing submissions. PCCPs may enable device manufacturers to bring new ML-based technologies to the market more quickly by avoiding later marketing submissions. Although initial industry feedback on the PCCP concept was generally positive following FDA’s publication of the Proposed Framework and Action Plan, developers of ML-DSFs should review the draft guidance and consider whether it achieves the Agency’s goals in a manner consistent with least burdensome principles. The deadline for comments on the draft guidance is July 3, 2023.