Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) - Discussion Paper and Request for FeedbackThis discussion paper proposes a framework for modifications to AI/ML-based SaMD that is based on the internationally harmonized International Medical Device Regulators Forum (IMDRF) risk categorization principles, FDA’s benefit-risk framework, risk management principles in the software modifications guidance , and the organization-based TPLC approach as envisioned in the Digital Health Software Precertification (Pre-Cert) Program. It also leverages practices from our current premarket programs, including the 510(k), De Novo, and PMA pathways. ... applying a [Total Product Lifecycle ] approach to the regulation of software products is particularly important for AI/ML-based SaMD due to its ability to adapt and improve from real-world use. In the Pre-Cert TPLC approach, FDA will assess the culture of quality and organizational excellence of a particular company and have reasonable assurance of the high quality of their software development, testing, and performance monitoring of their products. This approach would provide reasonable assurance of safety and effectiveness throughout the lifecycle of the organization and products so that patients, caregivers, healthcare professionals, and other users have assurance of the safety and quality of those products. This TPLC approach enables the evaluation and monitoring of a software product from its premarket development to postmarket performance, along with continued demonstration of the organization’s excellence ... Food and Drug AdministrationFDA_85a3d7be-fbea-11df-a90e-29337a64ea2a_dbee3b14-5a19-11e9-9d42-f7a07b73fa26To propose a framework for modifications to AI/ML-based software as a medical device_dbee3cf4-5a19-11e9-9d42-f7a07b73fa26ExpectationsEstablish clear expectations on quality systems and good ML practices (GMLP)_dbee3fe2-5a19-11e9-9d42-f7a07b73fa261Medical Device ManufacturersThe FDA expects every medical device manufacturer to have an established quality system that is geared towards developing, delivering, and maintaining high-quality products throughout the lifecycle that conforms to the appropriate standards and regulations. Similarly, for AI/ML-based SaMD, we expect that SaMD developers embrace the excellence principles of culture of quality and organizational excellence. As is the case for all SaMD, devices that rely on AI/ML are expected to demonstrate analytical and clinical validation, as described in the SaMD: Clinical Evaluation guidance (Figure 3). The specific types of data necessary to assure safety and effectiveness during the premarket review, including study design, will depend on the function of the AI/ML, the risk it poses to users, and its intended use. AI/ML algorithm development involves learning from data and hence prompts unique considerations that embody GMLP. In this paper, GMLP are those AI/ML best practices (e.g., data management, feature extraction, training, and evaluation) that are akin to good software engineering practices or quality system practices. Examples of GMLP considerations as applied for SaMD include:RelevanceConsider the relevance of available data to the clinical problem and current clinical practice_dbee40d2-5a19-11e9-9d42-f7a07b73fa261.1Consistency & GeneralizabilityEnsure that data is acquired in a consistent, clinically relevant and generalizable manner that aligns with the SaMD’s intended use and modification plans_dbee41b8-5a19-11e9-9d42-f7a07b73fa261.2SeparationAppropriately separate training, tuning, and test datasets_dbee429e-5a19-11e9-9d42-f7a07b73fa261.3ClarityEnsure an appropriate level of transparency (clarity) of the output and the algorithm aimed at users._dbee437a-5a19-11e9-9d42-f7a07b73fa261.4ReviewConduct premarket review for those SaMD that require premarket submission to demonstrate reasonable assurance of safety and effectiveness and establish clear expectations for manufacturers of AI/ML-based SaMD to continually manage patient risks throughout the lifecycle_dbee446a-5a19-11e9-9d42-f7a07b73fa262This framework gives manufacturers the option to submit a plan for modifications during the initial premarket review of an AI/ML-based SaMD. FDA’s premarket review and determination regarding the acceptability of such plans would provide reasonable assurance of safety and effectiveness and would include review of the SaMD’s performance, the manufacturer’s plan for modifications, and the ability of the manufacturer to manage and control resultant risks of the modifications. FDA has successfully explored this voluntary approach to review device modification plans in certain recent De Novo classifications regarding several in-vitro diagnostic next generation sequencing products. This paper proposes a framework for modifications to AI/ML-based SaMD that relies on the principle of a “predetermined change control plan.” Using this proposed regulatory approach, we believe that our oversight will enable responsible performance enhancements in AI/ML-based technologies. The predetermined change control plan would include the types of anticipated modifications – SaMD Pre-Specifications – based on the retraining and model update strategy, and the associated methodology – Algorithm Change Protocol – being used to implement those changes in a controlled manner that manages risks to patients. Pre-SpecificationsDocument anticipated modifications to “performance” or “inputs,” or changes related to the “intended use” of AI/ML-based SaMD._dbee4654-5a19-11e9-9d42-f7a07b73fa262.1SaMD Pre-Specifications (SPS): A SaMD manufacturer’s anticipated modifications to “performance” or “inputs,” or changes related to the “intended use” of AI/ML-based SaMD. These are the types of changes the manufacturer plans to achieve when the SaMD is in use. The SPS draws a “region of potential changes” around the initial specifications and labeling of the original device. This is "what" the manufacturer intends the algorithm to become as it learns.Algorithm Change ProtocolDelineate the data and procedures to be followed so that the modification achieves its goals and the device remains safe and effective_dbee4730-5a19-11e9-9d42-f7a07b73fa262.2Algorithm Change Protocol (ACP): Specific methods that a manufacturer has in place to achieve and appropriately control the risks of the anticipated types of modifications delineated in the SPS. The ACP is a step-by-step delineation of the data and procedures to be followed so that the modification achieves its goals and the device remains safe and effective after the modification. Figure 4 below provides a general overview of the components of an ACP. This is "how" the algorithm will learn and change while remaining safe and effective.Data Management_dbee4870-5a19-11e9-9d42-f7a07b73fa262.2.1Re-Training_dbee4960-5a19-11e9-9d42-f7a07b73fa262.2.2Performance Evaluation_dbee4a5a-5a19-11e9-9d42-f7a07b73fa262.2.3Update Procedures_dbee4b68-5a19-11e9-9d42-f7a07b73fa262.2.4Individual ConsiderationsTake into account individual consideration during premarket review of benefits and risks to patients_dbee4c62-5a19-11e9-9d42-f7a07b73fa262.3Scope and limitations for establishing SPS and ACP: The FDA acknowledges that the types of changes that could be pre-specified in a SPS and managed through an ACP may necessitate individual consideration during premarket review of benefits and risks to patients of that particular SaMD. The extent to which pre-approval of a SPS and an ACP can be relied on to support future modifications depends on various factors. The following are example scenarios that illustrate the general concept of establishing an appropriate SPS and its corresponding ACP ... There are many scenarios for which an appropriate SPS and ACP could be crafted, however, we also anticipate that in certain cases, the SaMD’s risk or the intended use may significantly change after learning. In these cases, it may not be appropriate for a proposed SPS and ACP to manage the risks to patients or align with the initial authorized intended use. For example, it would not be appropriate for a SPS and ACP initially indicated for a “low risk” (non-serious) healthcare situation or condition, such as using skin images to manage the healing of scars, to be leveraged for the same SaMD in diagnosing melanoma, which would be considered a “critical healthcare situation or condition.”PerformanceConsider improvements in performance, or changes in input, without affecting the intended use of the SaMD_dbee4d66-5a19-11e9-9d42-f7a07b73fa262.3.1Changes that involve improvements in performance, or changes in input, without affecting the intended use of the SaMD, may be accomplished with an appropriate level of pre-specification and an appropriate ACP that provides reasonable assurance that performance will be improved or maintained. The ACP may include the basis of validation and methods to adequately monitor and control for significant degradation in performance or introduce risks to patients. InformationConsider increases in the significance of the information provided to the user for the same healthcare situation or condition._dbee4e7e-5a19-11e9-9d42-f7a07b73fa262.3.2Certain changes related to the intended use, in particular, an increase in the significance of the information provided to the user for the same healthcare situation or condition. Using the IMDRF risk framework as the basis for an example, a SPS may include a modification related to the intended use within “drive clinical management,” which may shift the intended use from “identify early signs of a disease or conditions” to “aid in making a definitive diagnosis” for the same healthcare situation or condition. An appropriate ACP might be developed, reviewed, and agreed by FDA and the manufacturer to adequately improve the performance to a level that increases the confidence in its ability to be used as an aid in making a definitive diagnosis. UsageConsider the “indications for use.”_dbee4f78-5a19-11e9-9d42-f7a07b73fa262.3.4Certain changes related to the intended use, in particular, the “indications for use.” For example, a manufacturer may intend to expand the use of their SaMD to a new patient population for which there had been insufficient evidence available to initially support that indication for use. In some cases, an appropriate reference standard may initially not be available for the new patient population; a manufacturer’s ACP may include a characterization plan for the reference standard in the disease population to assure it provides a meaningful representation of the disease. In other cases, an input data type used by the AI/ML-based SaMD may not normally be available for the patient population; a developer’s ACP may include a demonstration of the clinical association between the disease and input data type in the new patient population, as well as a plan for data collection and algorithm testing in the patient population. Risk ManagementExpect manufacturers to monitor the AI/ML device and incorporate a risk management approach and other approaches outlined in “Deciding When to Submit a 510(k) for a Software Change to an Existing Device” Guidance in development, validation, and execution of the algorithm changes (SaMD Pre-Specifications and Algorithm Change Protocol)_dbee5068-5a19-11e9-9d42-f7a07b73fa263Learning, adaptation, and optimization are inherent to AI/ML-based SaMD. These capabilities of AI/ML would be considered modifications to SaMD after they have received market authorization from FDA. This paper proposes an approach to appropriately manage risks to patients from these modifications, while enabling manufacturers to improve performance and potentially advance patient care. As outlined in Figure 5, manufacturers are expected to evaluate the modifications based on risk to patients as outlined in the software modifications guidance. The software modifications guidance uses a risk-based approach and expects a manufacturer to perform a risk assessment and evaluate that the risks are reasonably mitigated. Depending on the type of modification, the current software modifications guidance results in either 1) submission of a new 510(k) for premarket review or 2) documentation of the modification and the analysis in the risk management and 510(k) files. If, for AI/ML SaMD with an approved SPS and ACP, modifications are within the bounds of the SPS and the ACP, this proposed framework suggests that manufacturers would document the change in their change history and other appropriate records, and file for reference, similar to the “document” approach outlined in the software modifications guidance. In the software modifications guidance, depending on the type of change, if the modification is beyond the intended use for which the SaMD was previously authorized, manufacturers are expected to submit a new premarket submission. For this proposed approach, we anticipate that there may be cases where the SPS or ACP can be refined based on the real-world learning and training for the same intended use of AI/ML SaMD model. In those scenarios, FDA may conduct a “focused review” of the proposed SPS and ACP for a particular SaMD. Manufacturers may leverage some of the following options to engage with FDA on the SPS and ACP for a particular SaMD: ConformanceContact the appropriate review division to obtain concurrence that the modification fits under current SPS and ACP; or_dbee5162-5a19-11e9-9d42-f7a07b73fa263.1ScopeSubmit a pre-submission for a discussion on the modification and how it is within the bounds of the current SPS and ACP; or_dbee527a-5a19-11e9-9d42-f7a07b73fa263.2ModificationSubmit a premarket submission or application of the modification to SPS and ACP. _dbee5388-5a19-11e9-9d42-f7a07b73fa263.3TransparencyEnable increased transparency to users and FDA using postmarket real-world performance reporting for maintaining continued assurance of safety and effectiveness_dbee54be-5a19-11e9-9d42-f7a07b73fa264To fully adopt a TPLC approach in the regulation of AI/ML-based SaMD, manufacturers can work to assure the safety and effectiveness of their software products by implementing appropriate mechanisms that support transparency and real-world performance monitoring. Transparency about the function and modifications of medical devices is a key aspect of their safety. This is especially important for devices, like SaMD that incorporate AI/ML, which change over time. Further, many of the modifications to AI/ML-based SaMD may be supported by collection and monitoring of real-world data. Gathering performance data on the real-world use of the SaMD may allow manufacturers to understand how their products are being used, identify opportunities for improvements, and respond proactively to safety or usability concerns. Real-world data collection and monitoring is an important mechanism that manufacturers can leverage to mitigate the risk involved with AI/ML-based SaMD modifications, in support of the benefit-risk profile in the assessment of a particular AI/ML-based SaMD.Performance MonitoringCommit to the principles of transparency and real-world performance monitoring for AI/ML-based SaMD. _dbee55d6-5a19-11e9-9d42-f7a07b73fa264.1Through this framework, manufacturers would be expected to commit to the principles of transparency and real-world performance monitoring for AI/ML-based SaMD. FDA would also expect the manufacturer to provide periodic reporting to FDA on updates that were implemented as part of the approved SPS and ACP, as well as performance metrics for those SaMD. This commitment could be achieved through a variety of mechanisms.UpdatesProvide appropriate updates._dbee5702-5a19-11e9-9d42-f7a07b73fa264.2Transparency may include updates to FDA, device companies and collaborators of the manufacturer, and the public, such as clinicians, patients, and general users. For modifications in the SPS and ACP, manufacturers would ensure that labeling changes accurately and completely describe the modification, including its rationale, any change in inputs, and the updated performance of the SaMD. Manufacturers may also need to update the specifications or compatibility of any impacted supporting devices, accessories, or non-device components. Finally, manufacturers may consider unique mechanisms for how to be transparent – they may wish to establish communication procedures that could describe how users will be notified of updates (e.g., letters, email, software notifications) and what information could be provided (e.g., how to appropriately describe performance changes between the current and previous version).ReportingTailor reporting types and frequencies appropriately._dbee5842-5a19-11e9-9d42-f7a07b73fa264.3Real-world performance monitoring may also be achieved in a variety of suggested mechanisms that are currently employed or under pilot at FDA, such as adding to file or an annual report, Case for Quality activities, or real-world performance analytics via the Pre-Cert Program. Reporting type and frequency may be tailored based on the risk of the device, number and types of modifications, and maturity of the algorithm (i.e., quarterly reports are unlikely to be useful if the algorithm is at a mature stage with minimal changes in performance over the quarter). Involvement in pilot programs, such as Case for Quality and the Pre-Cert Program, may also impact the reporting type and frequency given the insight into the manufacturer’s TPLC and organization. Participation in these programs could provide another avenue to support continued assurance of safety and effectiveness in development and modifications of AI/ML-based SaMD.2019-04-022019-04-08https://www.regulations.gov/document?D=FDA-2019-N-1185-0001OwenAmburOwen.Ambur@verizon.net