Re: Docket No. FDA-2017-N-4301: Fostering Medical Innovation: A Plan for Digital Health Devices; Software Precertification Pilot Program

Transcription

1 701 Pennsylvania Avenue, NW Suite 800 Washington, D.C Tel: Fax: September 20, 2018 Division of Dockets Management (HFA-305) Food and Drug Administration 5630 Fishers Lane, Room 1061 Rockville, MD Re: Docket. FDA-2017-N-4301: Fostering Medical Innovation: A Plan for Digital Health Devices; Software Precertification Pilot Program To Whom It May Concern: The Advanced Medical Technology Association ( AdvaMed ) appreciates the opportunity to provide further input on the Food and Drug Administration s ( FDA or Agency ) Digital Health Software Precertification Program Working Model v AdvaMed represents manufacturers of digital health technologies, medical devices, and diagnostic products that are transforming health care through earlier disease detection, less invasive procedures, and more effective treatment. Our members range from the smallest to the largest medical technology innovators and companies. We commend FDA for its ongoing efforts to create the Precertification Program. If properly developed and implemented, the program has the potential to reduce pre- and post-market burdens for both software developers and FDA and enable streamlined changes and modifications to software. Moreover, patients and consumers will benefit under the Precert program, as these important technologies will be available in a more timely manner. AdvaMed is pleased to provide the following comments on Components 2 (Review Determination) and 3 (Streamlined Review) of the Working Model. We note that these comments represent our current thinking on this topic. As the Working Model and the broader precertification program are further developed, explained, and analyzed, our views may change and/or evolve. Comments on Working Model Component 2, Review Determination: The goal of review determination should be for a developer to understand the following: 1. Whether its product is regulated as Software as a Medical Device (SaMD), or not regulated; 2. The risk level of the product, if it is regulated by FDA; and 3. Whether the product requires a streamlined review based on its risk classification and the precertification level of the developer. Review Determination should be simple, efficient, and follow a least burdensome approach. 1 Developing Software Precertification Program: A Working Model, v0.2 (June 2018), available at Bringing innovation to patient care worldwide

2 Docket. FDA-2017-N-4301 September 20, 2018 Page 2 of 5 We support FDA s goal to have software developers share specific and helpful information with the public about their SaMD. We have developed the attached template that can be used, potentially in a specific FDA software database, for this purpose. We believe basic information, which includes information in a SaMD Definition Statement, as described in the International Medical Device Regulators Forum s (IMDRF) "Software as a Medical Device": Possible Framework for Risk Categorization and Corresponding Considerations, are important for healthcare providers and patients to understand the intended use and function of software they are using. This information could be used for registration/listing, as well as for Streamlined Review if required, to avoid duplication of entry. See Appendix A for more information. While some information included as part of the Review Determination Template may be helpful for FDA, it may not be as relevant or helpful to the public. One example of this type of information is the precertification level of an organization. It is important to note that precertification is a voluntary process, and there may be business reasons why a developer would choose not to become precertified. Publication of precertification status could lead to unnecessary confusion and inaccurate assumptions either positive or negative about the safety and effectiveness of software from the developer. In addition, large companies may have certain business units that are precertified and others that are not, which could similarly lead to confusion. If FDA decides to make the precertification status of a developer public, we encourage FDA to also appropriately explain such concepts as the meaning and relevance of precertification, including that the standard for review for SaMD has not changed but is collected at different times, the voluntary nature of the program, and other important key messages to maintain public confidence in software technologies. We appreciate FDA s recognition that a decision tree or other guide is necessary to assist developers in determining the risk categorization of their software, specifically whether or not the software requires review. Such a resource will allow developers to self-identify the risk of their software and provide transparency and consistency for both developers and reviewers. Due to ongoing confusion regarding implementation of 21 st Century Cures as well as some software currently remaining under enforcement discretion, we encourage any such decision tree or guide to also address whether FDA will regulate the device. We have provided two such draft decision trees, which can serve as a starting point for FDA to consider in the development of such a resource. (See Appendices B and C for more information). Appendix B focuses on identifying the intended use of the software to determine whether or not it is regulated by FDA, incorporating relevant provisions of 21 st Century Cures and applicable FDA guidances. Appendix C focuses on risk categorization of software, specifically SaMD, which is regulated by FDA. It is important to note that both decision trees should be considered works in progress and, again, intended only as a starting point for FDA in the development of a useful guide for software developers. For example, we continue to have ongoing discussions about Appendix C and struggle with whether such risk categorization can actually be addressed through a decision tree as opposed to some other tool. Some of the questions are intended to be branching questions and determinative, such as whether the software is the only, definitive determinant for clinical action versus one of several pieces of information. While other questions, such as if the software is used for immediate or near-term clinical action, could apply to more than one risk category and may be a consideration for appropriate risk

3 Docket. FDA-2017-N-4301 September 20, 2018 Page 3 of 5 categorization rather than an essential principle that moves SaMD into one risk category or another. It is critical that any such decision tree or tool be thoroughly tested to ensure consistent application and results by various users. Alignment of this tool with existing resources/guidance for industry such as the FTC Mobile Health Apps Interactive Tool could also help to enhance understanding of applicable requirements by developers/manufacturers. We are happy to work with FDA to provide SaMD examples and further refine the types of decision points and questions necessary to achieve consistency in risk categorization. In addition, we recommend FDA provide examples in both the tool as well as its interpretations to describe the types of SaMD within each category. We agree with FDA s use of the IMDRF Risk-Categorization Framework for classification of SaMD, which reflects input from a variety of stakeholders and supports efforts toward global convergence. However, we strongly believe that the IMDRF language must be adapted to fit the U.S. regulatory paradigm. To that end, we have recommended language intended to clarify and appropriately categorize SaMD based on the significance of the information to the health decision and the state of the healthcare disease or condition, as applied within the U.S. framework. We developed this language based upon input from software developers, traditional device manufacturers, and health professionals, as well as pressure testing the decision trees with a variety of software technology examples. (See Appendix D for more information). We appreciate FDA s proposed approaches to streamlined review described in Table 4 of the Working Model. We recommend FDA change its requirement for streamlined review for Level 1 Precertified Organizations for Critical x Inform. The line should move up in the table (purple box for initial product should only include serious x drive and non-serious x diagnose/treat). We believe this is a more appropriate risk-based approach for SaMD as it is one of several pieces of inputs for a user and not necessary for determining clinical action. For Software products that are comprised of several different software applications on a common software platform (e.g. multiple function software), some software applications on the platform may not be subject to FDA oversight based on the review determination. In such cases, we encourage FDA to consider only the software applications under FDA review as those subject to the precertification program, rather than considering the entire software platform. Such a policy would be consistent with FDA s Draft Guidance on Multiple Function Device Products, which when applied to the precertification program would suggest that only the impact of the non-medical device applications on the medical software application under review should be considered. We have several additional general comments, which include: o We welcome additional guidance on how the Precertification program IMDRF Risk- Categorization Framework and Level of Review map to the existing medical device regulatory classification and product code system. This transparency will also be helpful for non-pilot participants seeking future premarket review and will foster enhanced understanding of the benefits of the Precertification program. For example, Table 4 Level of Review for Level 1 and Level 2 Precertified Organizations SaMD, could be appended with the corresponding Device Classification (e.g. I-III).

4 Docket. FDA-2017-N-4301 September 20, 2018 Page 4 of 5 o o o Understanding the applicability of existing guidance documents such as the Software Level of Concern Guidance to the program would be valuable for program implementation. We suggest FDA consider clarification regarding software product categories that are understood to be outside the pilot s scope at this time, but are not specifically excluded from the 0.2 working model (e.g. SiMD and combination products). There continues to be confusion regarding software that remains under enforcement discretion, and how such software would be incorporated under the Precertification program, including the potential registration and listing that would be handled under the Review Determination Template. Inclusion of specific examples of these products may aid in fostering common understanding. Comments on Working Model Component 3, Streamlined Review: AdvaMed applauds FDA for inclusion of a streamlined submission process in the precertification program for software that requires premarket review. A streamlined review approach that is modern, flexible, and risk-based should reduce the time and cost necessary to bring these innovative, rapidlyevolving products to patients and health professionals. Our members have discussed at considerable length what an ideal process for Streamlined Premarket Review of SaMD could be, and have previously proposed that all capabilities, commitments, practices, protocols, processes, and risk management approaches be assessed during the companyspecific excellence appraisal. We believe that only product-specific information should be reviewed during the streamlined review, when streamlined review is required, and continue to believe that the program would not alter the FDA s scientific standards for review, but instead change where and when such information is reviewed. We agree with FDA that emerging technologies reviewed under the program must still be proven safe and effective in order to protect patients. We have attached for FDA s consideration two proposals related to the program s streamlined premarket review. Appendix E is focused on the process for streamlined review; Appendix F is focused on the review elements or content that should be included in the streamlined review submission. We believe FDA should consider not only what information is evaluated but the manner in which it is evaluated. Regarding the process for premarket review, as described in Appendix E, we have leveraged FDA regulatory tools currently available, such as FDA s Pre-Submission Process, in order to allow for faster and more interactive review, such as review by demo. We believe it is inappropriate to conduct a paper review of SaMD when an interactive demonstration can more readily address questions reviewers may have and allow for real-time responses and demonstration of logic from developers. The process we propose allows the software developer flexibility to engage with FDA at various stages of SaMD development from early in the development process to when the SaMD is more fully formed. Early engagement is not required but can be a helpful option, particularly for novel products. Conclusion of the interactive review process results in a Confirmation Document, which is submitted to FDA as the file for review.

5 Docket. FDA-2017-N-4301 September 20, 2018 Page 5 of 5 Appendix F provides a chart of review elements necessary for SaMD as part of a streamlined premarket review; it also describes where data that is not submitted would be reviewed (e.g., as part of the Excellence Appraisal). As stated previously, all capabilities, protocols, and processes should be assessed during the company-specific excellence appraisal, and only product-specific information should be reviewed during the streamlined review, when streamlined review is required. This chart also provides a helpful depiction of where other relevant and appropriate information currently submitted during premarket review is assessed, either in the Excellence Appraisal, through Review Determination, Streamlined Review, or collected during Real World Performance. Please note that the process and review elements currently only address 510(k) and de novo submissions. Although we do not think there are significant differences in what a PMA submission should look like under the program, we intend to address PMA s in a future submission. * * * AdvaMed would like to thank the FDA for its consideration of these comments and looks forward to continuing to work with the Agency on this important issue. Please do not hesitate to contact me at or zrothstein@advamed.org if you have any questions. Respectfully submitted, /s/ Zachary A. Rothstein, Esq. Associate Vice President Technology and Regulatory Affairs Attachment

6 Appendix A: Review Determination Template The following information would be completed by software developers for each SaMD product and submitted to FDA through an online portal. FDA would use identified fields for registration and listing purposes, which would be publicly available, and other fields would be used only by FDA for purposes of review determination. If it is determined by the developer or by FDA (via audit) that streamlined review is required, the information entered here would be included as part of the streamlined review. Public Information as part of Registration/Listing n-public Information (submitted to FDA only) 1 Applicant Name 2 Proprietary and Established Name of the Device 3 Intended Use or Purpose (State of the healthcare situation or condition) 3a (Significance of Information of the SaMD to the healthcare decision) 4 Device Description 4a 4b (High-level summary of device including core functionality and key technology) Device summary separating device functions-under-review and other functions Existence and nature of any software technology (e.g. algorithm, machine learning) 5 Precertification Status 6 Risk Categorization (Identify if SaMD is Treat/Diagnose, Drive, or Inform and Critical, Serious, or n- Serious) Review Determination Template 1

7 Appendix B. Determining Regulatory Status of Software 2

8 Appendix C. Determining Risk Categorization of Regulated Software. 3

9 APPENDIX D: Interpretations of IMDRF Risk-Categorization Framework IMDRF language = black text Recommended interpretations = red text A. Significance of Information To treat or to diagnose - To provide therapy to a human body; - To diagnose/screen/detect a disease or condition Output of the SaMD is intended to be used to: - Definitively diagnose a disease or condition; - Provide direct treatment or definitive treatment information for a disease or condition; Output of the SaMD is the sole determinant for clinical action and requires no further steps or confirmatory testing. Output of the SaMD is intended for immediate or near-term clinical action. To drive clinical management - To aid in treatment by providing enhanced support to safe and effective use of medicinal products or a medical device. - To aid in making a definitive diagnosis. - To triage or identify early signs of a disease or conditions. Output of the SaMD is intended to be: - One of several inputs used for clinical action and/or decision-making; and - Necessary for clinical action or decision-making by the health care professional or patient, i.e. determinative. Output of the SaMD is intended for immediate or near-term clinical action. To inform clinical management - To inform of options - To provide clinical information by aggregating relevant information Output of the SaMD is intended to be: - One of several inputs used for clinical action and/or decision-making; and - t necessary for clinical action or decision-making by the health care professional or patient and may or may not lead to direct clinical action, i.e. informative or adjunctive. 4

10 B. State of the Healthcare Condition Critical Serious n-serious Situations or conditions where accurate diagnosis or treatment is of vital importance to avoid unnecessary interventions (e.g., biopsy) or timely interventions are important to mitigate long-term irreversible consequences on an individual patient s health condition or public health. Situations or conditions where accurate and/or timely diagnosis or treatment action is vital to avoid death, long-term disability or other serious deterioration of health of an individual patient or to mitigating impact to public health. SaMD is considered to be used in a critical situation or condition where: The type of disease or condition is: Life-threatening state of health, including incurable states, Could result in permanent impairment of body function or in the structure of the body, Requires major therapeutic interventions, Sometimes time critical, depending on the progression of the disease or condition that could affect the user s ability to reflect on the output information. Intended target population is fragile with respect to the disease or condition (e.g., pediatrics, high risk population, etc.) Intended for specialized trained users. The type of disease or condition is: Moderate in progression, often curable, Could result in temporary impairment of body function or in the structure of the body, Does not require major therapeutic interventions, Intervention is normally not expected to be time critical in order to avoid death, long-term disability or other serious deterioration of health, whereby providing the user an ability to detect erroneous recommendations. Intended target population is NOT fragile with respect to the disease or condition. Intended for either specialized trained users or lay users. te: SaMD intended to be used by lay users in a "serious situation or condition" as described here, without the support from specialized professionals, should be considered Situations or conditions where an accurate diagnosis and treatment is important but not critical for interventions to mitigate long term irreversible consequences on an individual patient's health condition or public health. SaMD is considered to be used in a nonserious situation or condition when: The type of disease or condition is: Slow with predictable progression of disease state (may include minor chronic illnesses or states), Is not likely to result in temporary impairment of body function or in the structure of the body, May not be curable; can be managed effectively, Requires only minor therapeutic interventions, and Interventions are normally noninvasive in nature, providing the user the ability to detect erroneous recommendations. Intended target population is individuals who may not always be patients. Intended for use by either specialized trained users or lay users. 5

11 as SaMD used in a "critical situation or condition". Justification for added text: The added text simply aligns the definition with current US definitions by taking into account risk of impairment of body function or structure. Justifications for deleted text: Patient population. The fragile nature of the patient population would impact the type of controls (or risk mitigations) that are necessary, but it would not impact the risk classification. This is consistent with prior GHTF guidance. As explained in the GHTF Classification guidance: The Risk Assessment (RA) takes account of the probability that harm will occur by modifying the evidence requirements at the conformity assessment stage rather than modifying the classification rules. Probability of harm is influenced by factors such as whether: o the technology is regarded as mature; o the device type is the source of many adverse event reports; o the device s manufacturer has a long experience of the device and the technologies it embodies; o the device user is a lay person Therefore, the nature of the user could impact the type or amount of evidence/information required, but not the classification itself. In addition, we believe that simply because the software is intended to be used only by a specialized user should not increase the risk to patients. Conversely, use by a specialized user should reduce the risk of the software. In reviewing the applicable GHTF documents and IMDRF SaMD documents, these descriptions seem appropriate with the changes noted above. Consideration was also given to how health hazard evaluations (HHEs) are currently conducted (including the FDA HHE). Additions based on this review are underlined in the recommended modifications above. te, if FDA determines it is necessary to keep the concepts in the deleted text, we recommend that FDA refer to these areas as factors that a developer should consider or guidance, rather than determinative. FDA should consider that SaMD may have general indications for use rather than for a specific disease or condition (e.g., minimally invasive surgery compared to minimally invasive neurosurgery). This is also true for target populations, which may be general rather than a specific sub-population. FDA should also clearly define terms, such as major therapeutic interventions and provide examples. FDA should consider language that references SaMD that may monitor or predict a condition, which may not necessarily be considered an intervention. 6

12 Appendix E: Streamlined Premarket Review Process 1

13 2

14 3

15 4

16 5

17 Appendix F: Review Elements for Software Precertification Streamlined Premarket Review ** Additional information required for a de novo streamlined review. Potential Review Elements (from Table 5 of FDA Pre-Cert V 0.2) 3rd party software Anomalies / Bugs Clinical Performance Excellence Appraisal (EA) The review process for selection and risk management of 3rd party/off the shelf software (OTSS) should be assessed during the EA. Overly burdensome and irrelevant requirements (e.g. CSV-level validation of an industry-standard development tool, simply because it is making a SaMD) should be eliminated. The final software product will be validated so there is no need to do additional work on each 3rd party component. The process and tools used to detect and correct bugs and anomalies, as well as the process used for risk analysis of the bugs, should be assessed as part of the EA. The processes for clinical evaluation for SaMD should be assessed during the EA. te that these processes may include various forms of clinical evaluation, as recognized by the FDA guidance and IMDRF guideline, Clinical Evaluation of Review Determination Process (RDP) [Submission of a SaMD Definition Statement] Streamlined Product Review Process (SPRP) [Submission of a Confirmation Template] te: contemplates an interactive review; but may be done in a traditional but streamlined review 3rd Party/OTSS software of importance would be listed in the Software Bill of Material (SBOM) A confirmation statement that the launched software contains no critical unresolved anomalies should be included in the streamlined review within a confirmation template (which could be submitted as a final document or developed during an interactive review). To support this, a summary of residual anomalies including risk level could be shown as part of the review, if needed. In cases in which a streamlined review is required, a description of the clinical evaluation testing should be provided in streamlined review. This can be accomplished via an interactive presentation. A summary/results of the clinical evaluation Real World Performance (RWP) The number of bugs detected/ corrected could also be tracked as a development and/or a post-market KPI; but it should be remembered that the number of bugs does not reflect the performance of the device. Real World Health Analytics can be collected to provide information that the product performs as intended and is not associated with any unexpected result when used in a real world setting, including but not limited to 6

18 Potential Review Elements (from Table 5 of FDA Pre-Cert V 0.2) Configuration management Excellence Appraisal (EA) SaMD. This includes alternatives to traditionally designed and monitored clinical trials, such as literature, beta testing or human factors/usability studies among others. The process for configuration management should be assessed during the EA only. Review Determination Process (RDP) [Submission of a SaMD Definition Statement] Streamlined Product Review Process (SPRP) [Submission of a Confirmation Template] te: contemplates an interactive review; but may be done in a traditional but streamlined review process testing and results, sufficient to support clinical performance claims appearing in the labeling, should be provided in a confirmation template. Please note that assurance of clinical evaluation may be shown through various forms of evidence including for example, literature, beta testing, human factors/usability studies, "in-silico" testing, and/or real world data and should rarely require a traditionally designed and monitored GCP-level clinical evaluation. Cover letter This can be a helpful part of the submission and should be included at least in the 2019 streamlined review. In the short term, the CDRH Submission Cover Sheet Form 3514 should also be included, and the submission identified as a SaMD Streamlined Review submission. Cybersecurity The process and approach for premarket cybersecurity (including security by design and security risk management) as well as postmarket management of cybersecurity should be assessed during the EA. Process-related cybersecurity documentation would not be required for submission in individual product submissions. Confirmation template should include a statement certifying compliance with relevant FDA recognized consensus standards dealing with IT, and a statement confirming that any residual risks are acceptable. This would cover cybersecurity information. The labeling should also contain instructions for use for any recommended cybersecurity controls to be implemented by the user. Real World Performance (RWP) adverse event/complaint tracking, trending, and reporting. It could also provide information to support additional claims or continual improvement. In the case of continually learning AI, the method used to confirm that clinical performance remains in the labeled range even as the software 'learns' should be described. Post-market changes aimed at improving cybersecurity would not require submissions if implemented in accordance with FDA guidance. 7

19 Potential Review Elements (from Table 5 of FDA Pre-Cert V 0.2) Declarations of conformity and summary reports Design and Development Artifacts for Marketed Devices Development Environment Excellence Appraisal (EA) During the excellence appraisal, organizations should identify the standards and guidelines to which they comply as a matter of process. The process to consider artifacts can be assessed in the excellence appraisal. Other than that, artifacts for a specific device need not be called out specifically; device safety and effectiveness will be accurately determined despite their presence. This description should be part of the EA. Review Determination Process (RDP) [Submission of a SaMD Definition Statement] Streamlined Product Review Process (SPRP) [Submission of a Confirmation Template] te: contemplates an interactive review; but may be done in a traditional but streamlined review In general, declarations of conformity to relevant standards and guidelines should substitute for intensive data reviews for precertified organizations. For particular device-related performance characteristics, summary data can be reviewed during the streamlined review (e.g. via presentation), and declarations of conformity can be provided in the confirmation document. For any specific performance characteristic claimed in the labeling, a statement of the obtained result should be included in the confirmation template, along with identification of the standard and declaration of conformity. Real World Performance (RWP) 8

20 Potential Review Elements (from Table 5 of FDA Pre-Cert V 0.2) Developmental Testing (e.g., unit, integration, functional, endto-end, regression, penetration testing) Device Description Excellence Appraisal (EA) The process, approach, and tools should be thoroughly covered in the EA. Review Determination Process (RDP) [Submission of a SaMD Definition Statement] The device description, including core functionality and key technology, should be provided in the SaMD Definition Statement. For a multifunctional device, only the core functions that require review should be described in detail. Brief identification of functions other than the functionunder-review should also be provided. Streamlined Product Review Process (SPRP) [Submission of a Confirmation Template] te: contemplates an interactive review; but may be done in a traditional but streamlined review Output from product specific development testing should be covered during interactive review. The vast majority of this should be reviewed during a presentation or demo. Automated tools (e.g., for auto-verification) could be used to provide summaries of development KPIs. Summaries could include defect categorization and severity rates, defect find and closure rates, and code coverage for risk-related requirements. A statement that the software was developed and tested consistent with the organization's certified process, and that all requirements were met, should be all that is necessary in the confirmation template for a precertified company (Level 2); For level 1 the statement could be accompanied by a written summary/ final "dashboard snapshot" of key development KPIs. The same device description from the RDP should appear (or be referenced) in the confirmation template; no further written description should be necessary. Key to the review should be a demo of the software, whether by submission of a wireframe in early interactive review, interactive demo, or other means. Real World Performance (RWP) 9

21 Potential Review Elements (from Table 5 of FDA Pre-Cert V 0.2) Excellence Appraisal (EA) Review Determination Process (RDP) [Submission of a SaMD Definition Statement] Streamlined Product Review Process (SPRP) [Submission of a Confirmation Template] te: contemplates an interactive review; but may be done in a traditional but streamlined review Device Summary The device description SaMD Definition Statement can suffice as the device summary and appear in the confirmation template. For descriptive purposes only, device functions other than the function-under-review ( other functions ) can be part of the software demo/ interactive review but only shown at a high level. other information from "other functions" should appear in the confirmation template, other than in the risk analysis and instructions for use in the labeling. specific software development/validation and verification information, etc. should be presented or reviewed for these "other functions." Executive summary Financial certification and disclosure statement. There is no need for an executive summary. The 510(k) or de novo summary, which should only contain information from the confirmation template, should suffice. In a traditional 510(k) or de novo with extensive documentation, an executive summary may make sense, but in a streamlined document such as is anticipated for the Streamlined Review process, an executive summary no longer has value. FDA s Form 3454, a summary of Financial Certifications, would only be required in the limited cases in which a clinical study is required that is covered under 21 CFR Part 54. This should be rare, confined only to the very highest risk software. Real World Performance (RWP) 10

22 Potential Review Elements (from Table 5 of FDA Pre-Cert V 0.2) Excellence Appraisal (EA) Review Determination Process (RDP) [Submission of a SaMD Definition Statement] Intended Use Intended Use statement of the SaMD Definition Statement should describe the significance of the Information of the SaMD to the healthcare decision. Indications for Use Performance Claims Instructions for Use Intended Use Population Indications for Use statement of the SaMD Definition Statement should describe the state of the healthcare situation or condition Desired Performance Claims should be described in the SaMD Definition Statement. If the numerical data for the claim is not yet available, a qualitative statement can be used or the data field left blank. Streamlined Product Review Process (SPRP) [Submission of a Confirmation Template] te: contemplates an interactive review; but may be done in a traditional but streamlined review The intended use statement from the SaMD definition statement should appear in the confirmation template. The indications for use statement from the SaMD Definition Statement should appear in the Confirmation template Performance claims will appear in the device labeling; and the labeled claims should also be part of the confirmation template. Supporting information for the claims should be reviewed during the review, ideally by presentation. See declaration of conformity and summary reports section. The interactive review 'demo' can provide a 'live illustration' of the instructions for use. The labeling should contain the instructions for use and be provided as part of the review. need to treat this as a separate item. Redundant to information provided for intended use and indications for use. need to treat this as a separate item. Redundant to information provided for intended use and indications for use. Labeling Review Yes (combine with instructions for use) Life Cycle This process should be assessed during the EA. Real World Performance (RWP) 11

23 Potential Review Elements (from Table 5 of FDA Pre-Cert V 0.2) Mechanism of Clinical Action (Clinical Algorithm) Modifications made since clearance Post-Cert Inspection Excellence Appraisal (EA) As part of the description of the development process and configuration/lifecycle management, the process for change control should be described. This includes the process for developing preapproved change protocols for certain kinds of changes. This also includes the process for version control. Under the precert program, only very few changes will require submission for Level 1 PreCert). A post-cert inspection should not be part of the streamlined review of an individual product. Review Determination Process (RDP) [Submission of a SaMD Definition Statement] The SaMD Definition Statement should include the existence and nature of an algorithm (e.g. what are the data sources or types; is it based on public vs proprietary formulas; uses simple calculations vs database lookup vs modebased logic vs artificial intelligence and/or machine learning). In the alternative, a description of the mechanism of action should be included within the Device Description (core functionality and key technology). In the SaMD Definition Statement for a modified device, only the specific major modification requiring re-submission should be included (according to Table 4 of Pre-Cert v 0.2). Streamlined Product Review Process (SPRP) [Submission of a Confirmation Template] te: contemplates an interactive review; but may be done in a traditional but streamlined review For cases in which review is required, an algorithm demonstration can be part of review of performance testing that the algorithm functions correctly; ideally, such review would be interactive with the developer and FDA, challenging the software together. The validation testing should be focused on demonstrating clinical performance for its intended use and would be a key component of clinical performance evaluation. A summary/results of the algorithm testing process and results, sufficient to support any labeling clinical performance claims, should be provided in the confirmation template submitted - the overall clinical evaluation testing will of necessity cover this. Only the major modification that triggered the re-submission should be included in the streamlined review. Real World Performance (RWP) There will be a standard package of postmarket information that is collected, often automatically, including change history and versioning. This is to be described during the excellence appraisal and should not ordinarily need review for a specific product. 12

24 Potential Review Elements (from Table 5 of FDA Pre-Cert V 0.2) Excellence Appraisal (EA) Review Determination Process (RDP) [Submission of a SaMD Definition Statement] Streamlined Product Review Process (SPRP) [Submission of a Confirmation Template] te: contemplates an interactive review; but may be done in a traditional but streamlined review Real World Performance (RWP) Post-Market Data Requirements The process and tools used to collect and evaluate post market data (e.g. real world performance) should be described and assessed during the EA. A standard "postmarket data collection package" can be established with KPIs for standard elements like bug fixes, error resolution, crashes, cybersecurity updates, etc. The process for defining new release intervals and related review cycles should be included as well. The process for setting requirements should be described. n-value-added requirement 'levels' (e.g. requirements from traditional CSV like the need for separate customer - product - software - and unit requirements) should be eliminated. 13 During the interactive demo contemplated in a streamlined review, any plans for postmarket data collection for a specific product (i.e. beyond the standard "postmarket data package") can be negotiated and agreed upon, based upon the risk, the critical elements of the software and the level of available premarket data, and any remaining concern areas. Agreements concerning non-standard product-specific postmarket data collection should be documented in the confirmation template - but not information on 'standard post-market data elements.' During an interactive review - demo - the relationship between the requirements and the output can be described. An automated summary of requirements testing could be produced as described in 'developmental testing.' Formal documentation of software requirements per design control should not be required for SaMD. The confirmation template statement described in "developmental testing" should suffice for requirements. Revision History This is a data element that is a standard part of software development. There is no need for its specific review. The content is covered in other sections. Risk Analysis Description of the risk analysis process The risk of the SaMD should Depending on the level of Precertification. Yes Real world performance collection will be the main source of the post market data. Either the SaMD should be built to contain functionalities that can provide this information or plans to collect the information must be in place.

25 Potential Review Elements (from Table 5 of FDA Pre-Cert V 0.2) SaMD Product Demo Excellence Appraisal (EA) is a key part of EA. It should include analysis of cybersecurity risks, patient safety risks, and a process to determine the impact of "other functions" (Besides the function-under-review) on the safety and effectiveness of the device. Review Determination Process (RDP) [Submission of a SaMD Definition Statement] be assessed according to the IMDRF N12 risk framework, with adaptations to the US system as recommended under separate cover. 14 Streamlined Product Review Process (SPRP) [Submission of a Confirmation Template] te: contemplates an interactive review; but may be done in a traditional but streamlined review Level 2: Risk analysis summary table can be presented during streamlined review as part of an interactive demo or presentation of software. Focus can be on top-level, more critical risks. Level 1: More detail on the risk analysis process, mitigations, and summary can be provided either during review or as a summary report document. In either case, (besides the info on cybersecurity risk) a statement that the hazard analysis has been reviewed and that the launched software contains no critical residual risks should appear in the Confirmation Document. The confirmation document should also contain a statement summarizing the overall SaMD risk (e.g. type and subtype per precert v0.2). An interactive product demo should be the first scheduled and main feature of the review process. It should replace the need for extensive documentation and written protocol descriptions and result data. The interactive product demo can be accomplished either by wireframe, interactive demo, or even reviewer-challenge of a beta testing version. An interactive process allows for real-time questions and answers and increases understanding of the product by reviewers. The product demo could cover elements of device description, device summary, intended use indications and population, instructions for use, clinical algorithm, 3rd party software, software architecture, User Interface/story/workflow. Real World Performance (RWP)

26 Potential Review Elements (from Table 5 of FDA Pre-Cert V 0.2) Server Performance / Load Testing Software Architecture Software Bill of Materials Excellence Appraisal (EA) The process for this should be noted and assessed in the EA. EA covers the configuration management process and can include the process for preparation of SBOM. This is an internal document describing the final configuration and should not necessarily need to be reviewed for every product. Review Determination Process (RDP) [Submission of a SaMD Definition Statement] Since software architecture is the first thing a software reviewer would want to look at to understand the product, a high level diagram/description of this should be in the SaMD Definition Statement, in the device description/key technological characteristic section. Streamlined Product Review Process (SPRP) [Submission of a Confirmation Template] te: contemplates an interactive review; but may be done in a traditional but streamlined review A statement summarizing the review elements covered by interactive product demo could be included in the confirmation template.. As it is a standard element of software development, it need not be reviewed separately for each device. The statement affirming that all requirements were met would cover this element. A software architecture diagram or document should be submitted, as this is basic to the understanding of the software. Software architecture should be reviewed in the interactive demo. A software architecture diagram could be included in the confirmation template. The SBOM could be presented during the interactive review upon request, and a statement attesting to its availability can be in the confirmation template. We understand that FDA is considering submission of SBOMs as a premarket requirement. Real World Performance (RWP) 15

27 Potential Review Elements (from Table 5 of FDA Pre-Cert V 0.2) Substantial equivalence (SE) comparison Excellence Appraisal (EA) Review Determination Process (RDP) [Submission of a SaMD Definition Statement] For 510(k) Only. Streamlined Product Review Process (SPRP) [Submission of a Confirmation Template] te: contemplates an interactive review; but may be done in a traditional but streamlined review For a 510(k), a legally marketed predicate must be identified, and a SE comparison table should be generated and reviewed during streamlined review. The comparison table as well as a statement affirming substantial equivalence to the legally marketed predicate should appear in the confirmation document for a 510(k). Real World Performance (RWP) Traceability The process for ensuring traceability can be described in the software development process assessment or covered by certification. Direct comparison studies need not always be performed. In the cases where clinical evaluation can be accomplished by performance testing to compare to an onmarket reference software that is also the legally marketed predicate. As with other clinical evaluation testing, a summary of the testing process and results can be presented during the review, and depending on the level of precert and subtype of software, a statement of results or protocol summary and results of the clinical performance evaluation process and results, sufficient to support any specific labeling clinical performance claims should be included in the confirmation statement. For Level 1 pre-cert (or in the event of a product audit), a traceability summary can be generated ideally by automated software and presented or submitted. The statement affirming that all requirements were met would cover this element. 16

28 Potential Review Elements (from Table 5 of FDA Pre-Cert V 0.2) Truthful and accuracy statement User Interface / Story / Workflow Verification and Validation Excellence Appraisal (EA) Review Determination Process (RDP) [Submission of a SaMD Definition Statement] Streamlined Product Review Process (SPRP) [Submission of a Confirmation Template] te: contemplates an interactive review; but may be done in a traditional but streamlined review Yes. Yes The process for software verification and validation should be described and assessed during the EA. The streamlined review should include specific product process for software development, verification and validation, including verification of correct software development by use of KPIs, dashboard updates and other information and a summary of clinical validation process, including testing and results. As noted above, a statement affirming that the software was developed under the manufacturer's certified system, and that all performance requirements were met, and that any residual risks are acceptable, should suffice in many cases. For any labeled performance claims, a summary of the process and results should also appear in the certification template. The level of submitted documentation necessary to support verification and validation should depend on the risk level of the software and the precert level of the manufacturer. Yes Real World Performance (RWP) 17

29 Potential Review Elements (from Table 5 of FDA Pre-Cert V 0.2) de novo request** Usability information** Excellence Appraisal (EA) Review Determination Process (RDP) [Submission of a SaMD Definition Statement] Streamlined Product Review Process (SPRP) [Submission of a Confirmation Template] te: contemplates an interactive review; but may be done in a traditional but streamlined review For a de novo, the request for de novo review should be included in the streamlined review - including a statement that no predicate is available and benefit-risk information sufficient to describe why this should qualify for a de novo. The confirmation template should reflect that the product will undergo de novo review, and de novo summary information, a high level risk identification and mitigation table, summary risk-benefit information, and identification of any necessary special controls. The process for the evaluation of safety considerations for device users in various use environments/user interfaces can be described and assessed during the EA. The process for postmarket collection and analysis of user experience data can be described and assessed in the EA as well. A summary of usability information/feedback can be presented during interactive review. In most cases, clinical evaluation involving testing in the hands of the intended user should provide sufficient evidence of usability. Alternatively, a declaration of conformity to EN/IEC may satisfy this requirement as well. Real World Performance (RWP) User experience analytics can be used to optimize future software versions. ** Additional information required for a de novo streamlined review. 18