The SaaS LMS and Total Cost of Ownership in FDA-Regulated Companies

The SaaS LMS and Total Cost of Ownership in FDA-Regulated Companies

The SaaS LMS and Total Cost of Ownership in FDA-Regulated Companies By Rob Sims, UL Compliance to Performance When Life Sciences companies evaluate corporate Learning Management Systems (LMS), they often calculate a total cost of ownership (TCO) in addition to gathering the organization s functional and IT requirements. From HR to Sales to Quality Assurance, the entire organization must determine that the LMS will meet their unique requirements. For example, Quality Assurance has specific quality and regulatory goals: job function qualification records, security, audit reporting, version control and more. When these needs are coupled with TCO, more and more organizations are recognizing the value of the Learning Management Systems (LMS) delivered via the Software as a Service (SaaS) model. According to industry analyst IDC, by 2020 SaaS-based providers will outpace traditional software deployment by over 25%, and installed applications will represent only10% of new enterprise installations (source: IDC 50th Anniversary, 2014). And a 2016 elearning Guild study cites that 57% of organizations are using a SaaS LMS. TCO is a primary driver for any SaaS (Software as a Service) solution, as a SaaS application typically includes pay-as-you-go pricing, requires less IT resources, no hardware investment, no disaster recovery investment and no hardware maintenance costs. For FDA regulated companies, two significant cost advantages are related to reduced validation effort and deployment time. For Life Sciences organizations, the validation effort is a time-consuming, resourceintensive process. However, with the SaaS LMS, validation becomes a shared responsibility between the client and the vendor, and this distribution of resources reduces the validation costs significantly, especially when the software is upgraded. That s because with a server-based application, the client s engineering and IT personnel are responsible for designing and maintaining the internal data center, computer hardware, software, security, disaster recovery, etc. In the SaaS model, much of this burden is shifted to the SaaS vendor. This makes an audit of the SaaS vendor a critical activity for the client s validation, auditing, IT and QA team. Page 2

Adhering to FDA Validation & 21 CFR Part 11 Regulations The FDA defines software validation as confirmation by examination and provision of objective evidence that software specifications conform to user needs and intended uses, and that the particular requirements implemented through software can be consistently fulfilled. (source: www.fda.gov). Even when a software vendor has a well defined process, each business is expected to ensure that the organization s use of the software is validated. The FDA expects developers to follow controls and procedures that are specified in a Software Development Life Cycle (SDLC) to ensure quality. The FDA requires that appropriate testing be performed, and this is achieved by executing test scripts that map back to the software requirements. Testing often consists of Installation Qualification (IQ), in which documentation verifies that the system is installed according to written specifications; Operational Qualification (OQ), in which documentation verifies that the system operates throughout all operating ranges; and Performance Qualification (PQ), in which tests must span the underlying cgmp business process, to ensure that users and administrators trained in the SOPs can accomplish business objectives in the production environment. In addition to validation requirements, the system must meet electronic record and electronic signature requirements as outlined in 21 CFR Part 11. Here are just three critical requirements of Part 11: KEY TOPICS: Adhering to FDA Validation & 21 CFR Part 11 Regulations The LMS Deployment: Auditing the LMS Vendor Reducing Costs Following the Deployment Summary Audit Trails: Use of time-stamped audit trails to independently record the date and time of operator entries and actions that create, modify or delete electronic records. Authority Checks: Use of authority checks to ensure that only authorized individuals can use the system, electronically sign a record, access the operation or computer system input or output device, alter a record or perform the operation at hand. Electronic Records: Signed electronic records need to contain information associated with the signing that clearly indicates these three items: 1) the printed name of the signer; 2) the date and time the signature was executed; and 3) the meaning (such as review, approval, responsibility or authorship) associated with the signature. In a similar way, EU Annex 11 also focuses on the product life cycle, as well as user requirements related to each system. However, Annex 11 places more emphasis on people and management accountability than FDA regulations specifically the individuals with responsibility for the business process and system maintenance. To address these regulatory obligations, clients perform a number of activities: create the Solution Design Document (SDD), develop User and Functional Requirements, develop test scripts and conduct IQ, OQ and PQ testing, among other activities. This effort drives up the IT and validation teams time and effort, and often adds months to any software implementation or enhancement project. Page 3

When using a SaaS LMS, clients must rely on the vendor to perform and maintain critical validation activities. This means the client must audit the vendor s QA and validation methodology, to ensure these activities are performed to meet the standards of the client s QA and validation teams. Typically, the three days or so taken to audit a vendor can dramatically reduce the time spent validating the system. The LMS Deployment: Auditing the LMS Vendor For the SaaS LMS, clients often conduct an audit of the vendor at the vendor s primary location, so the vendor can demonstrate adherence to quality software engineering and testing principles. The client s validation teams will typically review QA documentation and project files that were developed as part of the design, development, testing and implementation of the LMS. The vendor s QA team will need to demonstrate that a valid development methodology is in place and that thorough testing was used to provide confidence and assurance to the client that the LMS is fit for production use. Furthermore, the vendor s QA team must demonstrate that it is involved with these activities, which should occur throughout the entire SDLC stage: Business rules, graphical user interface design elements and interoperability with existing features Functional specification IQ and OQ Process for system releases to clients Test traceability and full product testing before any new LMS code is released to a production environment Typically, when a client audits ComplianceWire, our SaaS-based LMS, these are the documents that are usually requested for review: Business Requirements: This document focuses on the needs of the user and spells out exactly what the system will do Functional Requirements: This document focuses on how the system will do what the user is expecting System Design: This document focuses on capturing the system design based on the functional requirements. Depicting screen layout, system functions and other aspects of the user experience to fulfill the business requirements Requirements Traceability Matrix: This document captures the relationship between the business and functional requirements and the test scripts that satisfy them; this document should reflect the latest enhancements made to the platform Test Scripts: These documents support new enhancements or custom projects. These scripts should cover basic platform functionality, CFR functional and reporting features and high priority areas of the LMS. In addition, custom testing should be executed for any custom programming performed Page 4

Test Plan (TP) and Validation Plan (VP) SOPs: Because ComplianceWire is a single platform, these documents have been adopted as Standard Operating Procedures Validation Summary Report (VSR): This report summarizes our testing activities, discrepancies and other validation activities Based on real-world experiences with clients, following an audit of UL s data center and SDLC methodology, a client s audit and validation team will then focus on creating these documents: Validation Plan: Describes the internal activities that are of part of the overall validation approach to be conducted by the client User Requirements Specification: Provides details on the functionality that the clients demands from ComplianceWire. These required items are often categorized as critical, mandatory or nice to have, and are the basis for the validation/testing effort User Acceptance Test Scripts: Clients should have their own test scripts that either augment our own QA test scripts, or unique scripts that demonstrate that the company has tested specific usage of the system Validation of Configurations and Customizations: Clients often test their specific configurations of ComplianceWire (Security Roles, Custom Fields, Reports, etc.) and any customized solutions The following table summarizes the typical responsibilities as described above. Typical SaaS LMS Validation Responsibilities Vendor Documents Shared During an Audit: Business Requirements Functional Requirements System Design Requirements Traceability Matrix Client Responsibilities: Validation Plan User Requirements Specification User Acceptance Test Scripts Validation of Configurations and Customizations Test Scripts Test Plan and Validation Plan SOPs Validation Summary Report (VSP) Page 5

Reducing Costs Following the Deployment With a server-based LMS, the vendor s release schedule determines ongoing validation/ re-validation activities, and has the potential to drive up TCO, as it impacts the resources of QA, validation and IT. One client noted that IT staff costs associated with periodic review of the server-based LMS, to ensure that validation status has been sustained, is often not noted in the LMS TCO. And for many of our clients who previously relied on a server-based LMS, the costs of re-validating the software prevented them from upgrading the software to a new version, and thus not gaining the benefits of these new features. With a SaaS LMS, the vendor can reduce this effort. First, the vendor can provide clients with a preview period that includes the ability to test the new features on the vendor s test site. Regression test scripts should be provided to aid the client with their own internal validation effort. The vendor should also provide a validation summary report at the end of all release activities. Based on conversations with clients who have actually migrated from a server-based LMS to a SaaS-based LMS, we ve created two scenarios based on a large company (10,000 employees) and a small to mid-size company (500 employees). While the cost assumptions are not based on any single LMS implementation, we do consider actual costs conveyed to us from clients who have migrated their LMS from a server-based LMS, factoring in the cost of IT staff, validation staff and external IT contractors who are involved in the server-based installation and ongoing maintenance. Please note that we do not factor in the migration of historical data, as this cost will vary depending on the amount of historical data to be migrated, regardless of whether the LMS is server-based or SaaS. In both company scenarios, the Year 1 validation cost savings are reduced by the SaaS LMS, as this requires significantly less IT and validation resources. However, cost savings of the SaaS LMS continue during Year 2 through Year 5, because the validation team doesn t have to validate the entire software and IT infrastructure with each software enhancement. This benefit occurs regardless of company size. What we have seen is that in some cases, SaaS provides additional benefits to the small to mid-size company, as it provides built-in scalability to accommodate new users and expand operations without placing a burden on IT infrastructure. Some organizations consider IT and validation resources as a sunk cost because employees are performing the work. As our clients have pointed out, the server-based LMS upgrade still requires a great deal of effort, and organizations should not neglect the opportunity costs (the cost of passing by the second-best option when making a decision. To paraphrase one client: Other software projects are sacrificed, sometimes for months, when the IT and validation teams have to perform a single server-based upgrade. For that reason, in the scenarios that follow, we have presented costs associated with a server-based implementation that includes IT resources. Page 6

Scenario #1: Large Company 10,000 Global Users Year 1 LMS Cost Comparison Budget: Server Based SaaS Purchase of Licenses $300K - Subscription Fee - $300K 20% Annual Maintenance Fee $60K - Purchase of Hardware/Storage $330K - IT Support $120K - Project Management & Validation $500K $50K Configuration & Interface Dev $350K $75K Total Year 1 Budget: $1.66M $425K Scenario #2: Emerging Company 500 Global Users Year 1 LMS Cost Comparison Budget: Server Based SaaS Purchase of Licenses $25K - Subscription Fee - $25K 20% Annual Maintenance Fee $5K - Purchase of Hardware/Storage $20K - IT Support $10K - Project Management & Validation $50K $10K Configuration & Interface Dev $30K $10K Total Year 1 Budget: $140K $45K Year 2 through Year 5 Cost Comparison Budget: Server Based SaaS Subscription Fee - $1.2M 20% Annual Maintenance Fee $240K - Purchase of Hardware/Storage $120K - IT Support $480K - Validation Related to Upgrades $1M $100K Total Year 2 5 Budget: $1.84M $1.3M Total Budget: $3.5M $1.725M Total Cost Difference Over Five Years: Companies Save $1.7M with the SaaS LMS Year 2 through Year 5 Cost Comparison Budget: Server Based SaaS Subscription Fee - $100K 20% Annual Maintenance Fee $20K - Purchase of Hardware/Storage 10K - IT Support $40K - Validation Related to Upgrades $120K $10K Total Year 2 5 Budget: $190K $110K Total Budget: $330K $155K Total Cost Difference Over Five Years: Companies Save $175K with the SaaS LMS Summary: The SaaS LMS has been positioned to allow small and mid-sized companies to gain quality-focused functionality that addresses FDA regulatory needs, while also being scalable to accommodate future growth. From our real-world experience with our clients, we know that a SaaS LMS designed to address FDA validation requirements can lower the total cost of ownership, while also providing predictability of costs over time. Further, SaaS LMSs have been shown to provide reliable access to data anywhere, anytime via a web browser, to accommodate remote employee usage, as well as non-employee training that spans suppliers, clinical sites, vendors, agents and contractors. The critical question that Quality Assurance executives can ask during the evaluation phase is, What will our costs be when the software is upgraded in Years 2 through 5? When the IT team and the validation team estimate the time required of server-based resources, they will be able to calculate the opportunity costs for annual upgrades. Factoring in IT and validation resource costs, as well as the hardware and software maintenance costs related to the server-based LMS, are at the heart of the growth of the SaaS LMS within the FDA-regulated organization. Page 7

About UL Compliance to Performance UL Compliance to Performance provides knowledge and expertise that empowers Life Sciences organizations globally to accelerate growth and move from compliance to performance. Our solutions help companies enter new markets, manage compliance, optimize quality and elevate performance by supporting processes at every stage of a company s evolution. UL provides a powerful combination of advisory solutions with a strong modular SaaS backbone that features ComplianceWire, our award-winning learning and performance platform. UL is a premier global independent safety science company that has championed progress for 120 years. It s more than 12,000 professionals are guided by the UL mission to promote safe working and living environments for all people. 202 Carnegie Center Suite 301 Princeton, NJ 08540 609.627.5300 UL and the UL logo are trademarks of UL LLC 2016. ULComplianceToPerformance.com WP/16/113016/SAAS