A Comprehensive Guide to Software Risk Assessment following CSA Approach

Article Context:

1. ISO 14971:2019 vs FDA CSA Risk Analysis
2. Risk Based Approach
3. High Process Risk in CSA

As the pace of digital transformation and automation accelerates, the need for a robust and lean risk assessment methodology for software is becoming more and more critical in the life sciences industry. Historically, FMEA, or Failure Modes and Effects Analysis, has been used as a tool in identifying, evaluating, and mitigating potential risks associated with software applications. However, with the introduction of Computer Software Assurance (CSA), a simpler and more pragmatic way of performing risk assessment is gaining prominence.

ISO 14971:2019 versus FDA CSA Risk Analysis

Conducting a risk-based analysis for CSA for GxP system software differs from conducting a risk analysis for a medical device, as outlined in the document “ISO 14971:2019 – Medical Devices – Application of Risk Management to Medical Devices”. Unlike the risks considered in ISO 14971:2019 (which pertain to medical device risks), failures of GxP system software (non-medical device software) do not occur in a probabilistic manner. In other words, there is no means of estimating the likelihood of occurrence based on historical data or modelling. Instead, the risk assessment for non-medical device software considers factors that could affect or hinder the software's intended performance, such as proper system configuration and management, system security, data storage, data transfer, or operational errors.

Therefore, when conducting a risk-based analysis for non-medical device software, it's essential to consider which failures are reasonably foreseeable (as opposed to likely), as well as the associated risks stemming from each failure. The FDA’s CSA guidance addresses both process risks, which relate to potential compromises in the production or quality system, and medical device risks, which involve the potential for a device to cause harm to the patient or user. When discussing medical device risks, CSA guidance focuses primarily on the risk to the medical device resulting from a quality issue that could compromise safety.

The risks inherent in utilizing software within production or the quality system range across a spectrum from high to low risk. It is the responsibility of manufacturers to assess the risk level of each software feature, function, or operation within this spectrum, aligning their evaluation with the intended purpose of the software.

When should I Implement the Risk-Based Approach?

Whether it’s the FDA CSA’s guidance, GAMP 2nd Edition, or other literature for software validation and assurance, we hear the term “Risk-Based Approach” thrown out at nauseum. Yes, we understand that identification, assessment, and mitigations of risks are important for our GxP systems, but when are CSA Risk Assessments best utilized?

A CSA Risk-based approach is best employed at three different levels within the Computer System Lifecycle: At the System-Level, at the Requirements or Feature-level, and at the Change Management level.

The System-level risk assessment is one of the first deliverables that is procured by validation in the system lifecycle. This assessment is critical to understanding whether or not the system is GxP (meaning it must be validated for its intended use due to FDA predicate rules) and what the overall system risk profile is. Many auditors expect this assessment not only to categorize which systems require software assurance activities, but also to drive the rigor of validation. At the system-level, best practice is to have software assurance “tracks” based on the system risk. For example, a high-risk system may require the full gamut of SDLC deliverables and be periodically reviewed more frequently, whereas a low-risk system may utilize a streamlined assurance package and not have to be periodically reviewed at all.

After a manufacturer has identified a software feature, function, or operation intended for integration into production or the quality system, it is advisable to conduct a risk-based analysis on the requirements or features to determine suitable test assurance activities. This approach involves pinpointing foreseeable software failures; assessing whether they pose significant process risks; and methodically choosing and executing assurance activities proportionate to the risk associated with patient safety, product quality and data integrity. One of the most common ways this is typically assessed is through an FMEA-like analysis: identifying severity, probability, and detectability.

There are some drawbacks in this method, namely the inherent subjectivity. How do you decide if a software failure is 0.01% probable versus 1% probable? How is a given software failure a level 1 in detectability versus a level 4? The CSA methodology for a requirement’s risk assessment instead correlates risk probability with function complexity. The reasoning for this is that a requirement that is met out-of-the-box is more widely available than a requirement configured or customized for your organization, and therefore, less likely to have a bug or defect. Based on the impact-level and complexity of the feature or requirement, you can estimate risk level(s) and determine necessary validation rigor. Higher-risk functionality may call for execution of robust, scripted testing, where lower-risk functionality may allow for unscripted methods, such as ad-hoc or scenario-based testing.

Change control for GxP systems may also employ a risk-based framework. Maintenance activities and workflows may differ for high-risk major releases versus lower-risk changes, patches, or configuration updates. A one-size-fits-all methodology for change management is not a lean approach to CSA, and will thwart the adoption of new technologies such as SaaS systems, AI/ML, etc.

"High Process Risk” versus “Not High Process Risk” in CSA

In the context of CSA, the FDA guidance discusses that there are “high process risks” and “not high process risks” for GxP systems. The distinction between high process risk and not high process risk is crucial in assessing potential software feature, functional, or operational impact on the patient or product. When a software element is identified as posing a high process risk, it creates a direct impact scenario, wherein failure to operate as intended could directly compromise safety, leading to an increased risk associated with the medical device itself. To illustrate, this is akin to a tightrope walker performing without a safety net: any misstep could lead to an immediate and severe consequence, directly affecting patient safety and product quality.

Conversely, when a software component is deemed not to pose a high process risk, it suggests an indirect impact scenario, wherein failure to perform as intended may disrupt processes but does not directly compromise safety. This can be likened to a complex machine with interconnected gears: while a malfunction in one gear may disrupt overall function, the consequences might not be immediate or directly attributable to a single component, though patient safety and product quality could still be impacted downstream.

In conclusion, adopting a CSA Risk-Based approach represents a pivotal step in the life sciences industry. A CSA approach to risk assessments maintains the same level of software reliability, regulatory compliance, and, most importantly, patient safety, but allows for a streamlined and efficient means of assessing software risks, creating more value for the industry. Here at Compliance Group, we are here to help your teams on their CSA Journey!

To learn more on how to improve your CSA Risk-based Strategy, contact us at sales@complianceg.com.