“Risk Analysis” is a big topic in pharmaceutical and biotech product development. The International Conference on Harmonization (ICH) has even issued a guidance document on Risk Analysis (Q9). Despite the documentation available and the regulatory emphasis, these tools remain poorly understood. They are used to justify limiting the extent of fundamental understanding that can be gained on a process, while simultaneously used as a cure-all for management challenges that we face in pharmaceutical process development.
Risk analysis focuses only on failure modes. Failure mode effect and analysis (FMEA) was developed by the military, first published as MIL-P-1629, “Procedures for Performing a Failure Mode, Effects and Criticality Analysis” in November 1949. The procedure was established to discern the effects of system failure on mission success and personnel safety. Since then, we have found a much broader range of applications for this type of analysis. The methodology is used in the manufacture of airplanes, automobiles, software and elsewhere. People have devised “design” FMEAs and “process” FMEAs (DFMEA and PFMEA). These are all great tools, and help us to design and build better, safer products with better, safer processes.
Where Risk Analysis Falls Down
The FMEA is such a great hammer, it can make everything look like a nail. And when regulatory authorities are encouraging companies to use Risk Analysis for product design and process validation, the temptation to apply it further can be overwhelming. In particular, risk analysis is used in three inappropriate ways, in my estimation:
- Decision analysis
- Project management
- Work avoidance
Quite often, risk analysis tools are used to guide decisions. Here, the pros and cons of selecting a particular path are weighed, using the three criteria of risk analysis (occurrence, detectability and severity) as guides. However, not every decision path leads to a failure mode or an outcome that could be measured as a consequence. And detectability and occurrence may not be the only or most appropriate factors by which to weight a consequence. There are excellent decision tools that are designed specifically for weighting and evaluating preference criteria. A very simple tool is the Kepner Tregoe (KT) decision matrix. Decision analysis uses very detailed descriptions of the decision to model the potential outcomes. The KT decision matrix is sufficient for many decisions, large and small. But if you really want to study the anatomy of a decision, decision analysis is the most satisfactory method.
FMEAs are sometimes inappropriately applied in project management, to assign prioritization and order in which tasks should be completed. This may be handy in some instances, but is somewhat misleading or inappropriate in others. The riskiness of an objective should not be the sole determinant in its prioritization. Quite often, the fundamentals or building blocks need to be in place in order to best address the riskiest proposition in a project. Prematurely addressing the pieces of a project that present the greatest risk of failure may lead to that failure. On the other hand, being “fast to fail” and eliminate projects that might not bear fruit with the least amount of resources spent, is critical to overall company or project success. Project management requires consideration of failure modes, but also resource programming and timeline management. FMEAs can be an element of that formula, but should not be the focus.
Finally, and perhaps most painfully, FMEAs are used to justify avoiding work. Too often, risk analysis is applied to a problem, not to identify the elements that are most deserving of attention, but to justify neglecting areas that do not rank sufficiently high in the risk matrix. Sometimes, the smallest risks are the easiest to address, and in addressing them, variability can be removed from the process. Variability is the “elephant in the room” when it comes to pharmaceutical quality, as has been concisely pointed out by Johnston and Zhang.
The FMEA is a stellar tool, and it is applicable to problems in design, process and strategy across many industries. Its quantitative feel makes practitioners feel as though they are actually measuring something, and can make fine distinctions between risks that they were unable to articulate before. However, the FMEA can be applied rather too widely, or sometimes unscrupulously, yielding bad data and bad decisions.
